Libraries and scripts for crawling the TYPO3 page tree. Used for re-caching, re-indexing, publishing applications etc.
A PHP flexible web crawler that can login into a website.
A simple web crawler in php to run through the links of a given URL recursively
A simple crawler (spider) writen in php just for fun, with zero dependencies
A php crawler that finds emails on the internets
The source code from the Web Crawler tutorial series.
Web Crawler - with email/link scraping and proxy support