Libraries and scripts for crawling the TYPO3 page tree. Used for re-caching, re-indexing, publishing applications etc.
A simple crawler (spider) writen in php just for fun, with zero dependencies
Web Crawler - with email/link scraping and proxy support
A PHP flexible web crawler that can login into a website.
A simple web crawler in php to run through the links of a given URL recursively
The source code from the Web Crawler tutorial series.
PHP library providing functionality to verify that user-agents are who they claim to be.
A php crawler that finds emails on the internets