... I send out a list of most interesting libraries and apps in the "Go" section
to about 1100 subscribers.
Do you want it too?
Subscribe to "Go"
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
Polite, slim and concurrent web crawler.
Go robots.txt parser
The Go programming language
Parse robots.txt files in Go
The robots.txt exclusion protocol implementation for Go language
Antch, a fast, powerful and extensible web crawling & scraping framework for Go