robot_detection is a python module to detect if a given HTTP User Agent
is a web crawler. It uses the list of registered robots from
http://www.robotstxt.org: (Robots Database)