Scrapy, a fast high-level web crawling & scraping framework for Python.
<!DOCTYPE html> <html> <head> <title>Scrapy test site</title> </head> <body> <h1>Scrapy test site</h1> <ul> <li><a href="item1.html">Item 1</a></li> <li><a href="item2.html">Item 2</a></li> <li><a href="item999.html">Item 999 (not found)</a></li> </ul> </body> </html>