It is a Web Crawler optimized to crawl live articles from New York Times website, it is optimized to extract the article text and remove any unnecessary content such as adverstisement or other text snippets.
This Crawler is an extention to an implementation by James Mills From: http://code.activestate.com/recipes/576551-simple-web-crawler/