100 days of Code - Section 45 ...
scraping from your own html site (simple example) Scraping from ycomb site scraping from top 100 billboard site
use url.robots.txt to view what the website is allowing robots to scrape. User-agent : the person scraping Crawl-dealy : how often your script should scrape