sunlightlabs/regulations-scraper

Scraper of public comments on regulations.gov

regulations-scraper

This repo contains scraper code for maintaining a complete copy of all data on Regulations.gov (consisting mainly of Federal Register documents and public comments), extracting text from said documents, and doing named entity recognition (using Oxtail) and plagiarism detection/clustering (using cluster-explorer). Additionally, the project includes scrapers for a couple non-participating agencies, the SEC and CFTC, and shoehorns their content into the Regulations.gov data model.

regulations-scraper's Issues

need to scrape comments from regulations.gov using API

Hi there.

New to Github. Need to figure out how to write code to scrape comments off of regulations.gov with the new API, and then off of multiple web pages (though I think that is implied). Experienced researcher, new to coding though. Need help.

This, specifically, is the link to the comments I'm working with.

D