Government Corpus and Scrapers
A collection of government texts and scrapers to get them.
UCSB American Presidency Project
The best collection of documents relating to the USA Presidency that I could find is hosted by the UCSB American Presidency Project (APP). They have a great search tool, but I could not find a way to bulk download the documents to text files, thus the scraper.
Description from the APP site as of 2018-04-27:
The American Presidency Project (APP), non-profit and non-partisan, is the leading source of presidential documents on the internet. Our archives contain 129,483 documents and are growing rapidly.
The APP, hosted at the University of California, Santa Barbara, has been a collaboration between John T. Woolley (UCSB) and Gerhard Peters (Citrus College) since 1999.
The site includes the full text of executive orders, presidential debates, and much more. Checked boxes mean the scraper can get those documents.
- Messages and Papers of the Presidents
- Public Papers of the Presidents
- Public Papers and Addresses of Franklin D. Roosevelt
- Weekly Compilation of Presidential Documents
- Daily Compilation of Presidential Documents
- Annual Messages to Congress on the State of the Union
- Inaugural Addresses
- Farewell Addresses
- The President's Weekly Addresses
- Fireside Chats
- News Conferences
- Executive Orders
- Proclamations
- Presidential Signing Statements
- The Economic Report of the President
- Statements of Administration Policy
- White House Media Pool Reports
- Remarks by the Vice President
- Remarks and Statements by the First Lady
- Presidential Nomination Acceptance Speeches and Addresses
- Presidential Candidates Debates
- Documents Related to Presidential Elections
- Documents Related to the 2000 Election Dispute
- Documents Related to Presidential Transitions
- National Political Party Platforms
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent