jeffreyshen19/billscraper.js

A Javascript module to scrape, analyze, and cache Congressional bills

billscraper.js's Issues

Add more unit tests to fix code coverage

@malcolmessaid

Add more examples of searching to searchbill.js

Update the documentation in examples/searchbill.js to show more examples of how to use mongoose to search for specific queries.

Write unit tests for printBill.js

Create a file called printBill.test.js in test/parser. Add unit tests in a similar manner as cleanBill.js (just changed for printBill).

Add a linter to this library

It would be nice to add a linter to this project, so there isn't any inconsistency in code styling. For instance, having a linter to make sure require(path/to/file.js) becomes require(path/to/file) (see #9).

Add unit tests for cleanBill and generateDB.

Look at past unit tests for inspiration @malcolmessaid.

Add a CC0 license badge to the top of the README

Add "author" field to billStruct that contains the original author of the bill

Deprecation warning

When using the library, the following will be logged to the console:

(node:36783) DeprecationWarning: sys is deprecated. Use util instead.

Refactor index.js

Index.js looks really bad right now. Help me refactor it

Check inputs for generateDb.js

In generateDB.js (locating in lib/database), check that type is one of the items in validBillTypes, congress is either 113, 114, or 115, and session is either one or two. If these cases aren't true, throw a TypeError with a helpful message.

When throwing errors, make sure to pass the relevant file and line number

Errors can be thrown as following:

throw new Error(message, file name, line number);

However, some of the errors are thrown as just throw new Error(message). Add a file name and line number to all of these.

getBill.js will return an undefined object because of GPO website's rate limiting.

After accessing the GPO website repeatedly within a short period of time, it will return a null page, which will cause the following error to be thrown. While this can't be fixed (it is an issue on the GPO side), it would be nice to have some error handling for it.

Improve code coverage

Code coverage is currently sitting at a meager 57%. Fix this by adding more unit tests to cover more lines of code.

Add a save option to getBill to automatically save it to the database

generateDB does not work with sjres

Remove all ".js" extensions when using require

Replace all instances of require("path/to/file.js") with require("path/to/file").

tagBill.js returns some useless tags.

The tagBill function (located at lib/parser/tagBill.js) is designed to take in bill text, and output a relevant array of "tags" for the bill. However, I don't want this function to return tags like "paragraph" or "congress" or "america" because these are pretty useless tags. Therefore, I have an array called wordsToExcludeFromTags (see below), where all these words I want to ignore are included. Help me add more words to this array.

Exclude "examples" folder from code coverage report.

There is a folder "examples" that contains example javascript files. However, I don't want these to be included in my coverage report. Help me exclude them.

Add an option to not grab billText or not grab billStatus in createBill.js

Update the docs on gh-pages

Clone the latest version of the branch, then run the following commands:

git checkout gh-pages
git rebase master
npm run docs
git add .
git commit -m "Updated docs"
git push origin gh-pages

Submit a PR to the gh-pages branch

Make maxTags work

In tagBill.js, there is an integer option maxTags, intended to limit the size of the array tags to a size less than or equal to maxTags. It doesn't yet work, so implement the functionality to limit the size of tags. Add code right before the callback on l.56 in lib/parser/tagBill.js

Fix the styling on VALID_BILL_TYPES.md in docs

Fix the markdown styling in VALID_BILL_TYPES (one of the line breaks isn't working), and add documentation using billScraper.validBillTypes to get an array of valid bill types.

Add an array field to index.js

Add an array field called validCongresses to index.js that contains the integer entries 113, 114, and 115.

getBill.js will occasionally throw "getaddrinfo ENOTFOUND"

Clean up chamber field in lib/database/models/Bill.js

Currently, the chamber field can be either "IN THE SENATE OF THE UNITED STATES" or "IN THE HOUSE OF REPRESENTATIVES". I want these to be converted into "senate" and "house", respectively.