opencleveland/drocer

Scraper and parser of Cleveland City Council's records and the produced text.

drocer's Issues

Improve documentation

our documentation is a hot mess.

Goals:

• [ provide brief overview of what project does;
  how someone can get involved; (audience: people who never heard of project before] -
  done at:
• [more thorough] https://github.com/opencleveland/drocer/wiki

Remove Commented Out Code in records.py

There is commented out code in records.py beginning on line 1142 down to line 1165. There actually also appears to be commented out code on lines 663 to 675. Also on lines 765 to 812.

Not sure if this is actually used in some capacity, but, if it's not, it should be removed just for general code cleanliness.

graph/visualization of how something becomes a law

nice to have.

A brief guide to city record and city council.

this may be a separate project from the website, but:

a basic guide for people for people to ask:

A one-two pager for average residents with a 9th grade reading level to understand
what type of information is in city council, here, what do terms like resolution and ordinance mean; explaining the concept of 'emergency ordinance'

http://www.clevelandcitycouncil.org/about-city-council/legislative-process
(this page above is new, and a welcome addition to the city council website).

Finish parsing departments

And clean up department-related code

Dump all the PDFS to text

convert the pdfs to plain text.

After this is done, we can begin finding patterns for particular data that we're looking for.

For some inspiration or already existing projects, see:

https://github.com/18F/doc_processing_toolkit
http://textract.readthedocs.org/en/latest/
http://www.unixuser.org/~euske/python/pdfminer

Automating the text extraction process

1] downloading the PDFs
2] parsing them to the raw text (https://github.com/opencleveland/drocer/blob/master/records.py ) in proper directories.

setting up an AWS account (or something else) to run this every so often.

update PDF archive with ISO601 file names

Thanks to Meaghan, the PDFs are now named in a standardized format (ISO 8601) (i.e. 2015-08-03).

Need to update documentation and our google drive with these files.

github's repository search isn't returning some results

sorta of meta-issue.

I searched "854-17" in the repository search at the top of the page; and no results were returned.

However, if you search "854-17" (without parenthesis) in your browser's search function; (I copy and pasted "854-17"and pasted into my browser - FF57 - search).
on
https://github.com/opencleveland/drocer/blob/master/2017/2017-07-19.txt ; several results will be returned in

Develop schema for board of control

elliot has a draft of this.