b1tninja/misty

Mystique Community Organization python musings

misty

Mystical Communal Promulgations... mulgare! Capitulate! Lactate! Promulgate!

YouTube Demos

• misty - text to speech helper Ms. Ty
• query - whoosh full-text searching of Tesseract-OCR'd PDFs/TXTs/JPGs
• definitions.py - Synonyms, Antonyms, Definitions, See: recursive
• ud.py - UrbanDictionary
• ca.py - California Code Reader (pubinfo_*.zip dat/lob parser)
• sacrec.py - Sacramento County Clerk Recorder Index Synchronizer
• sacass.py - Sacramento Assessors Parcel data downloader

California Codes

ca.py is a California codes reader, capable of full text search and plain text printing of the pubinfo_2021.zip.

The plain-text capabilities come from Aaron Swartz 's html2text library.

Read more about the legislatures "pubinfo" format here, and my opinions about edicts of government on leginfo feedback page.

Sacramento County

sacrec.py is a Sacramento County Clerk Recorder Public Index dumper. It crawls the county's website by filing code and saves the responses.

sacrec_tsv.py takes those .json files eliminates any duplicates, sorts by PrimaryDocNumber and then saves to a ..csv. The lod2csv function is pretty handy way to convert a list of dictionaries into a csv.

sacass.py downlaods the jsons from the Sacramento Coutny Assessors Parcel Viewer. The APNs are obtaind from Address.csv.

Examples

wget -r -c -np -l 1 -A zip downloads.leginfo.legislature.ca.gov
python3 ca.py

Setup

Check out from Github

git clone https://github.com/b1tninja/misty --single-branch --depth 1

Ubuntu

sudo apt update
sudo apt install python3-pip

virtualenv for python (optional)

Unfamiliar with Python Virtual Environments? Read the venv docs.

Install virtualenv for python3

sudo apt install python3-virtualenv

or (but not both)

sudo pip3 --install virtualenv

Create a new virtualenv

virtualenvs are sort of user copies of a python environment to install the optional dependencies in.

cd misty
python3 -m virtualenv venv

Activate the "virtualenv"

This changes which python/pip binaries are used, so you don't need to modify your (whole-)system python environment

. venv/bin/activate

Install optional dependencies

Using requirements.txt

pip3 install -r requirements.txt

or pick and choose

pip3 install html2text
pip3 install tqdm

misty's Issues

ca.py - use on linux

Hello,

I saw your feedback on CA's legislative website. Agree 100% their documentation is severely lacking and access to the statutes in a human readable format is not acceptable. I thank you for creating the ca.py script, however, I am having trouble using it in Linux. Any suggestions? Thanks