h2b7/ranker_writer

Parsing the ranker website content (writers json file)

Script to get a json key_tree by key and saving the content of the key by key_tree

• yet another parser (not searched the www for scripts that may solve my task)

{"a": 1, "b": 2, "d": {"c": 3}}		# 'root -> d -> c'
{"a": 1, "b": 2, "d": [{"c": 3}]}	# 'root -> d -> c'

Dependencies

pip -V		# 22.1.1
python -V	# 3.10.5
pytest -V	# 6.2.5

Start

python -m venv env && source env/bin/activate
pip install -r requirements.txt

# check functions if necessary
pytest .

# run script
python src/run.py

Script structure

.
├── src
│   ├── __init__.py
│   └── run.py
├── tests
│   ├── __init__.py
│   ├── test_get_page_data.py
│   └── test_page_data_tree.py
├── data
│   ├── key_content-ignore_me.json
│   └── ranker_writer-ignore_me.json
├── README.md
├── CONTRIBUTORS.md
└── requirements.txt

TODO

• nested json
• json in the list
• check for multiple keys
  • return multiple keys (iterable result)
  • unique multiple keys (not every single item in the list)
• check for keys by value
• access to the data in the list
  • added default index = 0
  • add and get the index from key_tree
• handle errors on searching for a non string key
• handle errors on request and parsing
  • check for no url
• fix errors on reading and writing to the json file without filename
  • add tests

Coding process: https://youtu.be/DkBAIKMN7x0

ranker_writer's Issues

Similar projects (scripts) to look at

• https://github.com/venthur/python-gron

>>> cat tests/data/one.json | gron
< json = {};
< json.abool = true;

• https://github.com/mazen160/gronpy

>>> cat test.json | gron | grep language
< [glossary][GlossDiv][GlossList][GlossEntry][GlossDef][para]->A meta-markup language

• https://github.com/Prunoideae/json-grep

>>> cat numbers.txt | jg "{\"id\":\"^2.*$\",\"attr.*\":{\"gender\":\"female\"}}"
< [
< 	{
< 		"id": 2001,
< 		"phone": 113,
< 		"attributes": {
< 			"gender:": "female"
< 		}
< 	}
< ]

• https://github.com/ploubser/JSON-Grep

>>> cat my.json | jgrep "[foo.bar1=1 and foo.bar2=2]"
< [
< 	{
< 		"foo": [
< 			{
< 				"bar1": 1
< 			},
< 			{
< 				"bar2": 2
< 			}
< 		]
< 	}
< ]

• https://github.com/matthiasdiener/ripjson

>>> rj '.*es.*/cit' test.json
< address/city = "London"