Comments (2)
epogrebnyak
commented on July 22, 2024
manually (FIXME)-:
download zip/rar file for a specified month from Rosstat website
unpack MS Word files to a local folder
- downloader gets rar file for specified year, month from Rosstat website (2017+) or from S3 (2009-2016) to data\raw\year\month (folder not synced to github)
- unpacker will unpack MS Word files in data\raw\year\month
- word2csv makes interim CSV file in data\interim\year\month folder (saved at github)
Notes:
- MS Word files are not saved anywhere, only rar and interim CSV are saved.
- No functionality is provided to upload files at S3, saved manually there.
- Rosstat and S3 rar files are treated as immutable.
from parser-rosstat-kep.
epogrebnyak
commented on July 22, 2024
invoke add <year> <month> will do full cycle from downloading to parsing to saving to latest folder.
Remaining tasks:
- edit docs @ readme, @ rst
- (+sync readme vs rst)
- wrap unrar for linux usage
- cleaner global date/file helper
- download from S3
from parser-rosstat-kep.
Related Issues (20)
- add Vintage.upload() method HOT 1
- processed/latest folder needs better handling
- certain variables not found in Vinatage.validate() HOT 2
- review check procedure HOT 7
- Missing values should not be False at dataframe construction HOT 5
- shorter decimal representation in CSV file HOT 2
- replace Table class with Table2
- add coverable badge
- adapt code to create html with headers and charts HOT 8
- code review for `dev-sceleton` branch
- speed up manage.parse() HOT 3
- create parsing definition for 'profit' variable
- start of minimal example in julia HOT 1
- start of minimal example in go
- clean notebooks folder and dev_scrap branch
- duplicate code: get_year() vs clean year()
- why smaller code has longer running time?
- trace where duplicate values are created
- how to control warnings issue?
- industial goods production
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from parser-rosstat-kep.