underchemist / geojsplit Goto Github PK
View Code? Open in Web Editor NEWA python implementation of the node package geojsplit: https://github.com/woodb/geojsplit
License: MIT License
A python implementation of the node package geojsplit: https://github.com/woodb/geojsplit
License: MIT License
A warning is raised when using the geojsplit package. This happens when using a file as input.
Example of code raising the warning :
geojson = geojsplit.GeoJSONBatchStreamer(file_path)
Python==3.9.7
geojsplit==0.1.2
ijson=2.6.1
ijson/compat.py:48:DeprecationWarning:
ijson works by reading bytes, but a string reader has been given instead. This
probably, but not necessarily, means a file-like object has been opened in text
mode ('t') rather than binary mode ('b').
An automatic conversion is being performed on the fly to continue, but on the
other hand this creates unnecessary encoding/decoding operations that decrease
the efficiency of the system. In the future this automatic conversion will be
removed, and users will receive errors instead of this warning. To avoid this
problem make sure file-like objects are opened in binary mode instead of text
mode.
warnings.warn(_str_vs_bytes_warning, DeprecationWarning)
this is for another enhancement
cli runs too long, maybe multiprocessing
module can be used to improve the performance of this block
Lines 42 to 65 in 774da7b
I'm trying to run geojsplit
on https://github.com/simonepri/geo-maps/blob/master/info/earth-lands.md, the largest one.
The run seems to finish (according to verbose) but there is no output. I did not set the dry run flag or anything like that.
No errors in the terminal.
this is for an enhancement.
aside from setting a specific batch size for all splits, let users set a batch size per split file that will be generated.
revision in the main code can be something like this:
try:
page = 0 # nth fragment
origBatch = batch
while True:
data: List[Dict[str, Any]] = []
batch = origBatch
if batches is not None and page < len(batches):
batch = batches[page]
page += 1
for _ in range(batch):
data.append(next(features))
yield geojson.FeatureCollection(data)
except StopIteration:
originally, my aim is to have a file size limit per split... but this can do for now.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.