Comments (3)
Pyshp shouldn't need 2GB of memory to load any shapefile @mawg. I saw from your SO post that you used the shapes()
method which loads all 1.2GB of shapes into memory at once. There is another iterShapes()
method which is much more memory friendly and should fix your problem. See my answer on SO: http://stackoverflow.com/a/39649506/2898490
from pyshp.
I'll leave it so people can see how you solved it if they have the same issue. Could you post the answer affirmatively on GIS StackExchange too?
from pyshp.
Pardon me for being dumb. Do you want me to post a question there and answer it? I already have one on S.O, at http://stackoverflow.com/questions/39639579/memory-error-on-large-shapefile-in-python/39639735#39639735, but not on GIS StackExchange
from pyshp.
Related Issues (20)
- Document the reason for first field always being a DeletionFlag HOT 2
- "README: Testing" section seems out of date HOT 1
- add/modify test routine for running pytest / ship test file with installation HOT 2
- pyshp 2.3 closing bytesio objects with Writer HOT 5
- Add read geojson capability HOT 1
- Support for getting encoding from .cpg files HOT 2
- Enhance combination with shapely HOT 2
- Why was Writer.save() removed? HOT 1
- How to modify the shape value HOT 1
- struct.error: unpack requires a buffer of 32 bytes HOT 2
- How to get the field name for data? HOT 1
- Inconsistent bounding box filtering HOT 1
- check if polygon is closed fails HOT 1
- Build marked as failing. Tests are not running in CI. Github /actions/setup-python no longer supports Python 2.7. HOT 1
- PyShp's own Example from README.md fails under Doctest with Python 2.7. .__geo_interface__ = behaves differently in Python 2? HOT 1
- Doc test in README.md fails due to undefined variable HOT 1
- Run tests on Python 3.10 and higher versions (as well as 2.7, 3.5, ..., 3.9)
- Interest in I/O methods interfacing geopandas objects? HOT 3
- Doctests failed in CI due to dead link (failed download) HOT 2
- Find a better url to a zipped shapefile. Replace the current url on Github with a url external to Github, to guarantee test integrity.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyshp.