parsyl / parquet Goto Github PK
View Code? Open in Web Editor NEWA library for reading and writing parquet files.
License: MIT License
A library for reading and writing parquet files.
License: MIT License
The current implementation misses gzip compression
This library does not support the use of the latest apache thrift library as they have broken their API by introducing new arguments into their exported functions. That event makes it impossible to install this library with go get
because the older version is not defaulted to. By adding the specific required version (or by updating this library to be compatible while also still adding these versions to the go.mod) this library will become immediately usable again.
Hi, I have a use-case where I'm writing billions of data entries into a file. I couldn't find any information regarding this in the README, so I wanted to ask whether it's safe to use *ParquetWriter
across multiple goroutines to speed up the whole process and if you have any recommendations for doing that. I'm not quite sure but maybe Add
could be called concurrently and every once in a while Write
should be called depending on the size of RowGroup.
For now only flat parquet schemas are working correctly.
The code that was used to generate the code for writing values to the input struct needs to be brought back for cases when a field is not repeated.
In Apache Thrift 0.9.3 to 0.13.0, malicious RPC clients could send short messages which would result in a large memory allocation, potentially leading to denial of service.
Fix is in 0.14.0
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.