Comments (4)
utkarshgupta137
commented on June 20, 2024
I believe this is caused by this change.
cc: @ritchie46
from polars.
ritchie46
commented on June 20, 2024
We forked arrow for our needs. We don't want to support mutable dictionary keys we don't use. We want to keep our binary size in check.
from polars.
utkarshgupta137
commented on June 20, 2024
We forked arrow for our needs. We don't want to support mutable dictionary keys we don't use. We want to keep our binary size in check.
Ah, fair enough. We use polars most of the time, but we migrated from arrow2 to polars-arrow for converting/normalizing some JSON to arrow files.
Would you accept a PR which addresses our use case, but doesn't affect polars' performance/binary size or is the plan to eventually get rid of MutableDictionaryArray entirely?
from polars.
ritchie46
commented on June 20, 2024
But it likely will affect the binary size as it needs to compile the u16 branch in that case.
from polars.
Related Issues (20)
- exception thrown if converting arrow Table with struct and dictionary columns to polar dataframe
- converting pandas to Polars drops column if its name, when converted to string, matches another column's name
- pl.format should be clear it will return null when one of the arguments is null
- Off-by-one error when casting to Decimal with set precision
- Importing pyarrow after polars causes `SIGSEGV` HOT 4
- Polars assumes microseconds instead of reading numpy timedelta units
- Cannot create Array column containing large u64 value
- Multipling a Decimal by Int returns Int type HOT 2
- Split out `Expr.top_k` from `Expr.top_k_by`
- `pl.Datetime` `time_zone` parameter has no type or value check HOT 5
- Cast from `pl.Date` to `pl.Datetime` silently returns incorrect value when new dtype cannot hold value HOT 2
- exception thrown if converting chunked arrow Table with struct and dictionary columns to polar Dataframe
- Panic when constructing Series with dtype `Duration('ms')` with large `timedelta` objects
- Can the separator of the read csv function support regular splitting? HOT 5
- Casting float to Decimal fails silently HOT 2
- Use parquet statistics when collecting column statistics from scanned parquet
- Excessive Memory Consumption During Rolling Operations on Large DataFrames
- write_database() - Insert many rows with sql server using fast_executemany HOT 3
- fill_null doesn't support expr HOT 6
- `dt.total_nanoseconds` and `dt.total_microseconds` may overflow silently
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from polars.