Comments (10)
@ritchie46 I think this is a valid issue. Why do the following two produce the same result?
pl.DataFrame({"a": [1], "b": [2]}).select(
pl.concat_list("a", "b")
)
# shape: (1, 1)
# ┌───────────┐
# │ a │
# │ --- │
# │ list[i64] │
# ╞═══════════╡
# │ [1, 2] │
# └───────────┘
pl.DataFrame({"a": [[1]], "b": [[2]]}).select( # note the [[1]]
pl.concat_list("a", "b")
)
# shape: (1, 1)
# ┌───────────┐
# │ a │
# │ --- │
# │ list[i64] │
# ╞═══════════╡
# │ [1, 2] │
# └───────────┘
The documentation says "Horizontally concatenate columns into a single list column." If the columns are lists, then they should form a list of lists.
Edit: I just noticed there is more to the documentation in 1.0 which isn't on the current 0.2 documentation, that clarifies.
from polars.
It is a concat list operation. It concats the lists. If you want to create an extra level of nesting you need to implode()
the arguments.
from polars.
FYI I feel that pl.list
would be a better name if the intended behavior is what's written in the description. I agree that the word "concat" implies that you take existing lists and concatenate them together, but I believe the intent is to take existing columns and concanate them into a list, in which case the inner dtype of the list should be common supertype of the columns.
from polars.
The initial implosion is because there isn't any list yet. So in order to concat them, they are imploded.
I think we should improve the description here.
from polars.
@ritchie46 the description's improved in the 1.0 docs, I had missed that.
from polars.
@mcrumiller pinpoints my exact confusion. I agree there should be two functions, one that is a constructor as in
Iterable[T] -> List[T] , and another which concatenates Iterable[List[T]] -> List[T]
from polars.
I'm looking for the constructor. What could I use for that currently?
from polars.
Hi @NickCrews, can you be a little more specific about what you want to do?
from polars.
Yes, but I'll get to it in an hour or two once I can get back to my computer
from polars.
Actually, I think we should move all discussion to #8510.
Specifically, I responded in this comment in that issue.
from polars.
Related Issues (20)
- `test_read_database_cx_credentials` expected exception does not bifurcate over correct Python version HOT 1
- In `scan_parquet()`, `include_file_paths` returns twice the same column
- In `bin.size()`, the `unit` parameter is not described HOT 1
- `.implode().list` method is not valid before `.over`. HOT 3
- Upgrading to 0.42 breaks compilation HOT 4
- Expr `.implode().get().over()` has strange results. HOT 3
- upsample not working if arguments are only sorted within group HOT 1
- Behaviour change of Expr.list.drop_nulls for structs with all None fields HOT 1
- `.first` behind `.sort_by.slice` gets wrong result.
- `Expr.shuffle` uses different order per column HOT 2
- `.struct.field()` after `shuffle()` seems to produce incorrect results
- Multi-output, multi-sink lazy polars HOT 3
- Inconsistent Results Between Pandas and Polars using cut (and qcut)? HOT 2
- ExprStringNameSpace replace / replace_all literal flag ignored for dataframes with multiple rows
- `dt.round()` slow/fast path use different rounding HOT 2
- write_parquet with partition_by silently overwrites existing files HOT 7
- rank() on a Series of just 1 null assigns rank=1 to the null value.
- read_ndjson ignores provided schema list inner types if values are inferred null
- Dependencies not recognized
- Inconsistent rolling results when using temporal windows
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
>`
from polars.