`list_concat([list<T>, list<T>])` gives `list<T>`, not `list<list<T>>`

[NickCrews]

`list_concat([list, list])` gives `list`, not `list>`

from polars.

[mcrumiller]

@ritchie46 I think this is a valid issue. Why do the following two produce the same result?

pl.DataFrame({"a": [1], "b": [2]}).select(
    pl.concat_list("a", "b")
)
# shape: (1, 1)
# ┌───────────┐
# │ a         │
# │ ---       │
# │ list[i64] │
# ╞═══════════╡
# │ [1, 2]    │
# └───────────┘

pl.DataFrame({"a": [[1]], "b": [[2]]}).select(  # note the [[1]]
    pl.concat_list("a", "b")
)
# shape: (1, 1)
# ┌───────────┐
# │ a         │
# │ ---       │
# │ list[i64] │
# ╞═══════════╡
# │ [1, 2]    │
# └───────────┘

The documentation says "Horizontally concatenate columns into a single list column." If the columns are lists, then they should form a list of lists.

Edit: I just noticed there is more to the documentation in 1.0 which isn't on the current 0.2 documentation, that clarifies.

[ritchie46]

It is a concat list operation. It concats the lists. If you want to create an extra level of nesting you need to implode() the arguments.

[mcrumiller]

FYI I feel that pl.list would be a better name if the intended behavior is what's written in the description. I agree that the word "concat" implies that you take existing lists and concatenate them together, but I believe the intent is to take existing columns and concanate them into a list, in which case the inner dtype of the list should be common supertype of the columns.

[ritchie46]

The initial implosion is because there isn't any list yet. So in order to concat them, they are imploded.

I think we should improve the description here.

[mcrumiller]

@ritchie46 the description's improved in the 1.0 docs, I had missed that.

[NickCrews]

@mcrumiller pinpoints my exact confusion. I agree there should be two functions, one that is a constructor as in
Iterable[T] -> List[T] , and another which concatenates Iterable[List[T]] -> List[T]

[NickCrews]

I'm looking for the constructor. What could I use for that currently?

[mcrumiller]

Hi @NickCrews, can you be a little more specific about what you want to do?

[NickCrews]

Yes, but I'll get to it in an hour or two once I can get back to my computer

[NickCrews]

Actually, I think we should move all discussion to #8510.

Specifically, I responded in this comment in that issue.