Regression 0.20.15->0.20.16: ComputeError: conversion from `null` to `struct[100]` failed in column 'literal' for 0 out of 1 values

[antonioalegria]

Regression 0.20.15->0.20.16: ComputeError: conversion from `null` to `struct[100]` failed in column 'literal' for 0 out of 1 values

[reswqa]

Thanks @antonioalegria and @cmdlineluser. This should have been an issue for some time, but type_coercion for when-then-otherwise was changed to strict_cast in 0.20.16, the culprit was revealed then. But yes, we should fix this.

[reswqa]

After some discussion, I think this should be fixed if we enable outer validity for StructChunked, see #3462.

Until then, you may need to set type_coercion=False to workaround.

[cmdlineluser]

Can reproduce the error.

On 0.20.15 I get this:

df1 = pl.DataFrame({
    "a": [1, 2, 3, 4, 5],
    "b": [[{"a": 1}], [{"a": 1}, {"a": 2}], [{"a": 1}, {"a": 2}, {"a": 3}], [], None]
})

pl.__version__

df1.with_columns(
    pl.when((pl.col("b").is_not_null()) & (pl.col("b").list.len() > 0))
      .then(pl.col("b").list.to_struct("max_width", lambda x: f"{x}", 100))
)

# '0.20.15'
# shape: (5, 2)
# ┌─────┬────────────────────────┐
# │ a   ┆ b                      │
# │ --- ┆ ---                    │
# │ i64 ┆ struct[3]              │
# ╞═════╪════════════════════════╡
# │ 1   ┆ {{1},{null},{null}}    │
# │ 2   ┆ {{1},{2},{null}}       │
# │ 3   ┆ {{1},{2},{3}}          │
# │ 4   ┆ {{null},{null},{null}} │
# │ 5   ┆ {{null},{null},{null}} │
# └─────┴────────────────────────┘

Does the .when() actually do anything in this case?

df1.with_columns(
    pl.col("b").list.to_struct("max_width", lambda x: f"{x}", 100)
)

# shape: (5, 2)
# ┌─────┬────────────────────────┐
# │ a   ┆ b                      │
# │ --- ┆ ---                    │
# │ i64 ┆ struct[3]              │
# ╞═════╪════════════════════════╡
# │ 1   ┆ {{1},{null},{null}}    │
# │ 2   ┆ {{1},{2},{null}}       │
# │ 3   ┆ {{1},{2},{3}}          │
# │ 4   ┆ {{null},{null},{null}} │
# │ 5   ┆ {{null},{null},{null}} │
# └─────┴────────────────────────┘

[antonioalegria]

Where should I set type_coercion=False?