Data in csv files with less columns than schema shifts data.

[SiggyF]

Data in csv files with less columns than schema shifts data.

[ritchie46]

0.20.31
shape: (1, 3)
┌─────┬─────┬──────┐
│ a   ┆ b   ┆ c    │
│ --- ┆ --- ┆ ---  │
│ str ┆ str ┆ str  │
╞═════╪═════╪══════╡
│ a   ┆null ┆   c  │
└─────┴─────┴──────┘

This is not expected behavior.

I see a: a, b: c, c: None/mising

[SiggyF]

If the csv file has the following content:

a;c
a;c

Would you not expect column b to be empty and column c to contain c?

[ritchie46]

When you provide a schema it is position based. So every field is assigned to a column by the position on the line.

position 0 is value a goes to position 0 in the schema, which is column a
position 1 is value c goes to position 1 in the schema, which is column b
depleted line; fill remaining fields with null as they are missing on that line.

[SiggyF]

Thanks for the information. I thought we would end up in this section of the code:

polars/crates/polars-io/src/csv/read/parser.rs

Line 571 in 4d35be2

polars_bail!(ComputeError: r#"found more fields than defined in 'Schema'