Does not parse multiple lines in a column separated with '\n'

[ben-strasser]

Does not parse multiple lines in a column separated with '\n'

[niranjan92]

Any updates on this issue?

Regards,

[ben-strasser]

Hi,

this problem already appeared previously and I decided to not fix it at that
time. The reasons were:

• CSV files that contain newlines within a line that do not separate lines are
  defective in my opinion. The first step should consist of fixing the file.
• Splitting up the lines is done in LineReader. Unescaping quotes is done in
  CSVReader but only if the right policies are set. LineReader knows nothing
  about quotes and thus can not know whether a newlines is within quotes or not.
  The whole quoting logic is contained within a CSVReader policy. I therefore
  see no straightforward way of making this "just work" without major interface
  breaking changes to the design.
• The most frequent use case consists of parsing CSV-files that only contain
  numbers and perhaps non-escaped strings. I do not want to add logic to
  LineReader that makes this use case slower.
• Windows vs Linux newlines, i.e., "\r\n" vs "\n", make this whole problem
  significantly worse. Note that, these two can appear simultaneously within
  one file. Sidenote: This whole Linux vs Windows is also the reason why the
  newline characters are hardcoded within LineReader and not part of a
  template parameter, as they should in an ideal world.

If you know of a good way to support such ill-formed CSV files, then I can add
the change. If not, then this problem will stay.

Best Regards,
Ben Strasser

On Sat, 06 Jun 2015 12:06:31 -0700
Niranjan Godbole [email protected] wrote:

Any updates on this issue?

Regards,

---

Reply to this email directly or view it on GitHub:
#2 (comment)