koles/ya-csv

CSV parser/writer for Node.js

ya-csv's Issues

how to run this project ?

Hi,
I am new here, just downloaded your project, I would like to run it, can you please show me how ?
thanks

Last field is set as undefined if it is empty

If the last field in the csv is empty then it will be set as undefined instead of the expected ''

Example file:

id,name,specials
1,John Doe,
2,Jane Doe,Javascript
3,Jake Doe,

Expected output:

{ id: '1', name: 'John Doe', specials: '' }
{ id: '2', name: 'Jane Doe', specials: 'Javascript' }
{ id: '3', name: 'Jake Doe', specials: '' }

Received output:

{ id: '1', name: 'John Doe', specials: '' }
{ id: '2', name: 'Jane Doe', specials: 'Javascript' }
{ id: '3', name: 'Jake Doe', specials: undefined }

reader.pause() doesn't seem to work

I'm trying to pause the readstream after 500 lines but it does not seem to work :/
this is my code (in coffeescript):
reader.addListener 'data', (data) ->
lines++
console.log "passed line #{lines}",data
@Pause()
return
I expect this code to read only 1 line and than pause and it doesn't stop, am I missing smthing?

Add ability to parse strings

Right now it only parses streams or file but there is no way to parse a string (and there's no easy way to covert string to a stream either).

Writing to a file/stream breaks because writableStream does not have the setEncoding method

Hi,

the createCsvFileWriter and createCsvStreamWriter methods throw an exception in all lines that set the encoding since the streamWriter object (no longer?) has the .setEncoding method. The encoding should be defined as a part of the .write method - see http://nodejs.org/api.html#writable-stream-33.

I'm running Node 0.2.3.

Console.loged reader out put for a large file, and got "... 924 more items" instead of the other items.

It's a pretty big csv file. Instead of all the columns, a lot of the output ends with "... X more items". How do I output all of them?

var reader = csv.createCsvFileReader('FB-PageFocus-July2017-To-Sept112017.csv', {
    'separator': ',',
    'quote': '"',
    'escape': '"',
    'comment': '',
});
var writer = new csv.CsvWriter(process.stdout);
// reader.addListener('data', function(data) {
//     console.log([ data ]);
// });

reader.addListener('data', function(data) {
    console.log(data);
});

reader.addListener('error', function(e) {
    console.error('Oops!');
});

publish 0.9.3 to npm

Seems while the version in package.json was bumped and 0.9.3 was tagged it was never published to the npm.

incorrecly handled case of buffer ending before the escaped double-quote character

see also http://github.com/gooddata/GoodData-CL/issues/closed#issue/67

column

Why do you support only linux or darwin?

It works properly on windows when I force to set it up at my machines.
You don't need to miss a windows support.

Option to make escaping optional

I think it would be good to only use escaping for fields when it is required.

I have seen some CSV implementations, doing this, and unless this goes against any rule I think it's great to save some space.

How to read a csv file if it's residing in a password protected location?

In my application the csv files I needed are residing in a password protected location in the server. So how to access those files using this module?.

Improvement: Allow periods in column names for property grouping

Would be nice if it was possible to group properties into objects, e.g. like this:

reader.setColumnNames([ 'adress.street', 'adress.zip', 'adress.city']);

reader.addListener('data', function(data) {
   console.log(data.adress.street);
   console.log(data.adress.zip);   
   console.log(data.adress.city);
});

Escaped Quotes at the end of a chunk are incorrectly ignored.

If the last character of a chunk is a double quote and the next character at the beginning of the next chunk is also a double quote, both characters will be ignored and the output will not output any characters.

I believe this has to do with using data.charAt(i+1) and not looking at the previous field if data.charAt(i+1) is null.

Example:

test,123,"quote""d field here"

If the stream chunk ends here:

test,123,"quote""d field here"
               ^

The output value

test,123,"quoted field here

when it should read

test,123,"quote"d field here

Add unicode support!

At the moment the parser does not support anything but the simplest ascii file format (as far as I an tell). E.g. scandinavian åäöÅÄÖ characters all comes out as 0xFFFD.

There is no clear way to close a CsvFileWriter

After creating a CsvFileWriter and writing data to file there is no documented way to close the writer, resulting in a file descriptor leak. I've managed to close it by using writer.writeStream.end().

Example:

    writer = csv.createCsvFileWriter('file.csv', {'flags': 'a'});
    var csvData = new Array();
    csvData.push(id);
    csvData.push(name);
    writer.writeRecord(csvData);
    writer.writeStream.end(); // this is how to close the file descriptor

Support writing object records

Hi,

The CsvWriter object should support two additional options:
a) an option that defines the column names of objects to be written with writeRecord (e.g. { columnNames : ["name", "age"] } ).
b) an option that defines whether the column names should be written to the output as a first line (e.g. {columnsAsHeader : true} ).

This way, CsvWrite could be used to write objects (dictionaries) instead of just arrays.

incorrectly parsed file with 3 double quotes and no end of lines

The following CSV contains two lines, each containing 4 records. However, the ya-csv parser reports only 1 line with 5 columns.

== begin ==
one,"""two"", two-and-half","three","four"
1,2,3,4
== end == (no EOL after the previous line)

Caused by two bugs:

• field starting with """ is not marked quoted
• last line without a trailing end of line marker is ignored

Thanks to zd at gooddata for reporting this.

Don't crash when passed a binary file

Now it crashes all nodejs app:

node_modules/ya-csv/lib/ya-csv.js:131
 throw new Error("separator expected after a closin
Error: separator expected after a closing quote; found 

It should handle the error gracefully.

csv files with single quotes fail

Error: separator expected after a closing quote; found C

createCsvStreamRead readStream set to undefined?

This is more of a question than an issue. This method (createCsvStreamReader) sets the readStream to undefined if no options are passed through:

csv.createCsvStreamReader = function(readStream, options) {
    if (options === undefined && typeof readStream === 'object') {
        options = readStream;
        readStream = undefined;
    }
    options = options || {};
    if (readStream) readStream.setEncoding(options.encoding || 'utf8');
    return new CsvReader(readStream, options);
};

It seem like really strange behaviour, and my reader was not receiving any events because of this. Is this actually intended behaviour?

How to catch error

Hi,

first of all, thanks for this excellent module.
Question : How error are supposed to be handled ?
The api seems to provide an 'error' event.
But even if you add an 'error' listener, there's a lot of throw new Error ("...") in the code base that makes error handling difficult.

NPM install not loading

When I try and require the parser after installing through npm I get:

Type '.help' for options.
node> require('ya-csv')
Error: Cannot find module 'ya-csv'
    at loadModule (node.js:275:15)
    at require (node.js:411:14)
    at cwdRequire (repl:27:10)
    at [object Context]:1:1
    at Interface.<anonymous> (repl:78:19)
    at Interface.emit (events:26:26)
    at Interface._ttyWrite (readline:281:12)
    at Interface.write (readline:123:27)
    at Stream.<anonymous> (repl:59:9)
    at Stream.emit (events:26:26)

perhaps an index.js problem?

NPM verifies that it's installed:

$ npm ls ya-csv
npm it worked if it ends with ok
npm cli [ 'ls', 'ya-csv' ]
npm version 0.1.26
npm config file /Users/jlarson/.npmrc
npm config file /usr/local/etc/npmrc
npm GET /
[email protected]                             =koles remote
[email protected]                             =koles remote
[email protected]                             =koles remote
[email protected]                             =koles installed latest remote
npm ok