rustymarvin/line-by-line

A NodeJS module that helps you reading large text files, line by line, without buffering the files into memory.

line-by-line's Issues

Question: Who's the maintainer?

It seems like @RustyMarvin is the original author, but @Ousterjour is the one who released line-by-line on npmjs.org.

Are you the maintainer or is @Ousterjour the maintainer?

I ask, because I'm thinking of forking this so I can give it the ability to work with stdin, and I want to fork the right repo.

"end" emitted prematurely and twice for paused last line

When pausing the "line" events after each line (and resuming asynchronously), the "end" event is fired while still processing the last line; After resuming, "end" is emitted a second time.

I made a quick look into LineByLineReader.prototype._nextLine and modified the sequence for me. It works for my case, but can't tell whether all are covered:

• comment out the if (this._end) block at line 91+
• modify check for empty _lines at 100 to read:

    if (this._lines.length === 0) {
        if (this._end) {
            this.emit('end');
        } else {
            this._readStream.resume();
        }
        return;
    }

Premature "end" event or is this the normal flow?

I have a 10 line txt file and I'm getting a premature "end" event when the lines are about the 25% of the file.

For example for the following lines,

A
B
C
D
E
F
G
H
I

If I add them to an array with the following code,

lr.on('line', function (line) {
    lines.push(line);
});

And I print the lines array at the "end" event I get:

['A', 'I', 'B']

Shouldn't the event fire when there are no more "line" events?

Replace recursive process.nextTick usage with setImmediate

As per http://blog.nodejs.org/2013/03/11/node-v0-10-0-stable/

You've got a recursive process.nextTick happening in line-by-line - and we're seeing this crash node occasionally with:

(node) warning: Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral.
(node) warning: Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral.
(node) warning: Recursive process.nextTick detected. This will break in the next .
version of node. Please use setImmediate for recursive deferral.
(node) warning: Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral..........
RangeError: Maximum call stack size exceeded

More info here:
http://blog.peakji.com/node-js-process-nexttick-and-setimmediate/

And because setImmediate is not available in previous 'node' would be good to have a fallback like:

if (typeof setImmediate == 'undefined') { var setImmediate = process.nextTick; }

Great plugin - Thank You

pause doesn't pause really

I have this code and despite calling close, the event handler gets called again.

lr.on('line', function (line) {
   lr.pause();
   // if some condition met, emit lr.close()
   // else lr.resume()
});