rustymarvin/line-by-line

A NodeJS module that helps you reading large text files, line by line, without buffering the files into memory.

line-by-line's Issues

Premature "end" event or is this the normal flow?

I have a 10 line txt file and I'm getting a premature "end" event when the lines are about the 25% of the file.

For example for the following lines,

A
B
C
D
E
F
G
H
I

If I add them to an array with the following code,

lr.on('line', function (line) {
    lines.push(line);
});

And I print the lines array at the "end" event I get:

['A', 'I', 'B']

Shouldn't the event fire when there are no more "line" events?

Question: Who's the maintainer?

It seems like @RustyMarvin is the original author, but @Ousterjour is the one who released line-by-line on npmjs.org.

Are you the maintainer or is @Ousterjour the maintainer?

I ask, because I'm thinking of forking this so I can give it the ability to work with stdin, and I want to fork the right repo.

pause doesn't pause really

I have this code and despite calling close, the event handler gets called again.

lr.on('line', function (line) {
   lr.pause();
   // if some condition met, emit lr.close()
   // else lr.resume()
});

"end" emitted prematurely and twice for paused last line

When pausing the "line" events after each line (and resuming asynchronously), the "end" event is fired while still processing the last line; After resuming, "end" is emitted a second time.

I made a quick look into LineByLineReader.prototype._nextLine and modified the sequence for me. It works for my case, but can't tell whether all are covered:

• comment out the if (this._end) block at line 91+
• modify check for empty _lines at 100 to read:

    if (this._lines.length === 0) {
        if (this._end) {
            this.emit('end');
        } else {
            this._readStream.resume();
        }
        return;
    }

Replace recursive process.nextTick usage with setImmediate

As per http://blog.nodejs.org/2013/03/11/node-v0-10-0-stable/

You've got a recursive process.nextTick happening in line-by-line - and we're seeing this crash node occasionally with:

(node) warning: Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral.
(node) warning: Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral.
(node) warning: Recursive process.nextTick detected. This will break in the next .
version of node. Please use setImmediate for recursive deferral.
(node) warning: Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral..........
RangeError: Maximum call stack size exceeded

More info here:
http://blog.peakji.com/node-js-process-nexttick-and-setimmediate/

And because setImmediate is not available in previous 'node' would be good to have a fallback like:

if (typeof setImmediate == 'undefined') { var setImmediate = process.nextTick; }

Great plugin - Thank You