gkbps/line-by-line

A NodeJS module that helps you reading large text files, line by line, without buffering the files into memory.

Line By Line

is a NodeJS module that helps you reading large text files, line by line, without buffering the files into memory.

Installation:

npm install line-by-line

Usage:

Synchronous processing of lines:

var LineByLineReader = require('line-by-line'),
    lr = new LineByLineReader('big_file.txt');

lr.on('error', function (err) {
	// 'err' contains error object
});

lr.on('line', function (line) {
	// 'line' contains the current line without the trailing newline character.
});

lr.on('end', function () {
	// All lines are read, file is closed now.
});

Asynchronous processing of lines:

var LineByLineReader = require('line-by-line'),
    lr = new LineByLineReader('big_file.txt');

lr.on('error', function (err) {
	// 'err' contains error object
});

lr.on('line', function (line) {
	// pause emitting of lines...
	lr.pause();

	// ...do your asynchronous line processing..
	setTimeout(function () {

		// ...and continue emitting lines.
		lr.resume();
	}, 100);
});

lr.on('end', function () {
	// All lines are read, file is closed now.
});

Initialize with Stream:

var LineByLineReader = require('line-by-line'),
    lr = new LineByLineReader(S3.getObject({ Bucket, Key }).createReadStream());

API:

Class: LineReader(path [, options])

path specifies the file to read or Stream

options is an object with the following defaults:

{ encoding: 'utf8',
  skipEmptyLines: false }

encoding can be 'utf8', 'ascii', or 'base64'.

If skipEmptyLines set to true, empty lines don't trigger a 'line' event.

You can also pass start and end position to read from file region:

{ encoding: 'utf8',
  skipEmptyLines: true,
  start: 1000 }

Event: 'line'

function (line) { }

Emitted on every line read.

line contains the line without the line ending character.

Event: 'error'

function (error) { }

Emitted if an error occurred.

error contains the error object.

Event: 'end'

function () { }

Emitted if all lines are read.

pause()

Call this method to stop emitting 'line' events.

resume()

After calling this method, 'line' events gets emitted again.

close()

Stops emitting 'line' events, closes the file and emits the 'end' event.

License:

The MIT License (MIT)

Copyright © 2012 Markus von der Wehd