jcrugzz/changes-stream

A fault tolerant changes stream for couchdb

changes-stream's Issues

Improve performance for large documents

I have created a PR (#23) to reduce delay with parsing large documents.

The current approach splits the entire document buffer after every append.
The new approach splits each new chunk before it is appended.

This takes document parse time from quadratic time down to linear.

Pass in the Changes URL (vs the DB URL)

There are situations where folks proxy the CouchDB API or otherwise only expose parts of it--such as the changes feed--into their own API space. Since this library only (afaict) works with the changes feed directly and doesn't access other database bits, it'd be helpful if I could pass in the URL directly to the feed itself.

@sballesteros can speak more to this use case. 😄

Make the stream async-iterable

Since node v10, ReadableStreams are async-iterable, i.e. one can do the following:

for await (const chunk of readable) {
    data += chunk;
}

See the docs for more details.

It would be very convenient to be able to iterate through changes this way:

const ChangesStream = require('changes-stream');

const changes = new ChangesStream({ db: '...' });
for await (const change of changes) {
  ...
}

Catching errors

Hi there,

How can I wrap stream errors with user facing error messages for instance when couch responds with something else than a 200? (specifically in my case when a filter doesn't exist on a design document).

Cheers!

Jun

Memory leak

I'm using your library to open a stream to npm's CouchDB's replication stream and want to temporarily pause the stream. When pausing there is a massive memory leak.

I check the heap size every 2.5 seconds.

Program is using 51262680 bytes of Heap.
Program is using 87967800 bytes of Heap.
Program is using 118294960 bytes of Heap.
Program is using 150143144 bytes of Heap.
Program is using 183234064 bytes of Heap.
Program is using 216274768 bytes of Heap.

i ported this to Rust, just letting you know and saying thanks!

👋 i am just saying hi and that i ported this to Rust in case ya'll were interested. thanks for a cool package! https://github.com/ashleygwilliams/changes-stream-rust

graceful shut down

hi,

what's the best practice for a graceful shut down?
im piping it to other stream handlers

ECONNRESET errors

Having running watcher for a while I've got such errors in logs:

Error: socket hang up
  at createHangUpError (_http_client.js:323:15)
  at Socket.socketCloseListener (_http_client.js:364:25)
  at Socket.emit (events.js:194:15)
  at TCP._handle.close (net.js:597:12) code: 'ECONNRESET' 

Config is totally similar to one from docs.
Any ideas to investigate, perhaps?

The HTTP option Reject Unauthorized is always set to "true"

The HTTP option Reject Unauthorized is always set to true regardless of values set for options strictSSL and rejectUnauthorized due to following code snippet:
this.rejectUnauthorized = options.strictSSL || options.rejectUnauthorized || true;

This turns out to be a problem for npm Enterprise (npm/policy-follower) that depends on changes-stream package when using an SSL interceptor even Reject Unauthorized option for npm Enterprise is set to false:
changes-stream error: self signed certificate in certificate chain

Error: CouchDB disconnected gracefully

I get the following errors after leaving a connection open for a few hours.

[tracker - 27229]: { Error: CouchDB disconnected gracefully
[tracker - 27229]:     at ChangesStream._onEnd (./node_modules/changes-stream/index.js:297:13)
[tracker - 27229]:     at emitNone (events.js:91:20)
[tracker - 27229]:     at IncomingMessage.emit (events.js:185:7)
[tracker - 27229]:     at endReadableNT (_stream_readable.js:974:12)
[tracker - 27229]:     at _combinedTickCallback (internal/process/next_tick.js:74:11)
[tracker - 27229]:     at process._tickDomainCallback (internal/process/next_tick.js:122:9)

It then seems that the stream maybe isn't really closed but it would be nice to either mention this in the error message (alongside with some info about the stream which would also be solved with #16).

npm publish with the latest packages

The last published version v2.2.0 was pushed three years ago, 30/11/2016 - https://www.npmjs.com/package/changes-stream

However, you have merged other changes since, more specifically the one where you upgraded packages - cbfd7d6 and debug moved from ~0.8.0 to ~3.1.0.

If you could bump up the version number and push the package again, it would help remove security warnings from the apps who are using this package.

Thanks!