mathieuturcotte/node-backoff

Fibonacci and exponential backoff for Node.js

node-backoff's Issues

Test fails with recent sinon

Hello,

test fails with recent sinon. Could you update them ?

Cheers,
Xavier

Conditional backoff

This is more of a question than a bug.
I am trying to use backoff retries on conditional basis and have the following specific scenario to support.
In my usecase, for some conditions (like network timeout / server failure), I would want a retry of the function (using exponential backoff strategy) and in some other conditions (like wrong username/password), there should not be any retry, but call should just fail.
How can I have this conditional callback possible?
I am using backoff.call() API

TypeScript typings

Would be great to have typings for this package.

How to clean up a backoff instance?

I'd like to get rid of an instance from backoff.fibonacci(), especially the event listeners. What would be the best way? Thanks!

Retry count availability from FunctionCall

With the fix to #9 in place, there's no longer a convenient method of querying the retry count. It would be a convenient feature to provide in the new API.

Add an .isRunning() method to FunctionCall

It would be useful to be able to test if a FunctionCall instance is currently running. Since .start() throws an error if called twice, there should be a way to see prevent this by checking the current state.

Possible to backoff from inside function call?

I'm working with an api where sometimes the res.statusCode for a request is 503 but the err.code isn't. This means call.retryIf is never called because err is undefined.

My attempted remedy:

const reqRetryCodes = {
    'ECONNRESET': true,
    'ETIMEDOUT': true,
    'ENOTFOUND': true,
    503: true
}

        const _tryRequest = backoff.call(
            request,
            {
                url: myUrl,
                json: true
            },
            (err,res,body) => {
                if(err){
                    if(reqRetryCodes[err.code]===true){
                        _tryRequest.backoff(); //retry backoff
                        return;
                    }
                    reject(err);
                    return;
                }
                if(reqRetryCodes[res.statusCode]===true){
                    _tryRequest.backoff(); //retry backoff
                    return;
                }
                resolve(body);
            }
        );

        _tryRequest.setStrategy(new backoff.ExponentialStrategy());
        _tryRequest.start();

Gives:

TypeError: _tryRequest.backoff is not a function

memory leak

Using backoff 2.3.0 and node v0.10.28 on SmartOS, this program appears to leak memory quite badly:

var backoff = require('backoff');
var count = 2000;

function main()
{
        for (i = 0; i < count; i++) {
                startBackoff(function () {
                        console.error('never reached');
                });
        }
}

function myOperation(_, callback)
{
        setTimeout(callback, 5, new Error('test error'));
}

function startBackoff(callback)
{
        var retry;

        retry = backoff.call(myOperation, {},
            function (err, answers) { callback(err, answers); });

        retry.setStrategy(new backoff.ExponentialStrategy({
            initialDelay: 10,
            maxDelay: 500
        }));

        retry.start();
}

main();

When I run this, after about a minute:

# node testclient.js
FATAL ERROR: Evacuation Allocation failed - process out of memory
Abort (core dumped)

Here's the memory usage during the minute or so it takes to crash, measured using "ps -ostime,etime,pid,rss,vsz", which shows the start time of the process, the elapsed time since it started, the pid, the resident set size, and the virtual memory size:

   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       00:07 53361 71068 80808
   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       00:12 53361 88656 100248
   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       00:17 53361 104136 114608
   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       00:22 53361 122640 133024
   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       00:27 53361 140068 150432
   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       00:32 53361 158536 168864
   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       00:37 53361 175292 186264
   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       00:42 53361 197700 209792
   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       00:47 53361 210108 220064
   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       00:52 53361 229004 239520
   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       00:57 53361 245640 255896
   STIME     ELAPSED   PID  RSS  VSZ
18:44:05       01:02 53361 249652 260040

I'm running this inside a container that's got 256MB of DRAM allocated to it, which is why it dies shortly after that.

I also measured GC time using "nhttpsnoop" and noticed a spike just before it died (which is common for memory leaks):

[ 51.646622]  53361 gc     <-   23.259ms -      -                   
[ 52.295661]  53361 gc     <-   16.736ms -      -                   
[ 53.106390]  53361 gc     <-   18.716ms -      -                   
[ 53.584080]  53361 gc     <-  395.877ms -      -                   
[ 54.204881]  53361 gc     <-    9.381ms -      -                   
[ 55.312229]  53361 gc     <-   81.404ms -      -                   
[ 55.831290]  53361 gc     <-  518.915ms -      -                   
[ 56.944586]  53361 gc     <-  513.715ms -      -                   
[ 58.543084]  53361 gc     <-  569.716ms -      -   

I have the core file, and a cursory analysis shows almost 200K objects with properties _idleTimeout, _idlePrev, _idleNext, _idleStart, _onTimeout, and _repeat; 246K objects that look like the Error I'm creating above; and 450K 1-element-arrays, where in at least one of them, the only element is the Error I'm creating above. This could be a red herring, since I'm analyzing raw memory, but the program is clearly leaking lots of memory, and those are the objects that stand out.

Am I doing something obviously wrong?

Strategy with a Exponential jittered backoff

A possibile alternative exponential strategy to implment could be a jittered exponential backoff strategy as depicted in the Spotify SDK SPTDataLoader

[RxJS] backoff notifier handler for Observable.retryWhen()?

Could we make a new interface for Rx.Observable.retryWhen handler?

Here's a backoff example in RxJS document:
https://github.com/Reactive-Extensions/RxJS/blob/master/doc/api/core/operators/retrywhen.md#example-an-incremental-back-off-strategy-for-handling-errors

unable to define function?

Hello,
I'm having issues with your tutorial -

var call = backoff.call(get, url1+url2, function(err, res) {
console.log('=================Num tries : ' + call.getNumRetries());
if (err) {
console.log('Error: ' + err.message);
}
else {
console.log('Status: ' + res.statusCode);
}
});
call.setStrategy(new backoff.ExponentialStrategy());
call.failAfter(10);
call.start();

when I try to do this, I get errors like this :

ReferenceError: get is not defined
at Request._callback (c:\Users\SDS\WebProject\brighticscloudweb\routes\ia.js:389:49)
at Request.self.callback (c:\Users\SDS\WebProject\brighticscloudweb\node_modules\request\request.js:198:22)
at Request.emit (events.js:110:17)
at Request. (c:\Users\SDS\WebProject\brighticscloudweb\node_modules\request\request.js:1057:14)
at Request.emit (events.js:129:20)
at IncomingMessage. (c:\Users\SDS\WebProject\brighticscloudweb\node_modules\request\request.js:1003:12)
at IncomingMessage.emit (events.js:129:20)
at _stream_readable.js:908:16
at process._tickCallback (node.js:355:11)

It's unable to define the get function. Any help?

Aborts cannot be detected

As far as I can tell, once abort is called, no event is called and the completion handler doesn't execute.

How to use this package in current project?

I wanna use "backoff" to create a pooling request with expodental back off, and I added

var back = require('backoff')

And it said

Uncaught ReferenceError: require is not defined

First run without delay

Feature request:
Adding an option that the backoff will run without delay once and from the second time on the delay will start.

The motive is that the part of the code that is backed off anyway going to be running somewhere before the backoff and if it fails it will start the backoff.

is backoff pending?

What is the recommended approach for checking if a backoff is currently pending? I'm currently checking the if timeoutID_ !== -1 to determine if a backoff is currently pending. I'd rather not depend on an implementation detail like backoff.timeoutID_.

The following is my use case:

var Backoff = require('backoff');

var backoff = Backoff.exponential();

// if the browser goes online then let's reset the backoff
//  to provide a better user experience when attempting to reconnect
window.addEventListener('online', function() {
    // are we currently waiting to reconnect
    const is_pending_backoff = backoff.timeoutID_ !== -1;
    // reset the backoff
    backoff.reset();
    // since resetting the backoff will cancel a
    //  pending backoff lets go ahead and trigger
    //  a new backoff
    if( is_pending_backoff ) {
        backoff.backoff();
    }
} );

Maybe adding an isPending to the api is a good idea?

Backoff.prototype.isPending = function() {
    return this.timeoutID_ != -1;
}

Thanks for a great library!
Ryan

Is fibonacci that useful?

Fibonacci number can be calculated exponentially:

function fib(n) {
  var phi = (1/2+Math.sqrt(5)/2);
  return Math.round(Math.pow(phi, n) / Math.sqrt(5));
}
var a = 0, b = 1, tmp;
for (var i = 2; i < 100; i++) {
  tmp = b;
  b += a;
  a = tmp;
  console.log(b, fib(i));
}

Run this script and you will see...

So probably you would like to merge the fibonacci strategy into the exponential one.

FunctionCall has a default failAfter that cannot be overridden to be forever

The FunctionCall interface has a default failAfter of 5. The default for Backoff objects is not never fail. The difference in defaults may be reasonable (if it is documented), but currently there does not seem to be any way to specify that a FunctionCall should never fail.