Giter Site home page Giter Site logo

elennick / retry4j Goto Github PK

View Code? Open in Web Editor NEW
200.0 9.0 26.0 294 KB

Lightweight Java library for retrying unreliable logic - DEPRECATED

License: MIT License

Java 100.00%
retry-library retry retry-strategies retries java-library java java-8 java8 backoff-strategy deprecated

retry4j's Introduction

Coverage Status Maven Central License: MIT

DEPRECATED

Retry4j is no longer maintained and has had no releases for several years. Please feel free to fork and make your own changes/releases if you want to build upon it. Also consider checking out more modern libraries such as:

Retry4j

Retry4j is a simple Java library to assist with retrying transient failure situations or unreliable code. Retry4j aims to be readable, well documented and streamlined.

Table of Contents

Basic Code Examples

Handling Failures with Exceptions

Callable<Object> callable = () -> {
    //code that you want to retry until success OR retries are exhausted OR an unexpected exception is thrown
};

RetryConfig config = new RetryConfigBuilder()
        .retryOnSpecificExceptions(ConnectException.class)
        .withMaxNumberOfTries(10)
        .withDelayBetweenTries(30, ChronoUnit.SECONDS)
        .withExponentialBackoff()
        .build();
        
try {  
    Status<Object> status = new CallExecutorBuilder()
        .config(config)
        .build()
        .execute(callable);
    Object object = status.getResult(); //the result of the callable logic, if it returns one
} catch(RetriesExhaustedException ree) {
    //the call exhausted all tries without succeeding
} catch(UnexpectedException ue) {
    //the call threw an unexpected exception
}

Or more simple using one of the predefined config options and not checking exceptions:

Callable<Object> callable = () -> {
    //code that you want to retry
};

RetryConfig config = new RetryConfigBuilder()
    .exponentialBackoff5Tries5Sec()
    .build();

Status<Object> status = new CallExecutorBuilder().config(config).build().execute(callable);

Handling All Results with Listeners

Callable<Object> callable = () -> {
    //code that you want to retry
};

RetryConfig config = new RetryConfigBuilder()
        .exponentialBackoff5Tries5Sec()
        .build();

CallExecutor executor = new CallExecutorBuilder<>()
        .config(config).
        .onSuccess(s -> { //do something on success })
        .onFailure(s -> { //do something after all retries are exhausted })
        .afterFailedTry(s -> { //do something after a failed try })
        .beforeNextTry(s -> { //do something before the next try })
        .onCompletion(s -> { //do some cleanup })
        .build();
        
executor.execute(callable);

Dependencies

Maven

<dependency>
    <groupId>com.evanlennick</groupId>
    <artifactId>retry4j</artifactId>
    <version>0.15.0</version>
</dependency>

SBT

libraryDependencies += "com.evanlennick" % "retry4j" % "0.15.0"

Gradle

compile "com.evanlennick:retry4j:0.15.0"

Usage

General

Retry4j does not require any external dependencies. It does require that you are using Java 8 or newer.

Exception Handling Config

If you do not specify how exceptions should be handled or explicitly say failOnAnyException(), the CallExecutor will fail and throw an UnexpectedException when encountering exceptions while running. Use this configuration if you want the executor to cease its work when it runs into any exception at all.

RetryConfig config = new RetryConfigBuilder()
        .failOnAnyException()
        .build();

If you want to specify specific exceptions that should cause the executor to continue and retry on encountering, do so using the retryOnSpecificExceptions() config method. This method can accept any number of exceptions if there is more than one that should indicate the executor should continue retrying. All other unspecified exceptions will immediately interupt the executor and throw an UnexpectedException.

RetryConfig config = new RetryConfigBuilder()
        .retryOnSpecificExceptions(ConnectException.class, TimeoutException.class)
        .build();

If you want the executor to continue to retry on all encountered exceptions, specify this using the retryOnAnyException() config option.

RetryConfig config = new RetryConfigBuilder()
        .retryOnAnyException()
        .build();

If you want the executor to continue to retry when encountered exception's cause is among a list of exceptions, then specify retryOnCausedBy() config option

RetryConfig config = new RetryConfigBuilder()
        .retryOnCausedBy()
        .retryOnSpecificExceptions(ConnectException.class, TimeoutException.class)
        .build();

If you want the executor to continue to retry on all encountered exceptions EXCEPT for a few specific ones, specify this using the retryOnAnyExceptionExcluding() config option. If this exception strategy is chosen, only the exceptions specified or their subclasses will interupt the executor and throw an UnexpectedException.

RetryConfig config = new RetryConfigBuilder()
        .retryOnAnyExceptionExcluding(CriticalFailure.class, DontRetryOnThis.class)
        .build();

NOTE: When using retryOnSpecificExceptions and retryOnAnyExceptionExcluding, the call executor will also take into account if the encountered exceptions are subclasses of the types you specified. For example, if you tell the configuration to retry on any IOException, the executor will retry on a FileNotFoundException which is a subclass of IOException.

If you do not want to use these built-in mechanisms for retrying on exceptions, you can override them and create custom logic:

RetryConfig config = new RetryConfigBuilder()
        .retryOnCustomExceptionLogic(ex -> {
            //return true to retry, otherwise return false
        })
        .build();

If you create custom exception logic, no other built-in retry-on-exception configuration can be used at the same time.

Value Handling Config

If you want the executor to retry based on the returned value from the Callable:

RetryConfig config = retryConfigBuilder
        .retryOnReturnValue("retry on this value!")
        .build();

This can be used in combination with exception handling configuration like so:

RetryConfig config = retryConfigBuilder
        .retryOnSpecificExceptions(FileNotFoundException.class)
        .retryOnReturnValue("retry on this value!")
        .build();

In the above scenario, the call execution will be considered a failure and a retry will be triggered if FileNotFoundException.class is thrown OR if the String retry on this value! is returned from the Callable logic.

To retry only using return values, you can disable exception retries using the failOnAnyException() configuration option:

RetryConfig config = retryConfigBuilder
        .failOnAnyException()
        .retryOnReturnValue("retry on this value!")
        .build();

Timing Config

To specify the maximum number of tries that should be attempted, specify an integer value in the config using the withMaxNumberOfTries() method. The executor will attempt to execute the call the number of times specified and if it does not succeed after all tries have been exhausted, it will throw a RetriesExhaustedException.

RetryConfig config = new RetryConfigBuilder()
        .withMaxNumberOfTries(5)
        .build();

If you do not wish to have a maximum and want retries to continue indefinitely, instead us the retryIndefinitely() option:

RetryConfig config = new RetryConfigBuilder()
        .retryIndefinitely()
        .build();

To specify the delay in between each try, use the withDelayBetweenTries() config method. This method will accept a Java 8 Duration object or an integer combined with a ChronoUnit.

//5 seconds
RetryConfig config = new RetryConfigBuilder()
        .withDelayBetweenTries(5, ChronoUnit.SECONDS)
        .build();

//2 minutes
RetryConfig config = new RetryConfigBuilder()
        .withDelayBetweenTries(Duration.of(2, ChronoUnit.MINUTES))
        .build();

//250 millis
RetryConfig config = new RetryConfigBuilder()
        .withDelayBetweenTries(Duration.ofMillis(250))
        .build();

Backoff Strategy Config

Retry4j has built in support for several backoff strategies. They can be specified like so:

//backoff strategy that delays with the same interval in between every try
RetryConfig config = new RetryConfigBuilder()
        .withFixedBackoff()
        .build();

//backoff strategy that delays at a slowing rate using an exponential approach
RetryConfig config = new RetryConfigBuilder()
        .withExponentialBackoff()
        .build();

//backoff strategy that delays at a slowing rate using fibonacci numbers
RetryConfig config = new RetryConfigBuilder()
        .withFibonacciBackoff()
        .build();

//backoff strategy that retries with no delay
//NOTE: any value specified in the config for "withDelayBetweenTries()" will be ignored if you use this strategy
RetryConfig config = new RetryConfigBuilder()
        .withNoWaitBackoff()
        .build();

//backoff strategy that randomly multiplies the delay specified on each retry
//useful if you want to force multiple threads not to retry at the same rate
RetryConfig config = new RetryConfigBuilder()
        .withRandomBackoff()
        .build();

//backoff strategy that delays at a slowing rate and also randomly multiplies the delay
//combination of Exponential Backoff Strategy and Random Backoff Strategy
RetryConfig config = new RetryConfigBuilder()
        .withRandomExponentialBackoff()
        .build();

Custom Backoff Strategies

Custom backoff strategies can be specified like so:

RetryConfig config = new RetryConfigBuilder()
        .withBackoffStrategy(new SomeCustomBackoffStrategy())
        .build();

...where SomeCustomBackoffStrategy is an object that implements the com.evanlennick.retry4j.backoff.BackoffStrategy interface. The only mandatory method to implement is getDurationToWait() which determines how long to wait between each try. Optionally, the validateConfig() method can also be implemented if your backoff strategy needs to verify that the configuration being used is valid.

For examples creating backoff strategies, check out the provided implementations here.

Simple Configs

Retry4j offers some predefined configurations if you just want to get rolling and worry about tweaking later. These configs can be utilized like so:

new RetryConfigBuilder()
    .fixedBackoff5Tries10Sec()
    .build();

new RetryConfigBuilder()
    .exponentialBackoff5Tries5Sec()
    .build();

new RetryConfigBuilder()
    .fiboBackoff7Tries5Sec()
    .build();

new RetryConfigBuilder()
    .randomExpBackoff10Tries60Sec()
    .build();

CallExecutor

Executing your code with retry logic is as simple as building a CallExecutor using CallExecutorBuilder with your configuration and then calling execute:

new CallExecutorBuilder.config(config).build().execute(callable);

The CallExecutor expects that your logic is wrapped in a java.util.concurrent.Callable.

Call Status

After the executor successfully completes or throws a RetriesExhaustedException, a Status object will returned or included in the exception. This object will contain detailed information about the call execution including the number of total tries, the total elapsed time and whether or not the execution was considered successful upon completion.

Status status = new CallExecutorBuilder().config(config).build().execute(callable);
System.out.println(status.getResult()); //this will be populated if your callable returns a value
System.out.println(status.wasSuccessful());
System.out.println(status.getCallName());
System.out.println(status.getTotalDurationElapsed());
System.out.println(status.getTotalTries());
System.out.println(status.getLastExceptionThatCausedRetry());

or

    try {  
        new CallExecutorBuilder().config(config).build().execute(callable);
    } catch(RetriesExhaustedException cfe) {
        Status status = cfe.getStatus();
        System.out.println(status.wasSuccessful());
        System.out.println(status.getCallName());
        System.out.println(status.getTotalDurationElapsed());
        System.out.println(status.getTotalTries());
        System.out.println(status.getLastExceptionThatCausedRetry());
    }

Retry4jException

Retry4j has the potential throw several unique exceptions when building a config, when executing retries or upon completing execution (if unsuccessful). All Retry4j exceptions are unchecked. You do not have to explicitly catch them if you wish to let them bubble up cleanly to some other exception handling mechanism. The types of Retry4jException's are:

  • UnexpectedException - Occurs when an exception is thrown from the callable code. Only happens if the exception thrown was not one specified by the retryOnSpecificExceptions() method in the config or if the retryOnAnyException() option was not specified as part of the config.
  • RetriesExhaustedException - This indicates the callable code was retried the maximum number of times specified in the config via withMaxNumberOfTries() and failed all tries.
  • InvalidRetryConfigException - This exception is thrown when the RetryConfigBuilder detects that the invoker attempted to build an invalid config object. This will come with a specific error message indicating the problem. Common issues might be trying to specify more than one backoff strategy (or specifying none), specifying more than one exceptions strategy or forgetting to specify something mandatory such as the maximum number of tries.

NOTE: Validation on the RetryConfigBuilder can be disabled to prevent InvalidRetryConfigException's from ever being thrown. This is not recommended in application code but may be useful when writing test code. Examples of how to disable it:

new RetryConfigBuilder().setValidationEnabled(false).build()

or

new RetryConfigBuilder(false);

Listeners

Listeners are offered in case you want to be able to add logic that will execute immediately after a failed try or immediately before the next retry (for example, you may want to log or output a statement when something is retrying). These listeners can be specified like so:

CallExecutor executor = new CallExecutorBuilder().config(config).build();

executor.afterFailedTry(s -> { 
    //whatever logic you want to execute immediately after each failed try
});

executor.beforeNextTry(s -> {
    //whatever logic you want to execute immediately before each try
});

Two additional listeners are also offered to indicate when a series of retries has succeeded or failed. They can be specified like so:

executor.onSuccess(s -> {
    //whatever logic you want to execute when the callable finishes successfully
});

executor.onFailure(s -> {
    //whatever logic you want to execute when all retries are exhausted
});

NOTE: If you register a failure listener with the CallExecutor, it will toggle off the throwing of RetriesExhaustedException's. Handling a failure after retries are exhausted will be left up to the listener.

If you wish to execute any sort of cleanup or finalization logic that will execute no matter what the final results is (success, exhausted retries, unexpected exception throw) you can implement the following listener:

executor.onCompletion(s -> {
    //whatever logic you want to execute after the executor has completed, regardless of status
});

Listeners can be chained together:

new CallExecutorBuilder<>()
       .config(config)
       .onSuccess(s -> System.out.println("Success!"))
       .onCompletion(s -> System.out.println("Retry execution complete!"))
       .onFailure(s -> System.out.println("Failed! All retries exhausted..."))
       .afterFailedTry(s -> System.out.println("Try failed! Will try again in 0ms."))
       .beforeNextTry(s -> System.out.println("Trying again..."))
       .build()
       .execute(callable);

Async Support

Retry4j has some built in support for executing and retrying on one or more threads in an asynchronous fashion. The AsyncCallExecutor utilizes threading and async mechanisms via Java's CompletableFuture API. A basic example of this in action with a single call:

AsyncCallExecutor<Boolean> executor = new CallExecutorBuilder().config(config).buildAsync();
CompletableFuture<Status<Boolean>> future = executor.execute(callable);
Status<Boolean> status = future.get();

In the above case, the logic in the callable will begin executing immediately upon executor.execute(callable) being called. However, the callable (with retries) will execute on another thread and the original thread that started execution will not be blocked until future.get() is called (if it hasn't completed).

This executor can also be used to trigger several Callable's in parallel:

AsyncCallExecutor<Boolean> executor = new CallExecutorBuilder().config(retryOnAnyExceptionConfig).buildAsync();

CompletableFuture<Status<Boolean>> future1 = executor.execute(callable1);
CompletableFuture<Status<Boolean>> future2 = executor.execute(callable2);
CompletableFuture<Status<Boolean>> future3 = executor.execute(callable3);

CompletableFuture.allOf(future1, future2, future3).join();

If you wish to define a thread pool to be used by your AsyncCallExecutor, you can define and pass in an ExecutorService in the constructor. When using this pattern, it's important to remember that this thread pool will not shut itself down and you will have to explicitly call shutdown() on the ExecutorService if you want it to be cleaned up.

ExecutorService executorService = Executors.newFixedThreadPool(10);
new CallExecutorBuilder().config(config).buildAsync(executorService);

You can register retry listeners and configuration on an AsyncCallExecutor in the same fashion as the normal, synchronous CallExecutor. All calls in all threads that are triggered from an AsyncCallExecutor after its construction will use the same listeners and configuration.

Logging

Retry4j contains detailed internal logging using SLF4J. If you do not specify a SLF4J implementation, these logs will be discarded. If you do provide an implementation (eg: Logback, Log4J, etc) you can specify the log level on the com.evanlennick.retry4j package to set Retry4j logging to a specific level.

Other Notes

Retry4j follows semantic versioning: http://semver.org/. As it is still version 0.x.x and prior to 1.0.0, the API is subject to rapid change and breakage.

There are a number of other retry libraries for Java and the JVM that might better suit your needs. Please feel free to check out the following libraries as well if Retry4j doesn't fit:

retry4j's People

Contributors

dev3loperb avatar elennick avatar fdlk avatar mkopylec avatar mustafaakin avatar pwhittlesea avatar ruhan1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

retry4j's Issues

add ability to retry on returned value

right now retry4j only allows specifying retry behavior based on whether or not an exception is thrown during execution... it would be valuable to make it so that retries can also occur in response to specific values being returned from the logic being retried

this should also cover being allowed to specify on a null value which would address issue #42

Config without withDelayBetweenTries() specified throws a NullPointerException

Create a valid config like so:

new RetryConfigBuilder()
.withMaxNumberOfTries(1)
.withNoWaitBackoff()
.build();

...you should not have to specify .withDelayBetweenTries(Duration.ZERO), it should be implicit as part of using the no wait backoff strategy. However if you use the config above, a null pointer exception is thrown when attempting to execute on line 65 in the executor:

long millisBetweenTries = config.getDelayBetweenRetries().toMillis();

RetriesExhaustedException has no cause

I think the RetriesExhaustedException should have a cause exception which is the lastExceptionThatCausedRetry. Often applications just log exception stack traces and do nothing more. When the exception has no cause there is no information what happened.
What do you think?

create a call executor builder or builders

use the builder pattern to construct and utilize the CallExecutor and AsyncCallExecutor... something like:

CallExecutorBuilder.newCall(<here the callable>).withConfig(<the configuration>).onRetryFailThen(<the action>).execute();

add verification in RetryConfigBuilder

right now it is easy to create a chained configuration that has multiple declarations that should be allowed (ie: settings two backoff strategies) or has missing configs that are required (ie: number of retries should be mandatory)... create a mechanism to enforce or error out in a clear way if a config is valid

update documentation for 0.2.0

  • add maven/gradle/sbt dependency documentation
  • update docs with new object names and examples and new 0.2.0 stuff, etc

refactor listeners to be chainable

should refactor the way listeners are registered so you can define an executor in a more functional way... something like this:

    //do something
}).onSuccess(results -> {
    //do something
}).onCompletion(results -> {
    //do something
}).execute(callable);```

refactor README

readme has been built up little by little for a long time and is getting a bit tough to read and update

would be good to rewrite it to be better organized and have better examples... could possibly move to github pages or another location as well

Bug in retrying on any exception exculding specific ones

Hi.
I've discovered a bug in the loop
If more than one exception is added to RetryConfigBuilder.retryOnAnyExceptionExcluding(...) then the functionality works only for the first exception in the Set. To work properly, all of the exceptions in the Set must be matched to exception thrown from Callable<T> try.

I really need this fix ASAP :)

Allowing retry indefinitely

For some use cases, you dont really want to fix numer of retry. It would be useful of allowing unspecified number of retry to retry indefinitely.

CallExecutor fails if null is returned

It is quite common that retryable call will produce null result. Now it is impossible to return null because of:

    private Optional<T> tryCall(Callable<T> callable) throws UnexpectedException {
        try {
            T result = callable.call();
            return Optional.of(result); // TODO Allow nulls, use ofNullable(result);
        } catch (Exception e) {
            if (shouldThrowException(e)) {
                logger.trace("Throwing expected exception {}", e);
                throw new UnexpectedException(e);
            } else {
                lastKnownExceptionThatCausedRetry = e;
                return Optional.empty();
            }
        }
    }

add ability to retry on custom conditions

Currently retry4j only supports retrying on a set of exception types or values being returned from the callable logic. There also needs to be a way to retry on other conditions such as if a exception is returned with a certain value inside of it... for example:

HttpClientErrorException is thrown by apache HttpClient, you may want to retry only when ex.getRawStatusCode returned a 5XX error code and not a 4XX error code from

Exponential backoff algorithm not working correctly

I am running into these three issues in the current implementation:

  1. If numberOfTriesFailed equals 1, backoff value always equals 0. I'd expect it to equal the delayBetweenAttempts parameter.
  2. backoffStrategy.getMillisToWait(5, Duration.ofMillis(100)) equals 1500, I'd expect it to equal 1000 * 2^4 so 1600.
  3. If numberOfTriesFailed becomes large, backoff value becomes negative. I'd expect it to top off at max long value.

dont allow max number of tries to be less than 1

It's possible to set max number of tries less than 1. This becomes confusing if you accidentally set it that way because RetriesExhaustedException gets thrown immediately without executing anything and it might be confusing as to why this is happening. Should add some validation to prevent this from occurring.

clean up potential threading issues with listeners and call executors

There are potential threading issues with the way the listener and call executor API's are currently set up. For example, it is possible to modify the listeners on an executor at any time even after construction, which could lead to modifications in retry behave while something is in the middle of executing. Another possibility could be that someone specifies behavior inside of a listener that is not thread safe and causes confusing, unpredictable or undesired behavior. This issue is to see what can be done to make these API's safer and to document best practices and potential side effects of mis-using these API's in multithreaded scenarios.

This relates to issue #28

Need access to exception causing failure.

It would be useful if the exception that caused the retry was somehow available in the AfterFailedTryListener. In particular, logging the exception from this handler is probably a common pattern.

A possible solution would be to populate a new field containing the last seen exception on the CallResults object.

BeforeNextTry listener called after final try

My understanding is that the listener provided to CallExecutor#beforeNextTry(...) will be called prior to the execution of another try.
However, looking into the code, CallExecutor#handleRetry(long, int) is called after a failed attempt within the retry loop:

for (tries = 0; tries < maxTries && !attemptStatus.wasSuccessful(); tries++) {
    logger.trace("Retry4j executing callable {}", callable);
    attemptStatus = tryCall(callable);

    if (!attemptStatus.wasSuccessful()) {
        handleRetry(millisBetweenTries, tries + 1);
    }

    logger.trace("Retry4j retrying for time number {}", tries);
}

And beforeNextTryListener.onEvent(status) is called at the end of handleRetry if set:

private void handleRetry(long millisBetweenTries, int tries) {
    refreshRetryStatus(false, tries);

    if (null != afterFailedTryListener) {
        afterFailedTryListener.onEvent(status);
    }

    sleep(millisBetweenTries, tries);

    if (null != beforeNextTryListener) {
        beforeNextTryListener.onEvent(status);
    }
}

This results in both the afterFailedTryListener and beforeNextTryListener being called after every failed try.
This is as expected except that on the final attempt where both are still called.

Is this correct based on the semantics of 'before next try'. Surely the beforeNextTryListener should not be called if there are no more tries to attempt?

I am happy to create a merge request to fix if it is deemed this should be corrected.

add slf4j logging

i would like to include slf4j and add a bunch of granular logging so that anyone consuming retry4j can set logging levels on com.evanlennick.retry4j and get logs using whatever slf4j implementation theyve chosen

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.