spotify/dns-java

DNS wrapper library that provides SRV lookup functionality

spotify-dns-java

This small DNS wrapper library provides some useful pieces of functionality related to SRV lookups.

Resilience

Sometimes it is useful to default to previously returned, retained values, if a dns lookup should fail or return an empty result. This behavior is controlled by the retainingDataOnFailures() and retentionDurationMillis(long) methods in DnsSrvResolvers.DnsSrvResolverBuilder.

Watching for Changes

It's often useful to update where you try to connect based on changes in lookup results, and this library provides functionality that allows you to get notified when things change by implementing this interface (defined in the ChangeNotifier interface):

  interface Listener<T> {

    /**
     * Signal that set of records changed.
     *
     * @param changeNotification An object containing details about the change
     */
    void onChange(ChangeNotification<T> changeNotification);
  }

  /**
   * A change event containing the current and previous set of records.
   */
  interface ChangeNotification<T> {
    Set<T> current();
    Set<T> previous();
  }

Take a look at the PollingUsage example for an example.

Metrics

If you have a statistics system that can be integrated with using the munin protocol, the method metered() in DnsSrvResolvers.DnsSrvResolverBuilder enables this in conjunction with the spotify munin forwarder. Have a look at the BasicUsage example for details on how to set that up.

Usage

The entry point to lookups is through an instance of DnsSrvResolver obtained via the DnsSrvResolvers factory class.

To periodically check a set of records and react to changes, use the DnsSrvWatcher interface obtained via the DnsSrvWatchers factory class.

For example code, have a look at BasicUsage example and PollingUsage example

To include the latest released version in your maven project, do:

  <dependency>
    <groupId>com.spotify</groupId>
    <artifactId>dns</artifactId>
    <version>3.2.2</version>
  </dependency>

NOTE: version 3.1.0 is broken; you cannot use the retention feature in that version.

License

This software is released under the Apache License 2.0. More information in the file LICENSE distributed with this project.

dns-java's Issues

spring-boot metrics

Could you point me in the right direction to integrate this library with spring boot actuator metrics?

package com.spotify.dns.statistics does not exist

Sorry if this is just a set up issue on my part, but I'm having trouble using the statistics package.

I'm using gradle and have compile("com.spotify:dns:2.2.0") as a dependency as described in your readme. When I omit .metered(REPORTER) from the builder, everything works fine, but when I try to import

import com.spotify.dns.statistics.DnsReporter;
import com.spotify.dns.statistics.DnsTimingContext;

and include the StdoutReporter from your BasicExample I get a bunch of errors such as the following:

error: package com.spotify.dns.statistics does not exist

Is there some step I'm missing? Thanks.

Add support for polling SRV records and getting update notifications

I've started porting some code from another internal Spotify library that implements this functionality on top of dns-java. I figured I could open this issue to discuss different design options.

Some things I want the feature to achieve:

• Give a mechanism for getting notified when the set of SRV records for a fqdn change
• Make it possible to use derivative values of LookupResult as the current set, with notifications only happening if that derived set changes

Suggested API

interface DnsSrvWatcher<T> extends Closeable {
  ChangeNotifier<T> watch(String fqdn);
}

interface ChangeNotifier<T> {
  Set<T> current();
  void setListener(Listener<T> listener, boolean fire);
  void close();

  interface Listener<T> {
    void endpointsChanged(ChangeNotification<T> changeNotification);
  }

  interface ChangeNotification<T> {
    Set<T> current();
    Set<T> previous();
  }
}

And the construction API would simply be:

DnsSrvResolver resolver = DnsSrvResolvers.newBuilder()
    // configure
    .build();

DnsSrvWatcher<LookupResult> watcher =
    DnsSrvResolvers.newWatcherBuilder(resolver)
        .polling(1, TimeUnit.SECONDS)
        .usingExecutor(myScheduledExecutorService)
        .build();

or

DnsSrvResolver resolver = DnsSrvResolvers.newBuilder()
    // configure
    .build();

DnsSrvWatcher<String> watcher =
    DnsSrvResolvers.newWatcherBuilder(resolver, (lookup) -> lookupToString(lookup))
        .polling(1, TimeUnit.SECONDS)
        .usingExecutor(myScheduledExecutorService)
        .build();

There also a more customizable option in the builder called customTrigger(DnsSrvWatcherFactory) for when you have external events that need to trigger the queries instead of periodic polling.

Not working when used in Tomcat Servlet

This library unable to fetch the information when used in Rest API? Any issues? Works standalone JVM.

Question: Support of the Additional Section in the answer

As far as I understand the code, there is no way to retrieve the Additional Section from the SRV request answer. Is this correct? If so, is it planned to support this in a later release?

Background is that for example the service discovery in Mesos clusters via Mesos DNS makes use of the Additional Section to directly provide IP/Port mappings to service names.

See mesosphere/mesos-dns#313

LookupResult should contain properly named getters

The getters should be called getHost(), getPort() etc. Failure to do so prevents automatic Jackson serialization.

Remove cachingLookups while initialising DnsSrvResolvers in BasicUsage class

When I tried to run BasicUsage class, it is throwing the exception"Session not supported with caching lookup" as expected.
So if we don't call the deprecated method cachingLookups in BasicUsage class while initialising DnsSrvResolvers, will solve this problem and helps to understand and debug the BasicUsage class. It helps who-ever cloning this repository and try to understand how this library works.

Example:

Depending on jsr305 is breaking java modules

Depending on com.google.code.findbugs:jsr305:jar:2.0.1 is breaking java modules
See https://blog.codefx.org/java/jsr-305-java-9/ and https://www.google.com/search?q=jsr305%20split%20package for info.

With modules you can slim down your java app including runtime with a few hundred megabytes. So it would be nice to do this.

As you only use @nullable in three files (one is a test) it should be trivial to do:

• src/main/java/com/spotify/dns/DnsException.java
• src/main/java/com/spotify/dns/ServiceResolvingChangeNotifier.java
• src/test/java/com/spotify/dns/ServiceResolvingChangeNotifierTest.java

`SERVFAIL` when trying to resolve a service address

What happened?

When upgrading from com.spotify:dns version 3.1.5 to 3.2.2 some of the services started having SERVFAIL even though the service is there.

What was expected?

As there's no breaking change in the perceived API from com.spotify:dns, we expected the changes to not affect functionality.

How to reproduce

We didn't find a good way to reproduce. We didn't manage to pin down what is causing the problem. It seems related to some concurrency, as sometimes the problem doesn't appear. I am more than glad to show the issue happening in a service.

Context

We need to upgrade dnsjava:dnsjava to from version 2.x to 3.x. We checked that com.spotify:dns has done this change in version 3.2.0. We tested in some services and they seem to be working fine, so we decided to roll out the change for all of our users. What happened is that in some of them, from what we can see the ones using gRPC, they started getting SERVFAIL intermittently.

Here is an anonymised stack trace:

Jul 15, 2021 4:29:20 PM io.grpc.internal.ManagedChannelImpl$NameResolverListener handleErrorInSyncContext
WARNING: [Channel<38>: (${PROTOCOL}://${SERVICE})] Failed to resolve name. status=Status{code=UNAVAILABLE, description=null, cause=java.util.concurrent.CompletionException: com.spotify.dns.DnsException: Lookup of '${PREFIX}-${SERVICE}._grpc.services.${DOMAIN_ADDRESS}' failed with code: 2 - SERVFAIL 
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314)
	at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:683)
	at java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:658)
	at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2094)
	at com.spotify.grpc.DnsSrvNameResolver.lambda$resolver$4(DnsSrvNameResolver.java:160)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.spotify.dns.DnsException: Lookup of '${PREFIX}-${SERVICE}._grpc.services.${DOMAIN_ADDRESS}' failed with code: 2 - SERVFAIL 
	at com.spotify.dns.XBillDnsSrvResolver.resolve(XBillDnsSrvResolver.java:60)
	at com.spotify.grpc.DnsSrvNameResolver.lambda$resolver$0(DnsSrvNameResolver.java:162)
	at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:680)
	... 6 more
}

We tried bumping version of dnsjava:dnsjava from 3.0.2 to 3.4.0 and the problem seemed to go away, but after some minutes (around ~10min) of the service running it started again. I am not sure if this was a local problem.

When we did a dig srv ${PREFIX}-${SERVICE}._grpc.services.${DOMAIN_ADDRESS} some hosts are returned as expected. Changing the version back to com.spotify:dns:3.1.5 and dnsjava:dnsjava:2.x makes the problem go away.

Java version used during the test:

$ java -version
> openjdk version "11.0.10" 2021-01-19 LTS
> OpenJDK Runtime Environment Corretto-11.0.10.9.1 (build 11.0.10+9-LTS)
> OpenJDK 64-Bit Server VM Corretto-11.0.10.9.1 (build 11.0.10+9-LTS, mixed mode)

new release timeline

the new stats architecture is really nice - when will it be ready for prime time?

Race condition when registering a listener with a watcher.

This test seems to show that there is a race condition when registering i listener.

mikaelgoldmann@5091957

It appears that when setting fire = true, if there is also a change in the watched record, the listener can be called by two threads concurrently, possibly seeing the old result after the new result.

Provide a way to turn off response caching

First off, many thanks for this library - it's great, very easy to use!

It would be nice if there was a way to robustly turn off all response caching. DnsSrvResolverBuilder provides the cacheLookups option, but it looks like that only controls caching of Lookup instances - not caching of the actual responses, as dnsjava has a response cache shared between instances by default.

We were able to work around this by running:

    Lookup.getDefaultCache(DClass.IN).setMaxCache(0)
    Lookup.getDefaultCache(DClass.IN).setMaxNCache(0)

before performing any requests, but perhaps SimpleLookupFactory should set the cache on its Lookup instances to one which doesn't perform any caching?

Published jar's pom.xml lists outdated dnsjava dependency

I experienced this bug in my application, which is not a recent bug:
dnsjava/dnsjava#177

I am using the latest com.spotify:dns:3.2.2, but I was not specifying the downstream dnsjava dependency version explicitly. Note that this repo's pom.xml lists:

        <dependency>
            <groupId>dnsjava</groupId>
            <artifactId>dnsjava</artifactId>
            <version>3.4.2</version>
        </dependency>

However, I noted that my resolved jar in a distribution is:

$ ls -al *dns*
-rw-rw-r-- 1 root root  37162 Dec 14 18:06 dns-3.2.2.jar
-rw-rw-r-- 1 root root 399838 Dec 14 18:06 dnsjava-3.0.2.jar

This lines up with what gradle's dependencies tool shows:

|    |    |    |    +--- com.spotify:dns -> 3.2.2
|    |    |    |    |    +--- dnsjava:dnsjava:3.0.2

Finally, I observed that the pom.xml included in the com.spotify:dns:3.2.2 jar is not 3.4.2:

        <dependency>
            <groupId>dnsjava</groupId>
            <artifactId>dnsjava</artifactId>
            <version>3.0.2</version>
        </dependency>

I resolved my issue by forcing dnsjava to the latest version (3.5.2). It would be great if users of this library did not need to do that manually though!

consul integration review request

I've written about a prototype I completed (http://txt.fliglio.com/2014/10/spring-boot-actuator/) using dns-java to integrate spring boot with consul for service discovery and codahale metrics to track lookup times / failures.

Thanks for all the work you've put into this - I just wanted to pass along the work I did and possibly get feedback on whether my implementation is in line with how you would expect this library to be used.

Thanks!

Local variable is shadowing class field

When I was going through one of the class AggregatingChangeNotifier, I saw there is a class field called records and there is a private method in the same class called aggregateSet(), in that method there is a local variable called records. This local variable will override the class field.
Overriding or shadowing a variable declared in an outer scope can strongly impact the readability and maintainability, of a piece of code. Also there is a possibility of introducing a bug. So better to rename this local variable.

Screenshot for the same:

QUESTION: Method to return a Random node.

What is the best way to get a Random or Round Robin (like) node from LookupResult List?

Thanks

feature-request: async lookup support

It seems dns-java's resolver actually supports sendAsync method, but there's no way to do async lookups in DnsSrvResolver