m0rcq/dnsperf

DNS Performance Testing Tools

dnsperf's Issues

Testing DoH endpoints without forced reconnects (non one-client -> one-query)

[Status] Command line: dnsperf -m doh -c 4 -q 200 -l 5 -t 5 -O doh-uri=https://dns.cloudflare.com/dns-query -O doh-method=GET -s dns.cloudflare.com -d queries
[Status] Sending queries (to [2606:4700::6810:85e5]:443)
[Status] Started at: Tue Jul 13 11:27:58 2021
[Status] Stopping after 5.000000 seconds
[Status] Testing complete (time limit)

Statistics:

  Queries sent:         37886
  Queries completed:    37886 (100.00%)
  Queries lost:         0 (0.00%)

  Response codes:       NOERROR 32474 (85.72%), NXDOMAIN 5412 (14.28%)
  Average packet size:  request 35, response 67
  Run time (s):         5.066982
  Queries per second:   7477.034653

  Average Latency (s):  0.025906 (min 0.006251, max 0.146677)
  Latency StdDev (s):   0.020486

Connection Statistics:

  Reconnections:        0

  Average Latency (s):  2.518372 (min 0.013354, max 5.001711)
  Latency StdDev (s):   1.433612

Implementation Notes

Resperf DoH testing

zcat queryfile-example-10million-201202.gz | ./src/resperf -M doh -O doh-uri=https://dns.cloudflare.com/dns-query -O doh-method=GET -m 1000 -v -q 1000 -s dns.cloudflare.com
DNS Resolution Performance Testing Tool
Version 2.6.0

[Status] Command line: resperf -M doh -O doh-uri=https://dns.cloudflare.com/dns-query -O doh-method=GET -m 1000 -v -q 1000 -s dns.cloudflare.com
[Status] Sending
[Status] Reached 1000 outstanding queries
[Status] Waiting for more responses
[Status] Testing complete

Statistics:

  Queries sent:         10964
  Queries completed:    9997
  Queries lost:         967
  Response codes:       NOERROR 7178 (71.80%), SERVFAIL 144 (1.44%), NXDOMAIN 2673 (26.74%), REFUSED 2 (0.02%)
  Reconnection(s):      1946
  Run time (s):         81.275133
  Maximum throughput:   570.000000 qps
  Lost at that point:   0.00%

Memory leaks testing (valgrind memcheck)

no memory losses reported against latest feature/doh.

==30344== Invalid read of size 8
==30344==    at 0x10DEAD: cleanup (dnsperf.c:568)
==30344==    by 0x10DEAD: main (dnsperf.c:1325)
==30344==  Address 0x1ffed00538 is on thread 1's stack
==30344==  in frame #0, created by main (dnsperf.c:1237)
==30344==
==30344== Invalid read of size 8
==30344==    at 0x10DEC2: cleanup (dnsperf.c:570)
==30344==    by 0x10DEC2: main (dnsperf.c:1325)
==30344==  Address 0x1ffed00540 is on thread 1's stack
==30344==  in frame #0, created by main (dnsperf.c:1237)
==30344==
==30344== Warning: client switching stacks?  SP change: 0x1ffed003d0 --> 0x1fff000838
==30344==          to suppress, use: --max-stackframe=3146856 or greater
==30344==
==30344== HEAP SUMMARY:
==30344==     in use at exit: 3,859,239 bytes in 816 blocks
==30344==   total heap usage: 173,218 allocs, 172,402 frees, 46,985,795 bytes allocated
==30344==
==30344== LEAK SUMMARY:
==30344==    definitely lost: 0 bytes in 0 blocks
==30344==    indirectly lost: 0 bytes in 0 blocks
==30344==      possibly lost: 0 bytes in 0 blocks
==30344==    still reachable: 3,859,239 bytes in 816 blocks
==30344==         suppressed: 0 bytes in 0 blocks
==30344== Rerun with --leak-check=full to see details of leaked memory
==30344==
==30344== For counts of detected and suppressed errors, rerun with: -v
==30344== ERROR SUMMARY: 8196295 errors from 241 contexts (suppressed: 0 from 0)

Identify Google DoH random failures

Randomly we see "Error: failed to receive packet: Success" when using Google's DoH.

DNS Performance Testing Tool
Version 2.6.0

[Status] Command line: dnsperf -m doh -q 200 -l 60 -t 5 -O doh-uri=https://dns.google/dns-query? -s dns.google -d queries
[Status] Sending queries (to 8.8.4.4:443)
[Status] Started at: Thu Jul  8 20:30:36 2021
[Status] Stopping after 60.000000 seconds
Warning: received a response with an unexpected (maybe timed out) id: 0
Error: failed to receive packet: Success

Testing DoH endpoints results

Tested:

• Cloudflare DoH (https://dns.cloudflare.com/dns-query)
• Google DoH (https://dns.google/dns-query)
• DNSLify DoH (https://doh.dnslify.com/dns-query)
• nic.cz DoH (https://odvr.nic.cz/doh)
• Adguard DoH (https://dns.adguard.com/dns-query)
• nextdns.io DoH (https://dns.nextdns.io)
• Quad9 DoH (https://dns.quad9.net/dns-query)
• Comcast Xfinity DoH (https://doh.xfinity.com/dns-query)

Cloudflare POST run:

DNS Performance Testing Tool
Version 2.6.0

[Status] Command line: dnsperf -m doh -c 4 -q 200 -l 60 -t 5 -O doh-uri=https://dns.cloudflare.com/dns-query? -O doh-method=POST -s dns.cloudflare.com -d queries
[Status] Sending queries (to [2606:4700::6810:85e5]:443)
[Status] Started at: Fri Jul  9 07:28:22 2021
[Status] Stopping after 60.000000 seconds
[Status] Testing complete (time limit)

Statistics:

  Queries sent:         7040
  Queries completed:    7040 (100.00%)
  Queries lost:         0 (0.00%)

  Response codes:       NOERROR 7040 (100.00%)
  Average packet size:  request 33, response 66
  Run time (s):         60.001607
  Queries per second:   117.330191

  Average Latency (s):  0.019304 (min 0.009319, max 1.234181)
  Latency StdDev (s):   0.015485

Connection Statistics:

  Reconnections:        7033

  Average Latency (s):  0.013757 (min 0.009008, max 0.372826)
  Latency StdDev (s):   0.008589

Google DoH:

DNS Performance Testing Tool
Version 2.6.0

[Status] Command line: dnsperf -m doh -q 10 -l 30 -t 5 -O doh-uri=https://dns.google/dns-query? -s dns.google -d queries
[Status] Sending queries (to 8.8.8.8:443)
[Status] Started at: Thu Jul  8 16:52:22 2021
[Status] Stopping after 30.000000 seconds
[Status] Testing complete (time limit)

Statistics:

  Queries sent:         149
  Queries completed:    149 (100.00%)
  Queries lost:         0 (0.00%)

  Response codes:       NOERROR 112 (75.17%), NXDOMAIN 37 (24.83%)
  Average packet size:  request 33, response 64
  Run time (s):         30.044311
  Queries per second:   4.959342

  Average Latency (s):  0.100357 (min 0.096377, max 0.188675)
  Latency StdDev (s):   0.007455

Connection Statistics:

  Reconnections:        148

  Average Latency (s):  0.100593 (min 0.089813, max 0.148259)
  Latency StdDev (s):   0.005280

DNSLify:

DNS Performance Testing Tool
Version 2.6.0

[Status] Command line: dnsperf -m doh -q 200 -l 30 -t 5 -O doh-uri=https://doh.dnslify.com/dns-query? -s 185.235.81.1 -d queries
[Status] Sending queries (to 185.235.81.1:443)
[Status] Started at: Thu Jul  8 20:34:01 2021
[Status] Stopping after 30.000000 seconds
Warning: received a response with an unexpected (maybe timed out) id: 256
Warning: received a response with an unexpected (maybe timed out) id: 512
Warning: received a response with an unexpected (maybe timed out) id: 768
[Timeout] Query timed out: msg id 1
[Timeout] Query timed out: msg id 2
[Timeout] Query timed out: msg id 3
Warning: received a response with an unexpected (maybe timed out) id: 33024
[Timeout] Query timed out: msg id 129
[Status] Testing complete (time limit)

Statistics:

  Queries sent:         140
  Queries completed:    136 (97.14%)
  Queries lost:         4 (2.86%)

  Response codes:       NOERROR 102 (75.00%), NXDOMAIN 34 (25.00%)
  Average packet size:  request 33, response 65
  Run time (s):         30.093776
  Queries per second:   4.519207

  Average Latency (s):  0.105208 (min 0.090140, max 0.348431)
  Latency StdDev (s):   0.026542

Connection Statistics:

  Reconnections:        138

  Average Latency (s):  0.106668 (min 0.092155, max 0.199602)
  Latency StdDev (s):   0.017480

DNSLify 2nd run:

DNS Performance Testing Tool
Version 2.6.0

[Status] Command line: dnsperf -m doh -q 100 -c 4 -l 30 -t 5 -O doh-uri=https://doh.dnslify.com/dns-query? -s 185.235.81.1 -d queries
[Status] Sending queries (to 185.235.81.1:443)
[Status] Started at: Thu Jul  8 20:35:28 2021
[Status] Stopping after 30.000000 seconds
Warning: received a response with an unexpected (maybe timed out) id: 30208
[Timeout] Query timed out: msg id 118
Warning: received a response with an unexpected (maybe timed out) id: 50688
Warning: received a response with an unexpected (maybe timed out) id: 57856
[Timeout] Query timed out: msg id 198
[Timeout] Query timed out: msg id 226
[Status] Testing complete (time limit)

Statistics:

  Queries sent:         492
  Queries completed:    489 (99.39%)
  Queries lost:         3 (0.61%)

  Response codes:       NOERROR 366 (74.85%), NXDOMAIN 123 (25.15%)
  Average packet size:  request 33, response 65
  Run time (s):         30.097825
  Queries per second:   16.247021

  Average Latency (s):  0.116697 (min 0.090816, max 0.250829)
  Latency StdDev (s):   0.028446

Connection Statistics:

  Reconnections:        486

  Average Latency (s):  0.117103 (min 0.091646, max 0.300916)
  Latency StdDev (s):   0.031624