DNS Latency is the #1 “Key Performance Indicator” (KPI) for a DNS system’s success. This DNS Latency and Performance Test Tools document is provided to help Operators and others deploy effective DNS Resolver (rDNS), DNS Authoritative (aDNS), and other DNS Architectures. The following are tools that can be used to monitor, test, and troubleshoot DNS latency through DNS Resolvers Architectures and DNS Authoritative Architectures.

• GRC’s DNS Benchmark. GRC provides the Windows community (and Linux/Wine)  with a DNS Resolution testing utility to determine the exact performance of the DNS Resolver and the DNS Authoritative chain.  I’ve seen several big SPs use this tool in their NOC to monitor the DNS resolver clusters. They take an old workstation and set it up to run the queries on the customer’s side of the resolvers.
• DNSBlast. dnsblast is open source, simple, and really stupid load testing tool for DNS resolvers. It is useful in lab or commissioning environments. 
• DNS Performance Test – http://www.ghacks.net/2011/01/20/dns-performance-test/ – I’ve never used this one, but others have said it is useful for troubleshooting DNS issues.
• namebench – https://code.google.com/p/namebench/. This is a “sneaky” tool from Google that is trying to get people to use Google’s 8.8.8.8 DNS resolvers. I like the test to see if “censorship” is happening on your DNS resolvers. 🙂
• dnstop is a libpcap application (like tcpdump) that displays various tables of DNS traffic on your network. While not a tool to “measure” performance, dnstop is a valuable tool for DNS architectures that use open source DNS software.  Dnstop was created and maintained by The Measurement Factory.
• Tpsreport. Tpsreport is a DNS performance test tool provided by  Akamai’s under NDA.  The tpsreport tool can create a large number of queries:
  tpsreport -ip 1.2.3.4 -port 53 -name akamai.com -startingTPS 10 -rampInterval 1 -rampAmount 10 -rampMax 1000The process will begin at a slow rate of 10 transactions per second and ramp up to a maximum of 1k transactions per second.

• DNSPERF – DNSPERF a tool built by Nominum to evaluate the performance of Authoritative DNS deployments. This is the tool I’ve used in the past to do DNS latency benchmarking in the lab. Still, works. It is “scriptable” – allowing for a lab person to set up automated test. Some people use to monitor their big DNS resolvers (AT&T is one example).  http://linux.die.net/man/1/dnsperf 

• RESPERF – RSEPERF is the companion tool to DNSPERF. RESPERF test the DNS Resolvers. It is scriptable and ideal for lab testing DNS Resolvers. https://linux.die.net/man/1/resperf 

• Using JMeter to Evaluate DNS Resolution Performance. Andrey Pohilko wrote a nice user guide to use JMeter to evaluate the DNS Resolution time.
• Knot DNS Benchmark. The Knot DNS Team build an Authoritative DNS test script based on NLnet’s DISTEL test lab setup (presentation, p.28). Everything is open source posted here https://gitlab.labs.nic.cz/labs/dns-benchmarking/tree/master
• QUERYPERF – queryperf is an old DNS server query performance testing tool that was originally focused on DNS authoritative servers, but has several use cases for testing DNS Resolvers. 

Collections of other DNS Tools that include Testing Tools. There are several others who keep an eye on the list of DNS tools. Please also check them as they could be finding tools that have yet to be evaluated and included here:

• DNS-OARC’s Library. “DNS Operations, Analysis, and Research Center (DNS-OARC) brings together key operators, implementors, and researchers on a trusted platform so they can coordinate responses to attacks and other concerns, share information and learn together.” That means DNS-OARC is an excellent forum to find new tools the core DNS community is using. Signing up individually, asking questions, and encouraging your organization to join DNS-OARC helps everyone maintain quality DNS throughout the World.
• STATDNS. An excellent site maintained by Frederic Cambus. The resource page is packed with comprehensive links to DNS tools. 
• Linux Distribution Pages. Several of the Linux flavors have their distributions online. GENTO Linux is one example of all the DNS software & tools in one location.  

DNSSEC Testing

One of the major concerns people have with turning on DNSSEC support in their resolvers (yes, you normally need to turn it on), is the errors and misconfigurations in DNSSEC zones. The following are tools you can use to test & troubleshoot DNSSEC zones. 

• DNS VIZ (http://dnsviz.net/) – DNSViz is a tool for visualizing the status of a DNS zone. It was designed as a resource for understanding and troubleshooting deployment of the DNS Security Extensions (DNSSEC). It provides a visual analysis of the DNSSEC authentication chain for a domain name and its resolution path in the DNS namespace, and it lists configuration errors detected by the tool.

Web-based tools. There are also tools that are web-based that allow you to test the DNS lookup latency:

• Network Speed.com – http://network-speed.com/dnstime.php
• DNS Stuff – http://www.dnsstuff.com/tools

DNS Testing Methods

Thousand Eye’s  – How DNS+ Server Latency tests work 

This is a testing model that puts test three elements:

1. Test the latency between the stub resolver (the ThousandEyes probe) and the DNS Resolver. 
2. Test the recursion between the DNS Resolver and the DNS Authority.
3. Test the internal DNS Resolver delays.

The value is an end-to-end value. If the objective is to explore the path optimization between the DNS Stub (the client) and the DNS Resolver, then a cache hit would be used (for example one of the DNS Root domains). 

Tools to Help you Explore the DNS Resolution Impact

Many organizations and groups offer tools that help the exploration and troubleshooting of DNS resolution issues. This is a list that offers no endorsement but does provide information that people can use in their troubleshooting, analysis, and performance tuning. 

• Dareboost. Dareboost Test, Analyze and Optimize your website. The reporting is good. The nice thing is that the testing includes the impact on DNS.  See this blog for more information: DNS Mapping: one more feature for website speed tests with Dareboost, When you do the report, look for “More Metrics.” It provides a waterfall view with the DNS resolution impact. In a good location with quality DNS resolver, that would be in the 2 – 10 ms range. In places with overloaded, neglected, and non-optimized DNS resolver, you’ll see hundreds of MS DNS resolution latency. That will have an impact on web render time (and customer experience). 
• Neustar’s DNS Speed Test. This is an old and useful tool for testing the domain and the website performance. It is always one of the tools I check when troubleshooting issues with a site.
• DNSPerf is a service provided to monitor the major DNS Authoritative Cloud and DNS Resolver Cloud Operators. 
• DNS Check. DNS Check is a tool that allows Operators to test their EDNS0 Client Subnet settings. This is useful for DNS Resolver deployments that are optimizing for CDN deployments (which should be all Operators). 

DNS Performance Reference Articles and Reading List

The following is a list of reference articles, papers, and other materials that are useful for anyone diving into the area of “DNS Performance.” One element to be mindful is the role architecture plays in the performance of DNS. What works in the lab may not work as expected when deployed. Plus, what works for TLD DNS Architecture may not be optimal for an Authoritative DNS architecture optimized for eCommerce and will for sure not be the optimal architecture for a Broadband Operator’s DNS Resolver Architecture.  Also, be mindful that most articles will not explore the impact of security attacks on the DNS Resolver. I recommend anyone doing DNS architecture work to review the growing set up materials from the Operator’s Security Toolkit. 

How to Evaluate Performance of a DNS Resolver By Thomas Orthbandt Posted on August 1, 2012. This Nominum blog post focuses on thoughtfulness behind “cold cache” testing. Thomas points out that “The most important thing to test is recursion because it allows a resolver to find answers, not in its cache.” This is true, but recursion is a top of the resolver – upstream dependency. DNS recursion impact will include the Internet latency, the authoritative latency, and the ability of the DNS Resolver to respond to the results from the aDNS server. Thomas points are spot on. Testing metrics for rDNS servers need to be agreed with a full understanding of the interplay between the various factors.

Lies, Damn Lies and DNS Performance Statistics How to measure the real performance of a DNS caching resolver by Secure64. This is a good overview paper on factors to consider when building DNS Resolver performance test. It is an overview paper, worth reading, and provides a more comprehensive list of thing to test. What it does not cover is testing methodology, tools, and other details needed for effective testing.

Benchmarking DNS Reliably on Multi-core Systems. Internet Systems Consortium (ISC) has a complete walkthrough of using DNSPERF to set up a detailed test of name servers running multicore processors. It is a good walkthrough of the impact of tuning the Open Source deployment to effectively use the multicore resources.

A Comparative Analysis on Existing DNS Performance Measurement Mechanisms Ejaz Ahmad and Kashif Sarwar (2014). This is a quick read that essentially concluded that there is a lot of work to be done with DNS Resolver performance testing … “There is a huge gap for research on performance measurement on Caching layer of DNS.” The authors did some detailed write up around Queryperf. 

CAIDA’s DNS Research. The Center for Applied Internet Data Analysis (CAIDA) has consistently explored the robustness, quality, and performance of the world’s DNS system. This research is an excellent place to get more in-depth DNS insights. These insights help engineers build better testing architectures and more resilient DNS Architectures.