Nozomi Networks find an easier path for DNS Cache Poison Attacks on ICS, CPEs, and other IoT devices.
Nozomi Networks disclosed long-term and persistent vulnerability with all versions of uClibc and uClibc-ng (see Nozomi Networks Discovers Unpatched DNS Bug in Popular C Standard Library Putting IoT at Risk by Giannis Tsaraias and Andrea Palanca | May 2, 2022). This vulnerability allows for threat actors to more easily succeed with DNS Cache Poison attacks – reducing the volume of brute force needed to match the DNS Transaction ID and Source Port.
Nozomi Networks advisory points out that OpenWRT and many other Linksys, Netgear, and Axis, or Linux distributions such as Embedded Gentoo. But, as pointed out on the OpenWRT forum, “OpenWrt switched from using uClibc to musl by default already back in 2015, so no, unless you’re running a really, really old version or you’re using a custom build with uClibc.” Andrea Palanca (one of the co-authors) shared from another member of the OpenWRT project:
“uClibc-ng is only used in OpenWrt 19.07 and there only on Synopsys ARC CPUs, all other targets are using musl libc in OpenWrt 19.07. In later OpenWrt versions like 21.02 Synopsys ARC CPUs changed to glibc in OpenWrt and uClibc-ng support was removed. OpenWrt 19.07 is end-of-life since March 2022. Many vendors are using OpenWrt versions which are not supported anymore by the upstream OpenWrt community, they are on their own.”
Confusing, which dictates that organizations need to ask for specific from their vendors. “Routers/CPEs” might be OK in a vendor but then their cameras, monitors, IoT, and ICS devices might be on a different branch.
What to remember with this advisory is focused on IoT and ICS systems. Nozomi Networks did the test, did find problems, and the investigations are leading to the discovery of potential software forks using obsolete code. Remember, the responsible disclosure flow had Nozomi Networks first approach the Industrial Control System Information Sharing and Analysis Center (ICS-ISAC) is an indicator of the feasible risk beyond Home Modem (CPEs) to a range of other devices connecting IoT and ICS devices.
The apparent long-term vulnerability persistence was another point that Nozomi Networks did not point out. Yes, the library maintainer could not fix the vulnerability and asked for the incident team (Nozomi Networks & CERT/CC) to go full disclosure.
Why worry about DNS “Cache Poison’ Attacks?
DNS Cache Poison is a technique that has been used for decades. It is not very common but is used and often happens with threat-actors working to get long-term footholds in critical infrastructure (i.e .Nozomi Networks testing). Do you need a DNS “Cache Poison” Attach Refresher? Dr. Mike Pound’s eloquent video on DNS Cache Poisoning is a good refresher for anyone who is not clear at this point.
Who gets targeted with DNS Cache Poison Attacks?
Normally, the organization that is getting the DNS Cache Poison brute force is the “target.” The DNS Authoritative “zone owner” is the collateral damage. CPEs, IoT devices, ICS elements, and other devices inside an organization’s network would be the objective of the DNS Cache Poison.
Let us walk through an illustration (see graphic). A company “example.com” built an environmental IoT monitor that is deployed in organizations around the world. The threat actor would pick one of those organizations to target. Their goal is to get a man-in-the-middle attack set up to replace the firmware in the environmental IoT monitor. They know the IoT device has the vulnerability highlighted by Nozomi Networks & CERT/CC. They also know through observation the timing for the IoT devices’ “phone home” to see if there are new updates. Many times, these “phone home” updates happen at the same time. This allows the miscreant to have a window of exploitability to brute force the DNS Cache Poison.
If the poison is successful, the threat-actor would be able to replace a DNS record. Think of “softwareupdate.example.com” as the target for poisoning. The threat-actor changes the IP with their IP, allowing the IOT device to download their version of code.
The “target” in this case is the organization with the IoT device. The IoT’s code replacement creates a foothold in the organization.
How does Nozomi Networks Update the Risk?
- This uClibc and uClibc-ng have been in the code for a long while.
- The vulnerability is easily found with basic open source tools looking for ways to penetrate devices.
- Most organizations that would have vulnerable devices are NOT normally protecting their devices from these types of attacks.
- Most organizations with these vulnerable devices are normally NOT monitoring DNS Cache Poison attempts.
- Nozomi disclosed the vulnerability to more than 200 vendors with a 30-day notice before the public release.
- Nozomi Networks did not highlight the trigger for the DNS Cache Poison. The “trigger” is a classic DNS Cache Poison is something that gets the client to do a DNS query. That could be an application, spam, phish, an advertisement, or anything else to get the client to send out the DNS query.
- The DNS Transaction ID and Source Ports only work for one round trip query. The DNS Cache Poison Brute force needs to happen within the DNS Query Round Trip Time (RTT). In this example, the response for “softwaredownload.example.com” takes 100 ms. The Threat-Actor triggers a device to ask for that zone that then has the DNS Resolver do the DNS query. The brute force then hits the DNS resolver with the flood of packets trying to get in before the DNS query is answered by the authoritative server.
- What if this vector requires “no trigger?” As highlighted in the illustration, many IoT and ICS devices are very predictable. They are programs to “just operate.” Plug a device into a test network with Wireshark and watch. You will find the pattern of behavior. That pattern of behavior might be what a threat-actor needs to orchestrate a DNS Cache Poison using the patterns as the triggers.
DNS Cache Poison Detection – It is Noisy with Backscatter
DNS Cache Poison is noisy. Traditional DNS Cache Poison attacks need to “trigger” a lookup on the DNS Resolver (or proxy in this case). If the trigger succeeds, you then have a “race” flooding the device with packets trying to match the right port, DNS Transaction ID, and other fields. This match has to succeed before the legitimate DNS query response arrives. Yes, Nozomi found that the DNS Transaction ID and the source port are easier to predict, but it still requires brute force.
DNS Poison only works for that DNS lookup (unless you catch an NS update). So you end up hammering over and over.
In the meantime, all the “invalid” ports get an ICMP Port Unreachable heading to the spoofed DNS Authoritative server. The DNS Authoritative server just needs to look for these floods of ICMP messages to know there is an attack on the way. In our illustration, example.com would know someone trying to poison their zone.
Back in the “Kaminsky Poison” many organizations turned on Netflow in front of their DNS Authoritative servers. They then set up alarms with there was a flood of ICMP Port Unreachable. A few organizations used this to detect unknown DNS Cache Poison attacks on DNS Resolvers in different parts of the world.
Protecting Devices from DNS Cache Poison Attacks
Forcing everyone in the organization to use authorized DNS Resolvers is one way to add “Poison resiliency.” An Critical Infrastructure Organization would block all DNS queries from inside the organization. The only DNS Resolvers allowed would be those under the control of the IT/Security teams. DNS Cache Poison then becomes harder. Threat-Actors would need to spoof the internal DNS Resolvers to attempt to poison. Poison attempts would look like DDoS attacks and alarm the organization. In essence, the ability to DNS Cache Poison would be harder with this simple security architectural approach limited DNS queries in/out of the organization to authorized and approved DNS Resolvers.
Most ISPs/CSPs have their CPEs set up to have them use DHCP configured DNS Resolvers. Granted, individuals can change the DNS settings on their CPEs, but that number will be small – limiting the risk’s surface area.
Ingress ACLs on the iACL
One of the key defense tools that many organization neglects is a simple router ACL (see illustration):
- Ingress ACL Policy: Block all Source IP from the Internet = to our CIDR Block
- Egress ACL Policy: Block all Source IP from our Network = anything other than our CIDR block
In this illustration, the organization has an ingress ACL (traffic from the Internet) that blocks all IPs whose source IP equals the CIDR bock used in the organization. This prevents the threat-actor from spoofing into the organization to launch things like “DNS Cache Poisoning. At the same time, a simple EGRESS ACL (traffic from the organization’s network) only permits source IPs from that organization’s CIDR block. This is a decades-old BCP that in recent years has been neglected. You can combine this simple technique with Filtering Exploitable Ports and limit the DNS risk by controlling DNS traffic into and out of your network
Control DNS Traffic Into and Out of Your Network
Policy Recommendation: Block DNS traffic from inside your network, allowing only approved DNS Resolver and DNS Authoritative servers. All over devices must use the official DNS Resolver(s) as their “DNS Forwarder.”
Our objective is to tighten the policies in your network to have your devices all use approved, monitored, and controlled DNS Resolvers. It makes it harder for the threat actors.
Having all your devices funnel through controlled DNS Resolvers allows you to turn your DNS Resolver into a security tool. It can then be used for DNS Poison resiliency along with SPAM, Malware, Phishing, and other security protections.
Protecting Zones (Authoritative DNS) from DNS Cache Poison
Monitoring for ICMP Port Unreachable, reducing the DNS Query Round Trip Time (RTT), DNSSEC on critical domains, and multiplying the nameservers help detect and resist DNS Cache Poison attempts. DNSSEC is the other option, with impact limited by DNS Resolvers configuring DNSSEC validation.
ICMP Backscatter to Alert on DNS Cache Poison Attacks
As pointed out, DNS Cache Poison attacks are nosy, generating close to a 1:1 ICMP Port Unreachable to the “source.” That “source” would be one or more of the name servers. This is “noise” is often referred to as “ICMP Backscatter.” There are a variety of tools to monitor ICMP Backscatter. The key is to monitor for when it happens, track down the DNS Resolvers under DNS Cache Poison attack, and investigate.
The key is to investigate. For example, one company used Netflow to alarm when there is a surge of ICMP Port Unreachable packets. That investigation uncovered an APT threat actor who was trying to get their version of software on hardware devices. While DNS Cache Poison attacks are not common, they do happen and usually are linked to craft threat-actors.
Fast DNS Query RTT Makes it Harder for DNS Cache Poison
DNS Cache Poison works by having a “trigger” that kicks off a DNS Query. The Threat-Actor then has to flood the target with poison attempt to try to match the DNS Transaction ID and Source Port. The longer the DNS query RTT the long the threat-actor has to succeed. Reducing the RTT by deploying more DNS authoritative servers, using cloud DNS authoritative infrastructure, and DNS edge authoritative infrastructure all reduce the DNS query RTT. This RTT reduction help with the end-user experience while making it harder for DNS cache poising to succeed.
DNSSEC Missing from Cache Poison Defence?
DNSSEC will resist DNS Cache Poison attacks. But, as seen in the illustration below, DNSSEC validation is required on the DNS Resolver to be effective against DNS Poison attacks. What this means is that DNSSEC is only effective if the DNS Resolvers turn on DNSSEC Validation. Remember, DNSSEC has a cost per zone. It requires operational time and effort to maintain. When organizations have thousands of zones, DNSSEC is a none trivial operational expense in organizations that traditionally neglect their critical DNS infrastructure.
What is encouraging is APNIC Labs Data that illustrates DNSSEC Validation is on the rise. The work started in 2019 (see DNSSEC Validation (Revisited)) and provided community data on DNSSEC validation in the world DNS Resolvers. The trend is positive. Check out https://stats.labs.apnic.net/dnssec to get an update on the number of DNS Resolvers in your country that APNIC Labs is tracking with “DNSSEC Validation” turned on.
Yes, DNSSEC is still expensive. The good news – any organization can track which DNS Resolvers asking for your zone information is DNSSEC validation. The DNSSEC Flag is set to D when a query from a DNS Resolver is sent to a zone’s authoritative servers. A script in the logs can match the number of DNS Resolves to the “DNSSEC Request.” A high percentage will illustrate the return on investment for the additional operational DNSSEC costs. Additionally, if you compute the DNS Resolvers using DNSSEC to a per zone analysis, you come up with a cost-risk-benefit analysis to see if DNSSEC is worth the cost for each zone.
Multiple Nameservers Makes it Harder to Poison
Authoritative DNS Infrastructure can make DNS Cache Poisoning harder with multiple nameservers. Today’s DNS Resolvers USE all the nameservers. Different DNS Resolver software has techniques to find the fastest DNS Resolver(s), use those, then periodically “heartbeat check” to see if there is a change.
Nozomi Networks and the library maintainer are to be commended for transparent responsible disclosure. They asked for ICS-ISAC’s help, then rotated over to CERT-CC to get help. Yes, we have an industry exposure to this vulnerability. But, we also have workarounds that add DNS Cache Poison detection and resiliency.
Alias, all is not well until all the devices are updated to resist DNS cache poisoning. Given this, have a conversation with your vendors. Ask them how they are testing to resist DNS Cache Poison attacks.
The Action CheckList to help you minimize the risk of DNS Cache Poisoning:
- Deploy Ingress “Anti-Spoof” and Exgress “Anti-Spoof” ACLs on your Network Edge to minimize the risk.
- Route all DNS Traffic in your networks through official and controlled DNS Resolvers. Turn the DNS Resolver into a Security Tool
- Turn on DNSSEC Validation on your DNS Resolvers.
- Monitor for DNS Cache Poison Backscatter with tools like Netflow to monitor for ICMP Port Unreachable Floods.
- Move your DNS Authoritative closer to your customers with a Edge Based DNS Platform.
- Choose a DNS Authoritative Edge Platform with 6+ nameserver options to make it harder for the DNS Cache Poison threat actor.
- Consider DNSSEC deployment on critical DNS Zones. Use the DNS Logs to map how many of your customer’s DNS Resolvers have DNSSEC Validation turned on.
Are you looking for more practical, low-cost security Advice?
- You can sign up to the mailing list for updates here: Stay Connected with Senki’s Updates.
- Subscribe to Senki’s YOUTUBE Channel for videos on this and other security topics.
- Ask questions to Barry Greene – firstname.lastname@example.org
The materials and guides posted on www.senki.org here are designed to help organizations leverage the talent around them to get started with their security activities. Start with the Operator’s Security Toolkit and Meaningful Security Conversations with your Vendors. Each is no-nonsense security for all Operators. It provides details to help them build more security resilient networks. In the meantime, stay connected to the Senki Community to get updates on new empowerment and security insights.