DNS Amplification Attack
There are a lot of “misconfigured” DNS servers out there. Most DNS servers should just be answering for their own domain, or to their own network. If they get asked (by) anything else, they should be firm and say “Denied.” This is damn hard to set up. My own DNS server, the one that so kindly provided your browser with the wherewithal to even be looking here at this page, was also set up to resolve the entire internet to all comers. This was of course experimental, and absolutely nothing to do with me misconfiguring it. This experiment did not yield a lot of data, though. Ocassionally someone would come and use the DNS server to bypass their ISP’s in order to do something criminal, mainly click fraud. Then there was someone in New York who had me resolve a lot of banks’ IP addresses which looked plain illegal so I banned him/her. Other than that, it was a calm backwater.
Here is one of my cats, also failing to get her head round loading syslogs into OpenOffice Calc (?!?).
One day in January, however, all connection with tinternets was lost here. This is a major disaster, we cannot be without tinternets for more than 2 minutes. Therefore an investigation was required. The router was given a reboot and packet sniffers were set full throttle ahead. It transpired that there was an unusual level of DNS query going on. The really annoying thing about this was the fact that whereas the DNS server was serving the answers to these queries at breakneck speed, whilst running dozens of other programs, making coffee and doing her hair yet still with CPU time to spare (this is a Coppermine, for heaven’s sake), it was the little router, doing nothing but port forwarding, that was becoming unresponsive and falling over in the onslaught. The industrial grade router called Scary had to be replaced a while back by a consumer-grade D-Link called Wysiwyg. This saves about 70 Watt, and it shows.
With amended iptables excluding the “source” of the unusual level of querying, tinternets came back and it was investigation time.
First problem was the fact that my server was actually not giving the “refused” response. This has always been so much fun before. For instance, Google comes regularly to check whether I am resolving google-related domains correctly. At least this is what I surmise they are doing. Some supercomputer in San Diego keeps tabs on open resolvers, as does the Internet Systems Consortium, Inc from Redwood.
The query “. ns” is very short, the answer is quite long. Here is my ISP’s response to it.
; <<>> DiG 9.2.5 <<>> @ns1.bethere.co.uk . ns ; (1 server found) ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23031 ;; flags: qr rd ra; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 14
;; QUESTION SECTION: ;. IN NS
;; ANSWER SECTION: . 253561 IN NS k.root-servers.net. . 253561 IN NS l.root-servers.net. . 253561 IN NS m.root-servers.net. . 253561 IN NS a.root-servers.net. . 253561 IN NS b.root-servers.net. . 253561 IN NS c.root-servers.net. . 253561 IN NS d.root-servers.net. . 253561 IN NS e.root-servers.net. . 253561 IN NS f.root-servers.net. . 253561 IN NS g.root-servers.net. . 253561 IN NS h.root-servers.net. . 253561 IN NS i.root-servers.net. . 253561 IN NS j.root-servers.net.
;; ADDITIONAL SECTION: a.root-servers.net. 599163 IN A 188.8.131.52 a.root-servers.net. 430944 IN AAAA 2001:503:ba3e::2:30 b.root-servers.net. 432123 IN A 184.108.40.206 c.root-servers.net. 432123 IN A 220.127.116.11 d.root-servers.net. 432123 IN A 18.104.22.168 e.root-servers.net. 432123 IN A 22.214.171.124 f.root-servers.net. 432123 IN A 126.96.36.199 f.root-servers.net. 444802 IN AAAA 2001:500:2f::f g.root-servers.net. 432123 IN A 188.8.131.52 h.root-servers.net. 432123 IN A 184.108.40.206 h.root-servers.net. 428689 IN AAAA 2001:500:1::803f:235 i.root-servers.net. 432123 IN A 220.127.116.11 j.root-servers.net. 514703 IN A 18.104.22.168 j.root-servers.net. 430944 IN AAAA 2001:503:c27::2:30
;; Query time: 34 msec ;; SERVER: 22.214.171.124#53(126.96.36.199) ;; WHEN: Fri Feb 6 14:21:28 2009 ;; MSG SIZE rcvd: 500
As you can see, the message size is 500. A query for “google.com ns” is only 167. A “refused” response is only 17 long. Add a little for the overheads & such, that’s still a little packet. My server was sending a slightly shorter version of that long packet (size 272), but clearly often enough for the router not to like it.
The first IP to seemingly be asking for this big packet belonged to an outfit called ISPRIME. They appeared to be a not particularly choosy hosting firm.
Before long, there’s a host of victims. For reference purposes I include my list. There’s a couple of days that haven’t been logged, or only partially logged because of system failures, so there’s no particular value in this list. But I have it, so it may as well be here.
188.8.131.52 Jan 18,19,20,21,22,23,25 184.108.40.206 Jan 18,19,20,21,22,23 220.127.116.11 Jan 18,19,20,21,22,23,25,26 18.104.22.168 same
22.214.171.124 Jan 18, 19, 20, 21, 22, Feb 5, 6 126.96.36.199 Jan 18, 19, 20, Feb 5, 6 (. IN NS); Jan 20, 21, 22 (188.8.131.52.in-addr.arpa IN PTR) 184.108.40.206 Jan 18, 19, 20, 21, 22, Feb 5, 6; Jan 20, 21, 22 PTR 220.127.116.11 Nov 23, 24, 25, Dec 1, 2, 3, 4, 5, 6, 7, Jan 18, 19, 18.104.22.168 Ditto 22.214.171.124 Ditto 126.96.36.199 Ditto (motricity)
188.8.131.52 Jan 18 - PTR on Jan 20, 21, 22 184.108.40.206 ditto 220.127.116.11 ditto 18.104.22.168 ditto
22.214.171.124 Jan 18 - 126.96.36.199 Jan 18 - (abovenet)
188.8.131.52 Jan 23- (pccwglobal.net -- pharma) 184.108.40.206 Jan 27- (quality technolgy -- pharma) 220.127.116.11 Jan 18 -
18.104.22.168 Nov- (internap) 22.214.171.124 Nov-
126.96.36.199 Jan 20 (isprime) 188.8.131.52 Jan 20 (isprime)
184.108.40.206 Jan 27 (pharma)
220.127.116.11 Jan 28 (theplanet.com -- meds, spam)
18.104.22.168 Jan 18 22.214.171.124 Jan 19
126.96.36.199 (internetserviceteam -- pharma, spam)
By far the most of the queries are for the root nameservers, but occasionally there’s some variation.
For instance, 188.8.131.52 (Priority Colo) and 184.108.40.206 (XO Communications) (but only these two, and only on 3 consecutive days) get to do a PTR query for my IP address as well. This is a query which should normally be directed at the owner of the IP, my ISP in other words, and would normally be directed there automatically. Tthe fact that it’s not means it’s a specially constructed query. What good this would do, I have no idea. It does not result in a very large packet.
The most interesting queries were made for 220.127.116.11 (theplanet.com) on the 28th of January. Every now and again, in between the barrage of querying for the root servers which represents 99% of all their traffic, are queries for non-domains. By that I mean, there’s no “dot something” part in the query. The server will ask the root servers who to ask about this, and be told that it’s not a good query. It will send “server failure” or something like that back. And that is where it ends. Here are the ones caught in the logs.
Furiously interesting though. There is a pattern here. I never received the same string twice. The centre of the string (here: aaaafwu0000dfaaabaaafb) does not change. There’s some sort of counter going off here. Noone “benefits” though, as the reply to this is not particularly large. Assuming that the computers generating the queries on behalf of 18.104.22.168 are bots, is it perhaps possible that one of them is misconfigured? Whilst running malware, I have seen my viruses do the wrong thing because they were running on Windows 2000 but seemingly exclusively programmed for XP.
The fact that the originating IP address is spoofed makes it hard to discover who is originating the queries. The perpetrator had a list of DNS servers, and there’s several ways of getting or making one of those. On the assumption that he/she already had control of the botnet before the attack began, they could have set the bots to scan. This might be reflected in the logs.
My logs unfortunately only go back to late November, which might not be far enough.
Every couple of hours, an outfit called Motricity (seemingly – after this incident one cannot assume anything anymore about UDP packets) queries for the root servers.
Another strange one is 22.214.171.124 which on November 29, December 11 and January 13, 14, 16, 22, 25, 28 and 29 submitted queries for rather random domains. The scariest one was for www.vimicro.com, on account that this happened just as I was installing my webcam, which has a Vimicro chipset. (I have syslog scrolling along in real-time most of the time.) Another query was for www.mypcera.com.
At least the queries always originate from UDP port 53, unlike the others. Whois yields:
descr: Beijing CE Huatong Information Technology Co., Ltd.
descr: Block A, Building No.2, No.1, Disheng North Street,
descr: Beijing Economic-Technological Development Area, Beijing
It is quite satisfying to be kicking porno and spam servers off tinternets. Though a multitude of IP addresses were targeted, none were blameless hosts. Most seem disinterested in what they host; from holier-than-thou religious sites to pharma. None were 100% family friendly. However, the urge to enhance the reply, making it as large as could possibly be, was resisted after contemplation of the notion that it was highly unlikely that a group of right-on lefto h4x0rs was aiming their botnet at the reduction of e-filth on tinternets. The most likely cause here is always a competitor, ie. someone who stands to gain financially from the enhanced traffic they’ll get. And to be at the beck and call of another filth merchant is too much to bear. Having iptabled them all away, they don’t even get a “refused” response now.
I note that ISPrime’s ISP has kindly blackholed me though. I can access their website by IP but not name which means that ISPrime is blocking all DNS from me, queries along with replies. They could have kept port 53 open, as this was not used at all during the attack.
My pet hate is websites that try to look professional, but sport spelling mistakes. ISPrime manages this well, as per snippet below.
Not to mention the phrase “Exclusive Hosting Like You” — oops, I’ve mentioned it now. What on earth does it mean? And the mention of “armed security” being present on their hosting site in Amsterdam. Last time I checked, carrying guns and knives was still not actually allowed in the Netherlands. So are these security people doing illegal things? What is hosted there to be in such a great need of defence? Precious porn indeed.
Someone else has posted extracts from logs, showing the same string pattern. ( http://www.simpledns.com/newsitem.aspx?id=2368)
09:53:39 Not authoritative for 'jalbmlaaaafwx0000dfaaabaaaabdcen', sending servfail to 126.96.36.199 09:55:43 Not authoritative for 'nfdincaaaafwx0000dfaaabaaaabjmbh', sending servfail to 188.8.131.52 09:57:48 Not authoritative for 'dcghlcaaaafwx0000dfaaabaaaabakdd', sending servfail to 184.108.40.206 09:57:54 Not authoritative for 'doghfbaaaafwx0000dfaaabaaaabkhpg', sending servfail to 220.127.116.11 09:59:59 Not authoritative for 'ncedhaaaaafwx0000dfaaabaaaabhbcp', sending servfail to 18.104.22.168 10:02:03 Not authoritative for 'mfbodbaaaafwx0000dfaaabaaaabjohc', sending servfail to 22.214.171.124 10:04:08 Not authoritative for 'dcmndmaaaafwx0000dfaaabaaaabobjk', sending servfail to 126.96.36.199 10:06:12 Not authoritative for 'jejlpbaaaafwx0000dfaaabaaaabopoa', sending servfail to 188.8.131.52 10:08:11 Not authoritative for '', sending servfail to 184.108.40.206 (recursion was desired) 10:08:17 Not authoritative for 'kdhjlcaaaafwx0000dfaaabaaaabnpdp', sending servfail to 220.127.116.11 10:10:21 Not authoritative for 'jnlnfbaaaafwx0000dfaaabaaaabfjip', sending servfail to 18.104.22.168 10:12:26 Not authoritative for 'albgbfaaaafwx0000dfaaabaaaabchdd', sending servfail to 22.214.171.124 10:14:30 Not authoritative for 'ohjbbnaaaafwx0000dfaaabaaaabjkph', sending servfail to 126.96.36.199 10:16:35 Not authoritative for 'moennnaaaafwx0000dfaaabaaaabkmae', sending servfail to 188.8.131.52 10:18:40 Not authoritative for 'keopjfaaaafwx0000dfaaabaaaabkpfm', sending servfail to 184.108.40.206 10:20:44 Not authoritative for 'neopgnaaaafwx0000dfaaabaaaabdhld', sending servfail to 220.127.116.11
Whilst digging through my own logs rather more in-depth than usual, I couldn’t help but like these queries. They are from a russian spam server (23mail.ru) with some sense of humour? Very jaunty.
client 18.104.22.168#63532: query: enDellION.me.uk IN MX client 22.214.171.124#27190: query: FEDoRA.endELlIoN.me.UK IN A client 126.96.36.199#14868: query: eNDELLIoN.ME.uK IN MX
Finally there is actually some further reading about this DNS DDoS. Basically hundreds of sites having the same article, here’s a random one.