How to hand-write DNS messages
routley.ioOh how many fun things have been skipped! For example, errors in message header are a nibble. There is compression of sorts, and labels referenced by an offset. There is an unique notation of emails in the SOA records; and some domains and nameservers are just plain magic. If you set out to implement a complete DNS resolver, you are IN for a fun, fun, fun time with printouts of quite a few RFCs dating back to 1987 IIRC. But the most fun part is reading these documents and seeing a story - of networks that never came to be, of dawn of the internet, of very much constrained computers and networking equipment and the genius of the thing that still is the root of the internet as we know it even some 30 years later.
I recently wrote a simple nonrecursive one in Python to see how many domains I could resolve per second. You quickly hit a limit when you try to use something like Google's 8.8.8.8 server too fast. If you do it recursively starting from root servers, can you go as fast as your bandwidth allows?
I don't think you can give a recursive query to a root server.
Sorry, what I meant with "recursive" was doing the recursion myself, instead of handing it off to a server like Google's DNS server. If I follow the replies starting from root server one by one, am I likely to hit some limit?
It seems unlikely that you were able to overload the Google public dns server, and it also seems unlikely that you were saturating your connection. Thus you probably hit a rate limit. If you go recursive, and cache at least the TLD servers, distributing requests evenly across the 13 logical servers that typically manage a TLD such as .com, you’ll probably see a much higher per second throughput than through the google server, but still constrained by rate limits.
You shouldn't, but as there are multiple root servers you should round robin them instead of hitting the same one every time.
Writing a simple DNS resolver is a common assignment in networking courses, and I do recommend it --- keeping in mind that the protocol was designed to work on machines with a fraction of the memory and CPU power as those today, helps understanding some of the otherwise odd design decisions; for example, requests and replies have the same format and header to allow the same buffer to be reused for both receiving the request and sending the reply. The QCLASS is another field from a time when people thought DNS would be used for networks other than the Internet.
A minor correction: DNS deals only with the hostname or authority component of the URL, so any mention of "URL encoding" doesn't make sense in this context.
I'd say any mention of URL should be avoided for the sake of confusion.
I can't count the number of times I've been asked by a developer running a web server on a random port to "make sure to include the port" on the A record I'm adding.
"The QCLASS is another field from a time when people thought DNS would be used for networks other than the Internet."
Depending on how one reads the RFC, it appears that the "sophomoric"1 compression scheme, which has been the source of numerous exploits/vulnerabilities in DNS software, also related to this thought about other classes besides Internet (IN). In the section on compression, RFC 1035 says that "pointers can be used where the format of the RR is not class specific."
1 https://news.ycombinator.com/item?id=503120
But in practice, are the RRs in todays DNS packets class nonspecific? Other than IN and CH (sometimes used for testing), are there other classes that todays DNS software can be expected to handle? Do todays queries almost always fall into the same type (IN)? How many query classes are there?2
2 https://www.iana.org/assignments/dns-parameters/dns-paramete...
Note even support for the "ANY" class has been the source of controversy.
Compression is what makes writing DNS software more difficult than it needs to be. However, according to one authors reading of the RFC, compression is only "required" in recursive DNS servers, not authoritative ones.3
3 https://cr.yp.to/djbdns/notes.html
"DNS, however, was not designed by experienced programmers."
This is another reason I prefer using local and remote authoritative servers versus open shared resolvers/recursive caches, such as the ones run by third parties such as Google, Cisco or various ISPs.
No (sophomoric) compression needed. Whatever the reason was for having a method of compression in DNS packets4, I personally do not need it.
4 The blog post: "QDCOUNT: An unsigned 16 bit integer specifying the number of entries in the question section. We'll be sending 1 question."
Take a guess what would happen if he sent more than one question. The RFC just says the qdcount is "usually" one.
RFC 1035 is written for a future that never happened. (For better or worse.)
I was going to post complaining about that very point. Any discussion of URL is irrelevant and misleading in the context of raw DNS queries for beginners. I think the author incorrectly assumes domain name is interchangeable with URL for some reason.