# DNS Load Balance

# Context

DNS, or Domain Name System, is the yellow page of the Internet. When people access to the website through domain name enqueuezero.com, the web browser interacts with DNS server first to get the IP addresses for dialing.

Load Balance is essential for distributing traffic across multiple hosts in the server farm.

DNS Load Balancing provides the most straightforward load balancing strategy through DNS servers.

# Overview

DNS load balancing is an implementation of configuring the IP addresses of the domain name as multiple A records. As a result, the requests from the client are distributed across a group of server machines.

For example, enqueuezero.com (opens new window) has 4 IP addresses that can serve requests -,,, We can configure multiple A records for enqueuezero.com in the domain name provider. In this case, it's Google Domains. Below screenshot shows how to configure A records in Google Domains:

Google Domains

Below command shows, the DNS interrogation gets 4 IP addresses.

% dig enqueuezero.com

; <<>> DiG 9.10.6 <<>> enqueuezero.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61900
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

; EDNS: version: 0, flags:; udp: 512
;enqueuezero.com.        IN    A

enqueuezero.com.    599    IN    A
enqueuezero.com.    599    IN    A
enqueuezero.com.    599    IN    A
enqueuezero.com.    599    IN    A

;; Query time: 260 msec
;; WHEN: Tue Dec 25 07:55:24 NZDT 2018
;; MSG SIZE  rcvd: 108

To prove such configuration can distribute traffics, let's ping the domain in different servers. As you can see, the ping command talks to servers in different IP addresses based on where the ping starts, as the DNS server returns the host that responds the fastest.

# From Auckland, NZ
$ ping -c 1 enqueuezero.com
PING enqueuezero.com ( 56 data bytes
64 bytes from icmp_seq=0 ttl=60 time=43.267 ms
# From CA, US
$ ping -c 1 enqueuezero.com
PING enqueuezero.com ( 56 data bytes
64 bytes from icmp_seq=0 ttl=60 time=52.162 ms

Short conclusion: By configuring multiple A records for the domain, DNS load balancing can distribute traffic across multiple server hosts.

# Solutions

# Terraform

Terraform DNS provider (opens new window) supports any DNS provider that supports DNS updates (RFC 2136).

You can apply multiple IP addresses to the resource dns_a_record_set like below.

resource "dns_a_record_set" "@" {
  zone = "enqueuezero.com."
  name = "@"
  addresses = [
  ttl = 300

You can also use Terraform AWS route53 (opens new window) if you're using AWS Route53 for load balancing, or Terraform Azure DNS resources (opens new window) if you're using Azure service. Below code shows how to utilize AWS Route53 for DNS load balancing.

resource "aws_route53_record" "www" {
  zone_id = "${aws_route53_zone.primary.zone_id}"
  name    = "enqueuezero.com"
  type    = "A"
  ttl     = "300"
  records = ["${aws_eip.lb.public_ip}"]

# ExternalDNS

github.com (opens new window)

ExternalDNS makes Kubernetes resources discoverable via public DNS servers.

After setting IAM policies and a hosted zone, you can apply below resource to the Kubernetes cluster.

apiVersion: extensions/v1beta1
kind: Deployment
  name: external-dns
    type: Recreate
        app: external-dns
      - name: external-dns
        image: registry.opensource.zalan.do/teapot/external-dns:latest
        - --source=service
        - --source=ingress
        - --domain-filter=external-dns-test.my-org.com # will make ExternalDNS see only the hosted zones matching provided domain, omit to process all available hosted zones
        - --provider=aws
        - --policy=upsert-only # would prevent ExternalDNS from deleting any records, omit to enable full synchronization
        - --aws-zone-type=public # only look at public hosted zones (valid values are public, private or no value for both)
        - --registry=txt
        - --txt-owner-id=my-identifier

When the external-dns pod is up, it will add the Ingress service IP addresses to the DNS in the hosted zone. Check the full tutorial of setting ExternalDNS up: github.com/kubernetes-incubator/external-dns (opens new window)

The reason we can't see IP addresses in this example is the external-dns exposes dynamic public IP addresses of the ingress service assigned by the overlay network IP manager.

# Patterns

# Seeming Round-Robin

You cannot set any round robin algorithms to the scheduler (DNS server) since it is not defined in the DNS protocol. However, the traffic is distributed to the servers in a seeming round-robin way.

The DNS cache on specific DNS server node deviates the stats. It happens when the DNS results are cached on a DNS server, and unfortunately, this DNS server serves more traffic than others. Things like geography, population distribution, city size, infrastructure can affect the

# Performance

There is no much performance issue when the traffic grows when applying the DNS load balancing, since the DNS caching system works so well.

From an operational perspective, engineering teams can collect and monitor the DNS latencies from different regions. The popular tools includes worldPing (opens new window), PingDom (opens new window), etc.

# Scale

When the number of the servers in the farm goes up, we can add more A records to the domain. There is no limitation of the number of the records.

In practical, we can mix using the A record and CNAME group records.

enqueuezero.com.    599    IN    CNAME    lb1.enqueuezero.com
enqueuezero.com.    599    IN    CNAME    lb2.enqueuezero.com
lb1.enqueuezero.com.    599    IN    A
lb1.enqueuezero.com.    599    IN    A
lb2.enqueuezero.com.    599    IN    A
lb2.enqueuezero.com.    599    IN    A

# Fail Over

Setting a proper TTL for DNS records is essential. It's the number of the seconds that the DNS records expire. When one of the backend hosts is down, it's critical to modify the DNS records as soon as possible.

The DDNS or Standard Dynamic update DNS (RFC 2126) enables updates for the DNS records remotely. The other DNS providers might provide HTTP APIs for updating the DNS records.

Tools like ExternalDNS can listen to the Kubernetes API events and automatically updates the DNS records when ingress services get updated.

# Conclusions

DNS load balancing distributes requests across multiple IP addresses by configuring various DNS A records. Modern tools enable programmatically updating DNS records. When the incident happens, some of them can even automatically update the DNS records. The downside of DNS load balancing is that it cannot distribute requests evenly to the backend servers.

# References