Advanced DNS with Consul Service Discovery

This talk covers why only DNS is not ideal in modern day infrastructure where microservices embrace rapid deployments of applications. HashiCorp Consul let's easily manage microservices with a few service discovery features!

Transcript

1. Advanced DNS with Consul Service Discovery Edgaras Apšega Lead IT Systems Engineer Adform @apsega Cloud Native Lithuania
2. $ whoami Edgaras Apšega Systems Engineer @ Adform
3. Agenda Why DNS is not perfect? Why do we need Service Discovery? What problems does HashiCorp Consul solve? Cloud Native Lithuania
4. DNS
5. DNS Translates Domain names to IP addresses; Typically A type records with TTL.
6. TERMINAL > dig +short apsega.lt 104.24.102.191 104.24.102.192 > curl -sI 104.24.102.191 | head -1 HTTP/1.1 200 OK > curl -sI 104.24.102.192 | head -1 HTTP/1.1 503 Service Temporarily Unavailable DNS query DNS balances queries between domain name records; Are all servers under DNS responding correctly?
7. Service Discovery
8. Service Discovery Microservices oriented way of communication between services.
9. Service Discovery Service load balancers aren’t efficient in a dynamic world.
10. Eureka Netflix Open Source Consul HashiCorp Open Source and Enterprise ZooKeeper Apache Open Source Most popular products
12. Consul architecture All the agents that are in a datacenter participate in a gossip protocol; The servers in each datacenter are all part of a single Raft peer set.
13. HashiCorp Consul features 1. Service Discovery (DNS with 0 TTL or HTTP API with rich metadata); 2. Health Checking (HTTP, TCP, scripts); 3. Prepared Queries (failover to other datacenters based on RTT); 4. Key/Value Store (dynamic configuration, leader election); 5. Service Mesh (auto mTLS, Intentions);
14. Consul Service Discovery with Health Checks
15. Service Discovery With Health Checks CODE EDITOR { "service": { "name": "webserver", "tags": ["production", “nginx”], "port": 80, "check": { "interval": "5s", "http": "http://localhost:80", "timeout": "1s" } } }
16. Consul UI Displays health status of nodes
17. TERMINAL > dig +short webserver.service.consul 10.8.192.72 10.8.192.125 > dig +short production.nginx.service.consul 10.8.192.72 > dig +short nginx.service.eu-west-1.consul 10.8.192.125 DNS interface 1. Simple service query; 2. Service query with tags; 3. Service query for specific datacenter.
18. Consul UI Displays unhealthy nodes
19. TERMINAL > dig +short webserver.service.consul 10.8.192.125 DNS interface Returns only healthy nodes.
20. Live demo
21. Consul with load balancers
22. Consul template A convenient way to populate values from Consul into the file
23. Consul template with Nginx 1. Nginx upstream configuration 2. Upstream Consul template CODE EDITOR upstream from_consul { <…> include /opt/consul-template.d/from_consul_nginx_upstreams.conf; <…> } {{ range service ”webserver" }} server {{ .Address }}:{{ .Port }} max_fails=1 fail_timeout=15s weight=1; {{ else }} server 127.0.0.1:80 max_fails=1 fail_timeout=15s weight=1; {{ end }}
24. Consul with HAProxy CODE EDITOR resolvers mydns nameserver dns1 192.168.1.253:53 accepted_payload_size 8192 # allow larger DNS payloads backend webservers balance roundrobin server-template web _webserver._tcp.service.consul resolvers mydns check init-addr none
25. Consul implementation in Adform
26. DNS forwarding All specific domain queries are forwarded to Consul servers
27. 2,300+ Registered nodes 690+ Registered services 700+ DNS Queries per second 11 Clusters 6 Production regions Between 8 datacenters
28. Thank You! edgaras@apsega.lt @apsega 29 Cloud Native Lithuania