Advanced DNS with Consul Service Discovery

Advanced DNS with Consul Service Discovery

This talk covers why only DNS is not ideal in modern day infrastructure where microservices embrace rapid deployments of applications. HashiCorp Consul let's easily manage microservices with a few service discovery features!

4f1d62c5f8ed9147f4ab05362ca096fe?s=128

Edgaras Apšega

October 24, 2019
Tweet

Transcript

  1. Advanced DNS with Consul Service Discovery Edgaras Apšega Lead IT

    Systems Engineer Adform @apsega Cloud Native Lithuania
  2. $ whoami Edgaras Apšega Systems Engineer @ Adform

  3. Agenda Why DNS is not perfect? Why do we need

    Service Discovery? What problems does HashiCorp Consul solve? Cloud Native Lithuania
  4. DNS

  5. DNS Translates Domain names to IP addresses; Typically A type

    records with TTL.
  6. TERMINAL > dig +short apsega.lt 104.24.102.191 104.24.102.192 > curl -sI

    104.24.102.191 | head -1 HTTP/1.1 200 OK > curl -sI 104.24.102.192 | head -1 HTTP/1.1 503 Service Temporarily Unavailable DNS query DNS balances queries between domain name records; Are all servers under DNS responding correctly?
  7. Service Discovery

  8. Service Discovery Microservices oriented way of communication between services.

  9. Service Discovery Service load balancers aren’t efficient in a dynamic

    world.
  10. Eureka Netflix Open Source Consul HashiCorp Open Source and Enterprise

    ZooKeeper Apache Open Source Most popular products
  11. None
  12. Consul architecture All the agents that are in a datacenter

    participate in a gossip protocol; The servers in each datacenter are all part of a single Raft peer set.
  13. HashiCorp Consul features 1. Service Discovery (DNS with 0 TTL

    or HTTP API with rich metadata); 2. Health Checking (HTTP, TCP, scripts); 3. Prepared Queries (failover to other datacenters based on RTT); 4. Key/Value Store (dynamic configuration, leader election); 5. Service Mesh (auto mTLS, Intentions);
  14. Consul Service Discovery with Health Checks

  15. Service Discovery With Health Checks CODE EDITOR { "service": {

    "name": "webserver", "tags": ["production", “nginx”], "port": 80, "check": { "interval": "5s", "http": "http://localhost:80", "timeout": "1s" } } }
  16. Consul UI Displays health status of nodes

  17. TERMINAL > dig +short webserver.service.consul 10.8.192.72 10.8.192.125 > dig +short

    production.nginx.service.consul 10.8.192.72 > dig +short nginx.service.eu-west-1.consul 10.8.192.125 DNS interface 1. Simple service query; 2. Service query with tags; 3. Service query for specific datacenter.
  18. Consul UI Displays unhealthy nodes

  19. TERMINAL > dig +short webserver.service.consul 10.8.192.125 DNS interface Returns only

    healthy nodes.
  20. Live demo

  21. Consul with load balancers

  22. Consul template A convenient way to populate values from Consul

    into the file
  23. Consul template with Nginx 1. Nginx upstream configuration 2. Upstream

    Consul template CODE EDITOR upstream from_consul { <…> include /opt/consul-template.d/from_consul_nginx_upstreams.conf; <…> } {{ range service ”webserver" }} server {{ .Address }}:{{ .Port }} max_fails=1 fail_timeout=15s weight=1; {{ else }} server 127.0.0.1:80 max_fails=1 fail_timeout=15s weight=1; {{ end }}
  24. Consul with HAProxy CODE EDITOR resolvers mydns nameserver dns1 192.168.1.253:53

    accepted_payload_size 8192 # allow larger DNS payloads backend webservers balance roundrobin server-template web _webserver._tcp.service.consul resolvers mydns check init-addr none
  25. Consul implementation in Adform

  26. DNS forwarding All specific domain queries are forwarded to Consul

    servers
  27. 2,300+ Registered nodes 690+ Registered services 700+ DNS Queries per

    second 11 Clusters 6 Production regions Between 8 datacenters
  28. Thank You! edgaras@apsega.lt @apsega 29 Cloud Native Lithuania