Advanced DNS with Consul Service Discovery

This talk covers why only DNS is not ideal in modern day infrastructure where microservices embrace rapid deployments of applications. HashiCorp Consul let's easily manage microservices with a few service discovery features!

Transcript

1. Advanced DNS with Consul Service Discovery
   Edgaras Apšega
   Lead IT Systems Engineer
   Adform
   @apsega
   Cloud Native Lithuania
   

   View Slide

2. $ whoami
   Edgaras Apšega
   Systems Engineer @ Adform
   

   View Slide

3. Agenda
   Why DNS is not perfect?
   Why do we need Service Discovery?
   What problems does HashiCorp Consul solve?
   Cloud Native Lithuania
   

   View Slide

4. DNS
   

   View Slide

5. DNS
   Translates Domain
   names to IP
   addresses;
   Typically A type
   records with TTL.
   

   View Slide

6. TERMINAL
   > dig +short apsega.lt
   104.24.102.191
   104.24.102.192
   > curl -sI 104.24.102.191 | head -1
   HTTP/1.1 200 OK
   > curl -sI 104.24.102.192 | head -1
   HTTP/1.1 503 Service Temporarily Unavailable
   DNS query
   DNS balances queries
   between domain
   name records;
   Are all servers under
   DNS responding
   correctly?
   

   View Slide

7. Service Discovery
   

   View Slide

8. Service
   Discovery
   Microservices oriented way
   of communication between
   services.
   

   View Slide

9. Service
   Discovery
   Service load balancers
   aren’t efficient in a dynamic
   world.
   

   View Slide

10. Eureka
    Netflix
    Open Source
    Consul
    HashiCorp
    Open Source and Enterprise
    ZooKeeper
    Apache
    Open Source
    Most
    popular
    products
    

    View Slide

11. View Slide

12. Consul
    architecture
    All the agents that are in
    a datacenter participate
    in a gossip protocol;
    The servers in each
    datacenter are all part
    of a single Raft peer set.
    

    View Slide

13. HashiCorp Consul features
    1. Service Discovery (DNS with 0 TTL or HTTP API with rich metadata);
    2. Health Checking (HTTP, TCP, scripts);
    3. Prepared Queries (failover to other datacenters based on RTT);
    4. Key/Value Store (dynamic configuration, leader election);
    5. Service Mesh (auto mTLS, Intentions);
    

    View Slide

14. Consul Service
    Discovery with Health
    Checks
    

    View Slide

15. Service
    Discovery
    With Health Checks
    CODE EDITOR
    {
    "service": {
    "name": "webserver",
    "tags": ["production", “nginx”],
    "port": 80,
    "check": {
    "interval": "5s",
    "http": "http://localhost:80",
    "timeout": "1s"
    }
    }
    }
    

    View Slide

16. Consul UI
    Displays health status of nodes
    

    View Slide

17. TERMINAL
    > dig +short webserver.service.consul
    10.8.192.72
    10.8.192.125
    > dig +short production.nginx.service.consul
    10.8.192.72
    > dig +short nginx.service.eu-west-1.consul
    10.8.192.125
    DNS
    interface
    1. Simple service query;
    2. Service query with
    tags;
    3. Service query for
    specific datacenter.
    

    View Slide

18. Consul UI
    Displays unhealthy nodes
    

    View Slide

19. TERMINAL
    > dig +short webserver.service.consul
    10.8.192.125
    DNS
    interface
    Returns only healthy nodes.
    

    View Slide

20. Live demo
    

    View Slide

21. Consul with load
    balancers
    

    View Slide

22. Consul template
    A convenient way to populate values from Consul into the file
    

    View Slide

23. Consul
    template
    with Nginx
    1. Nginx upstream
    configuration
    2. Upstream Consul
    template
    CODE EDITOR
    upstream from_consul
    {
    
    include /opt/consul-template.d/from_consul_nginx_upstreams.conf;
    
    }
    {{ range service ”webserver" }}
    server {{ .Address }}:{{ .Port }} max_fails=1 fail_timeout=15s weight=1;
    {{ else }}
    server 127.0.0.1:80 max_fails=1 fail_timeout=15s weight=1;
    {{ end }}
    

    View Slide

24. Consul with HAProxy
    CODE EDITOR
    resolvers mydns
    nameserver dns1 192.168.1.253:53
    accepted_payload_size 8192 # allow larger DNS payloads
    backend webservers
    balance roundrobin
    server-template web _webserver._tcp.service.consul resolvers mydns check init-addr none
    

    View Slide

25. Consul implementation
    in Adform
    

    View Slide

26. DNS forwarding
    All specific domain queries are forwarded to Consul servers
    

    View Slide

27. 2,300+
    Registered nodes
    690+
    Registered services
    700+
    DNS Queries per second
    11
    Clusters
    6
    Production
    regions
    Between 8
    datacenters
    

    View Slide

28. Thank You!
    edgaras@apsega.lt
    @apsega
    29
    Cloud Native Lithuania
    

    View Slide