Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Advanced DNS with Consul Service Discovery

Advanced DNS with Consul Service Discovery

This talk covers why only DNS is not ideal in modern day infrastructure where microservices embrace rapid deployments of applications. HashiCorp Consul let's easily manage microservices with a few service discovery features!

Edgaras Apšega

October 24, 2019
Tweet

More Decks by Edgaras Apšega

Other Decks in Technology

Transcript

  1. Advanced DNS with Consul Service Discovery
    Edgaras Apšega
    Lead IT Systems Engineer
    Adform
    @apsega
    Cloud Native Lithuania

    View full-size slide

  2. $ whoami
    Edgaras Apšega
    Systems Engineer @ Adform

    View full-size slide

  3. Agenda
    Why DNS is not perfect?
    Why do we need Service Discovery?
    What problems does HashiCorp Consul solve?
    Cloud Native Lithuania

    View full-size slide

  4. DNS
    Translates Domain
    names to IP
    addresses;
    Typically A type
    records with TTL.

    View full-size slide

  5. TERMINAL
    > dig +short apsega.lt
    104.24.102.191
    104.24.102.192
    > curl -sI 104.24.102.191 | head -1
    HTTP/1.1 200 OK
    > curl -sI 104.24.102.192 | head -1
    HTTP/1.1 503 Service Temporarily Unavailable
    DNS query
    DNS balances queries
    between domain
    name records;
    Are all servers under
    DNS responding
    correctly?

    View full-size slide

  6. Service Discovery

    View full-size slide

  7. Service
    Discovery
    Microservices oriented way
    of communication between
    services.

    View full-size slide

  8. Service
    Discovery
    Service load balancers
    aren’t efficient in a dynamic
    world.

    View full-size slide

  9. Eureka
    Netflix
    Open Source
    Consul
    HashiCorp
    Open Source and Enterprise
    ZooKeeper
    Apache
    Open Source
    Most
    popular
    products

    View full-size slide

  10. Consul
    architecture
    All the agents that are in
    a datacenter participate
    in a gossip protocol;
    The servers in each
    datacenter are all part
    of a single Raft peer set.

    View full-size slide

  11. HashiCorp Consul features
    1. Service Discovery (DNS with 0 TTL or HTTP API with rich metadata);
    2. Health Checking (HTTP, TCP, scripts);
    3. Prepared Queries (failover to other datacenters based on RTT);
    4. Key/Value Store (dynamic configuration, leader election);
    5. Service Mesh (auto mTLS, Intentions);

    View full-size slide

  12. Consul Service
    Discovery with Health
    Checks

    View full-size slide

  13. Service
    Discovery
    With Health Checks
    CODE EDITOR
    {
    "service": {
    "name": "webserver",
    "tags": ["production", “nginx”],
    "port": 80,
    "check": {
    "interval": "5s",
    "http": "http://localhost:80",
    "timeout": "1s"
    }
    }
    }

    View full-size slide

  14. Consul UI
    Displays health status of nodes

    View full-size slide

  15. TERMINAL
    > dig +short webserver.service.consul
    10.8.192.72
    10.8.192.125
    > dig +short production.nginx.service.consul
    10.8.192.72
    > dig +short nginx.service.eu-west-1.consul
    10.8.192.125
    DNS
    interface
    1. Simple service query;
    2. Service query with
    tags;
    3. Service query for
    specific datacenter.

    View full-size slide

  16. Consul UI
    Displays unhealthy nodes

    View full-size slide

  17. TERMINAL
    > dig +short webserver.service.consul
    10.8.192.125
    DNS
    interface
    Returns only healthy nodes.

    View full-size slide

  18. Consul with load
    balancers

    View full-size slide

  19. Consul template
    A convenient way to populate values from Consul into the file

    View full-size slide

  20. Consul
    template
    with Nginx
    1. Nginx upstream
    configuration
    2. Upstream Consul
    template
    CODE EDITOR
    upstream from_consul
    {
    <…>
    include /opt/consul-template.d/from_consul_nginx_upstreams.conf;
    <…>
    }
    {{ range service ”webserver" }}
    server {{ .Address }}:{{ .Port }} max_fails=1 fail_timeout=15s weight=1;
    {{ else }}
    server 127.0.0.1:80 max_fails=1 fail_timeout=15s weight=1;
    {{ end }}

    View full-size slide

  21. Consul with HAProxy
    CODE EDITOR
    resolvers mydns
    nameserver dns1 192.168.1.253:53
    accepted_payload_size 8192 # allow larger DNS payloads
    backend webservers
    balance roundrobin
    server-template web _webserver._tcp.service.consul resolvers mydns check init-addr none

    View full-size slide

  22. Consul implementation
    in Adform

    View full-size slide

  23. DNS forwarding
    All specific domain queries are forwarded to Consul servers

    View full-size slide

  24. 2,300+
    Registered nodes
    690+
    Registered services
    700+
    DNS Queries per second
    11
    Clusters
    6
    Production
    regions
    Between 8
    datacenters

    View full-size slide

  25. Thank You!
    [email protected]
    @apsega
    29
    Cloud Native Lithuania

    View full-size slide