Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Consul 0.6 and Consul-Template

Consul 0.6 and Consul-Template

Consul is a tool for service discovery, configuration, and monitoring of modern microservices architectures that is broadly deployed and used. In this talk, we will give a brief overview of Consul and the problems it solves and then discuss new features introduced in Consul 0.6. Consul also has a large ecosystem of tooling that makes it easy to integrate into existing infrastructure. One of these tools is consul-template and we will discuss it’s use and new features.

Armon Dadgar

January 19, 2016
Tweet

More Decks by Armon Dadgar

Other Decks in Technology

Transcript

  1. HASHICORP ORDER PROCESSING WEB APP LOAD BALANCING How to ensure

    request leveling across providers? NODE 1 NODE 2 NODE N
  2. HASHICORP ORDER PROCESSING WEB APP ANTI-PATTERN Load Balancer is a

    Single Point of Failure (SPOF) NODE 1 NODE 2 NODE N LOAD BALANCER
  3. HASHICORP ORDER PROCESSING WEB APP HEALTH CHECKING How to avoid

    routing to unhealthy hosts? NODE 1 NODE 2 NODE 3 LOAD BALANCER
  4. HASHICORP WEB APP CONFIGURATION How to efficiently push dynamic configuration?

    WEB 1 WEB 2 WEB N maintenance: false feature_a: true role: "web"
  5. Terminal HASHICORP $ consul rtt node-10-0-1-8 Estimated node-10-0-1-8 <-> node-10-0-1-6

    rtt: 0.781 ms (using LAN coordinates)$ $ sleep 30 $ consul rtt node-10-0-1-8 Estimated node-10-0-1-8 <-> node-10-0-1-6 rtt: 0.719 ms (using LAN coordinates)
  6. Terminal HASHICORP $ curl localhost:8500/v1/catalog/nodes?near=node-78r16zb3q | jq '.[].Node' "node-78r16zb3q" "node-10-0-4-190"

    "node-10-0-1-7" "node-10-0-4-240" $ curl localhost:8500/v1/catalog/service/vault? near=node-78r16zb3q | jq '.[].Node' "node-10-0-1-71" "node-10-0-3-119" "node-10-0-3-249"
  7. Terminal HASHICORP $ dig vault.service.consul ; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>>

    vault.service.consul ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9406 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;vault.service.consul. IN A ;; ANSWER SECTION: vault.service.consul. 0 IN A 10.0.1.71 vault.service.consul. 0 IN A 10.0.3.119 vault.service.consul. 0 IN A 10.0.3.249 ;; Query time: 2 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Fri Dec 11 02:09:16 UTC 2015 ;; MSG SIZE rcvd: 146
  8. Terminal HASHICORP $ dig beta.vault.service.consul ; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>>

    beta.vault.service.consul ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2613 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;beta.vault.service.consul. IN A ;; ANSWER SECTION: beta.vault.service.consul. 0 IN A 10.0.3.119 ;; Query time: 2 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Fri Dec 11 02:09:20 UTC 2015 ;; MSG SIZE rcvd: 126
  9. HASHICORP { "Name": "vault-with-failover", "Service": { "Service": "vault", "Failover": {

    "NearestN": 3 }, "Tags": ["!beta"] }, "DNS": { "TTL": "5s" } }
  10. Terminal HASHICORP $ dig vault-with-failover.query.consul ; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>>

    vault-with- failover.query.consul ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1039 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;vault-with-failover.query.consul. IN A ;; ANSWER SECTION: vault-with-failover.query.consul. 0 IN A 10.0.1.71 vault-with-failover.query.consul. 0 IN A 10.0.3.249 ;; Query time: 2 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Fri Dec 11 02:09:30 UTC 2015 ;; MSG SIZE rcvd: 150
  11. HASHICORP Read Path Optimization LMDB (C) B-Tree API (Go) Go

    <= 1.4, Mark & Sweep LMDB Off-Heap Serialization on boundary
  12. HASHICORP Read Path Optimization MemDB (Go) Immutable Radix API (Go)

    Go >= 1.5, Soft Realtime MemDB On-Heap Pure Go
  13. HASHICORP Enhanced ACLs (0.4) Key / Value Store (0.5) Service

    RegistraFon (0.6) Service Discovery (0.6) User Events (0.6) Keyring Updates
  14. HASHICORP { "check": { "id": "ssh", "name": "SSH TCP on

    port 22", "tcp": "localhost:22", "interval": "10s", "timeout": "1s" } }
  15. HASHICORP { "check": { "id": "mem-util", "name": "Memory utilization", "docker_container_id":

    "f972c95ebf0e", "shell": "/bin/bash", "script": "/usr/local/bin/check_mem.py", "interval": "10s" } }
  16. HASHICORP backend frontend maxconn {{ key "frontend/maxconn" }} balance roundrobin{{range

    service "app.frontend"}} service {{.ID}} {{.Address}}:{{.Port}}{{end}} backend frontend maxconn 256 balance roundrobin server web1 10.0.1.100:80 server web2 10.0.2.200:80
  17. HASHICORP {{ with $service := range services }} backend {{$service.Name}}

    maxconn {{ key (printf “service/%s/maxconn”, $service.Name) }} balance roundrobin{{range service $service.Name}} service {{.ID}} {{.Address}}:{{.Port}}{{end}} 1) Query for all services (1 Query) 2) Query maxconn for each service (N Queries) 3) Query the instances of each service (N Queries) 4) Total of 2*N+1 Queries
  18. HASHICORP Consul 2*N+1 Queries Leader 1 Write 1 Query Followers

    M+2*N Queries 1 Write N = Num Services M = Instances of CT
  19. HASHICORP Number of Services Number of Instances Previous Queries New

    Queries 10 5 105 25 25 10 510 60 50 20 2020 120 100 30 6030 230