About Consul

About Consul

July Tech Festa 2014 slides #techfesta #consul

C21584e232717e771226626c0bc2f30c?s=128

Michael H. Oshita

June 22, 2014
Tweet

Transcript

  1. ABOUT CONSUL 6/22/2014 @ijin July Tech Festa 2014

  2. ABOUT ME (@IJIN) • Michael H. Oshita • Japanese American

    • 10+ Years in Japan • Software/Infra/Cloud Engineer • http://ijin.github.io
  3. ABOUT ME (@IJIN) τϥϒϧˑ͠Ύʔͨʔͣ

  4. τϥϒϧˑ͠Ύʔͨʔͣ #08 http://bit.ly/torash08

  5. None
  6. CONSUL

  7. CONSUL HTTP APIͱDNSͰૢ࡞͕Ͱ͖Δ෼ࢄܕΫϥελ

  8. SERVICE DISCOVERY • ؆୯ʹserviceΛొ࿥͠ɺࣗಈͰΫϥελ্Ͱݕग़Մೳ • DNS·ͨ͸HTTPʹΑͬͯࢀর

  9. FAILURE DETECTION • ϔϧενΣοΫػೳ • ࢮΜͩϊʔυ΁ͷϦΫΤετᷖճ !

  10. MULTI DATACENTER • ෳ਺ͷσʔληϯλʔΛ·͍ͨͰ௨৴ • ଞͷDC্Ͱͷserviceͷࢀর

  11. KEY/VALUE STORE • ಈతͳઃఆ৘ใͷஔ͖৔ॴͱͯ͠ • Long PollingͰઃఆมߋͷଈ࣌௨஌

  12. TECHNOLOGY • Consensus Protocol (Raft, FSM) In Search of an

    Understandable Consensus Algorithm - Stanford University • Gossip Protocol (Serf) SWIM: Scalable Weakly-consistent Infection-style Process Group Membership - Cornel University • Sessions (Chuby) The Chubby Lock Service for Loosely-Coupled Distributed Systems - Google • TLS, Jespen Testing !
  13. ARCHITECTURE

  14. CONSISTENCY

  15. CONSISTENCY default ! ! consistent: ! ! stale: "?state" or

    "?consistent" parameter ɾେମStrongly Consistent ɾ৽Leaderঢ֨࣌ʹstale readͷՄೳੑ͋Γɻ ɾ଎͍ ! ɾStrongly Consistent ɾLeader͸QuromͰ֬ೝ ɾLatency͕ൃੜ͢ΔͷͰएׯ஗Ί ! ɾStale Readɺޡࠩ͸֓Ͷ50ms͙Β͍ ɾLeaderҎ֎ͷServer͕Ԡ౴Մ ɾߴ଎
  16. CONSUL AGENT Nodes & Services foo bar baz redis 01.web

    02.web node service
  17. CONSUL AGENT • DNS Interface • HTTP API • Health

    Check !
  18. CONSUL AGENT DNS Interface <node>.node.<datacenter>.<domain> Node Lookup:

  19. CONSUL AGENT DNS Interface

  20. CONSUL AGENT DNS Interface <tag>.<service>.service<datacenter>.<domain> Service Lookup:

  21. CONSUL AGENT DNS Interface

  22. CONSUL AGENT DNS Interface

  23. CONSUL AGENT DNS CACHING Stale Reads ɾࢀর͸σϑΥϧτͰLeader NodeͷΈ ɾ༗ޮԽ͢ΔͱଞͷServer Node͔ΒࢀরՄೳ

    ɾεϧʔϓοτ޲্ ɾmax_staleͰ࣌ؒΛઃఆ ! TTL ɾDNS TTL͸σϑΥϧτͰ0 ɾdnsmasq౳ͰΩϟογϡՄೳʹ { "dns_config": { "service_ttl": { "*": "5s", "web": "30s" } } new! (0.3.0)
  24. CONSUL AGENT HTTP API /v1/kv/<key> key/value endpoint: RESTful: GET, PUT,

    DELETE
  25. CONSUL AGENT HTTP API key/value: base64 raw

  26. CONSUL AGENT HTTP API /v1/agent/checks /v1/agent/services /v1/agent/members /v1/agent/self /v1/agent/join/<address> /v1/agent/force-leave/<node>

    /v1/agent/service/deregister/<serviceID> agent endpoint: /v1/agent/register /v1/agent/deregister/<checkID> /v1/agent/pass/<checkID> /v1/agent/warn/<checkID> /v1/agent/fail/<checkID> /v1/agent/service/register
  27. CONSUL AGENT HTTP API /v1/catalog/register /v1/catalog/deregister /v1/catalog/datacenters /v1/catalog/nodes catalog endpoint:

    /v1/catalog/services /v1/catalog/service/<service> /v1/catalog/node/<node>
  28. CONSUL AGENT HTTP API /v1/health/node/<node> /v1/health/checks/<service> /v1/health/service/<service> /v1/state/<state> health endpoint:

  29. CONSUL AGENT HEALTH CHECK { "service": { "check": { "script":

    "curl localhost > /dev/null 2>&1", "interval": "10s" } } check definition:
  30. CONSUL AGENT HEALTH CHECK Exit Code 0 - Check is

    passing ! Exit Code 1 - Check is warning ! Any Other - Check is failing check script:
  31. CONSUL AGENT HEALTH CHECK

  32. BLOCKING QUERIES ɾLong PollingΛ࢖ͬͯঢ়ଶมߋΛଈ࣌ݕ஌ ! ɾԠ౴͸λΠϜΞ΢τ͔࣌ҟৗൃੜ࣌ ! ?wait=<interval>&index=<idx>

  33. USE CASES ɾInternal DNS ! ɾInventory Management ! ɾFailover

  34. DEMO

  35. BENCHMARK GET 8 CPU Cores, 2Ghz 16GB RAM 160GB SSD

    disk 1Gbps NIC Requests/sec rps 0 2750 5500 8250 11000 10,469.54 1,054.031 v0.2.0 v0.3.0
  36. BENCHMARK GET response time distribution # of responses 0.00 3000.00

    6000.00 9000.00 12000.00 response time (s) 0.001 0.007 0.014 0.017 0.023 0.030 0.033 0.062 0.094 0.125 0.156 v0.2.0 v0.3.0 8 CPU Cores, 2Ghz 16GB RAM 160GB SSD disk 1Gbps NIC
  37. WEB UI

  38. CONSUL-API Library in GO https://github.com/armon/consul-api

  39. IN CLOSING consulΠΠΑʂ