About Consul

July Tech Festa 2014 slides #techfesta #consul

Transcript

1. ABOUT CONSUL 6/22/2014 @ijin July Tech Festa 2014
2. ABOUT ME (@IJIN) • Michael H. Oshita • Japanese American • 10+ Years in Japan • Software/Infra/Cloud Engineer • http://ijin.github.io
3. ABOUT ME (@IJIN) τϥϒϧˑ͠Ύʔͨʔͣ
4. τϥϒϧˑ͠Ύʔͨʔͣ #08 http://bit.ly/torash08
6. CONSUL
7. CONSUL HTTP APIͱDNSͰૢ࡞͕Ͱ͖Δ෼ࢄܕΫϥελ
8. SERVICE DISCOVERY • ؆୯ʹserviceΛొ࿥͠ɺࣗಈͰΫϥελ্Ͱݕग़Մೳ • DNS·ͨ͸HTTPʹΑͬͯࢀর
9. FAILURE DETECTION • ϔϧενΣοΫػೳ • ࢮΜͩϊʔυ΁ͷϦΫΤετᷖճ !
10. MULTI DATACENTER • ෳ਺ͷσʔληϯλʔΛ·͍ͨͰ௨৴ • ଞͷDC্Ͱͷserviceͷࢀর
11. KEY/VALUE STORE • ಈతͳઃఆ৘ใͷஔ͖৔ॴͱͯ͠ • Long PollingͰઃఆมߋͷଈ࣌௨஌
12. TECHNOLOGY • Consensus Protocol (Raft, FSM) In Search of an Understandable Consensus Algorithm - Stanford University • Gossip Protocol (Serf) SWIM: Scalable Weakly-consistent Infection-style Process Group Membership - Cornel University • Sessions (Chuby) The Chubby Lock Service for Loosely-Coupled Distributed Systems - Google • TLS, Jespen Testing !
13. ARCHITECTURE
14. CONSISTENCY
15. CONSISTENCY default ! ! consistent: ! ! stale: "?state" or "?consistent" parameter ɾେମStrongly Consistent ɾ৽Leaderঢ֨࣌ʹstale readͷՄೳੑ͋Γɻ ɾ଎͍ ! ɾStrongly Consistent ɾLeader͸QuromͰ֬ೝ ɾLatency͕ൃੜ͢ΔͷͰएׯ஗Ί ! ɾStale Readɺޡࠩ͸֓Ͷ50ms͙Β͍ ɾLeaderҎ֎ͷServer͕Ԡ౴Մ ɾߴ଎
16. CONSUL AGENT Nodes & Services foo bar baz redis 01.web 02.web node service
17. CONSUL AGENT • DNS Interface • HTTP API • Health Check !
18. CONSUL AGENT DNS Interface <node>.node.<datacenter>.<domain> Node Lookup:
19. CONSUL AGENT DNS Interface
20. CONSUL AGENT DNS Interface <tag>.<service>.service<datacenter>.<domain> Service Lookup:
21. CONSUL AGENT DNS Interface
22. CONSUL AGENT DNS Interface
23. CONSUL AGENT DNS CACHING Stale Reads ɾࢀর͸σϑΥϧτͰLeader NodeͷΈ ɾ༗ޮԽ͢ΔͱଞͷServer Node͔ΒࢀরՄೳ ɾεϧʔϓοτ޲্ ɾmax_staleͰ࣌ؒΛઃఆ ! TTL ɾDNS TTL͸σϑΥϧτͰ0 ɾdnsmasq౳ͰΩϟογϡՄೳʹ { "dns_config": { "service_ttl": { "*": "5s", "web": "30s" } } new! (0.3.0)
24. CONSUL AGENT HTTP API /v1/kv/<key> key/value endpoint: RESTful: GET, PUT, DELETE
25. CONSUL AGENT HTTP API key/value: base64 raw
26. CONSUL AGENT HTTP API /v1/agent/checks /v1/agent/services /v1/agent/members /v1/agent/self /v1/agent/join/<address> /v1/agent/force-leave/<node> /v1/agent/service/deregister/<serviceID> agent endpoint: /v1/agent/register /v1/agent/deregister/<checkID> /v1/agent/pass/<checkID> /v1/agent/warn/<checkID> /v1/agent/fail/<checkID> /v1/agent/service/register
27. CONSUL AGENT HTTP API /v1/catalog/register /v1/catalog/deregister /v1/catalog/datacenters /v1/catalog/nodes catalog endpoint: /v1/catalog/services /v1/catalog/service/<service> /v1/catalog/node/<node>
28. CONSUL AGENT HTTP API /v1/health/node/<node> /v1/health/checks/<service> /v1/health/service/<service> /v1/state/<state> health endpoint:
29. CONSUL AGENT HEALTH CHECK { "service": { "check": { "script": "curl localhost > /dev/null 2>&1", "interval": "10s" } } check definition:
30. CONSUL AGENT HEALTH CHECK Exit Code 0 - Check is passing ! Exit Code 1 - Check is warning ! Any Other - Check is failing check script:
31. CONSUL AGENT HEALTH CHECK
32. BLOCKING QUERIES ɾLong PollingΛ࢖ͬͯঢ়ଶมߋΛଈ࣌ݕ஌ ! ɾԠ౴͸λΠϜΞ΢τ͔࣌ҟৗൃੜ࣌ ! ?wait=<interval>&index=<idx>
33. USE CASES ɾInternal DNS ! ɾInventory Management ! ɾFailover
34. DEMO
35. BENCHMARK GET 8 CPU Cores, 2Ghz 16GB RAM 160GB SSD disk 1Gbps NIC Requests/sec rps 0 2750 5500 8250 11000 10,469.54 1,054.031 v0.2.0 v0.3.0
36. BENCHMARK GET response time distribution # of responses 0.00 3000.00 6000.00 9000.00 12000.00 response time (s) 0.001 0.007 0.014 0.017 0.023 0.030 0.033 0.062 0.094 0.125 0.156 v0.2.0 v0.3.0 8 CPU Cores, 2Ghz 16GB RAM 160GB SSD disk 1Gbps NIC
37. WEB UI
38. CONSUL-API Library in GO https://github.com/armon/consul-api
39. IN CLOSING consulΠΠΑʂ