Slide 1

Slide 1 text

ABOUT CONSUL 7/20/2014 @ijin ~In Depth~ hbstudy #60

Slide 2

Slide 2 text

ABOUT ME (@IJIN) • Michael H. Oshita • Japanese American • 10+ Years in Japan • Software/Infra/Cloud Engineer • http://ijin.github.io

Slide 3

Slide 3 text

ABOUT ME (@IJIN) τϥϒϧˑ͠Ύʔͨʔͣ

Slide 4

Slide 4 text

τϥϒϧˑ͠Ύʔͨʔͣ #08 http://bit.ly/torash08

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

CONSUL

Slide 7

Slide 7 text

CONSUL HTTP APIͱDNSͰૢ࡞͕Ͱ͖Δ෼ࢄܕΫϥελ

Slide 8

Slide 8 text

SERVICE DISCOVERY • ؆୯ʹserviceΛొ࿥͠ɺࣗಈͰΫϥελ্Ͱݕग़Մೳ • DNS·ͨ͸HTTPʹΑͬͯࢀর

Slide 9

Slide 9 text

FAILURE DETECTION • ϔϧενΣοΫػೳ • ࢮΜͩϊʔυ΁ͷϦΫΤετᷖճ !

Slide 10

Slide 10 text

MULTI DATACENTER • ෳ਺ͷσʔληϯλʔΛ·͍ͨͰ௨৴ • ଞͷDC্Ͱͷserviceͷࢀর

Slide 11

Slide 11 text

KEY/VALUE STORE • ಈతͳઃఆ৘ใͷஔ͖৔ॴͱͯ͠ • Long PollingͰઃఆมߋͷଈ࣌௨஌

Slide 12

Slide 12 text

TECHNOLOGY • Consensus Protocol (Raft, FSM) In Search of an Understandable Consensus Algorithm - Stanford University • Gossip Protocol (Serf) SWIM: Scalable Weakly-consistent Infection-style Process Group Membership - Cornel University • Sessions (Chuby) The Chubby Lock Service for Loosely-Coupled Distributed Systems - Google • TLS, Jespen Testing !

Slide 13

Slide 13 text

rpc forwarding & gossip protocol server server (leader) server ARCHITECTURE client client client

Slide 14

Slide 14 text

rpc forwarding & gossip protocol server server (leader) server ARCHITECTURE client client client peer set

Slide 15

Slide 15 text

rpc forwarding & gossip protocol server server (leader) server ARCHITECTURE client client client gossip

Slide 16

Slide 16 text

rpc forwarding & gossip protocol server server (leader) server ARCHITECTURE client client client query forward query query forward

Slide 17

Slide 17 text

ARCHITECTURE

Slide 18

Slide 18 text

CONSISTENCY

Slide 19

Slide 19 text

consensus protocol using raft server server (leader) server CONSISTENCY server server

Slide 20

Slide 20 text

consensus protocol using raft server server (leader) server write CONSISTENCY server server

Slide 21

Slide 21 text

consensus protocol using raft server server (leader) server write CONSISTENCY server server write

Slide 22

Slide 22 text

consensus protocol using raft server server (leader) server write CONSISTENCY server server write write

Slide 23

Slide 23 text

consensus protocol using raft server server (leader) server write CONSISTENCY server server write write quorum[(n/2)+1] reached, commited!

Slide 24

Slide 24 text

CONSISTENCY writes: ! "?state" or "?consistent" parameter ɾdefault ɾstrongly consistent ɾstale ! consensus protocol using raft reads: ! Always Strongly Consistent

Slide 25

Slide 25 text

CONSISTENCY default "?state" or "?consistent" parameter ɾMostly Strongly Consistent ɾStale reads possible when leader promotion ɾFast ! server server (old leader) server (new leader) A' A leader lease leader lease

Slide 26

Slide 26 text

CONSISTENCY consistent "?state" or "?consistent" parameter ɾStrongly Consistent ɾLeader validates via Qurum ɾLatency affected by Round-trip ! server server (leader) server A A A leader lease

Slide 27

Slide 27 text

CONSISTENCY stale "?state" or "?consistent" parameter ɾStale Reads, within ~50ms ɾNon-Leader Servers can answer ɾVery Fast ! server server (leader) server A A' A''

Slide 28

Slide 28 text

DEPLOYMENT TABLE Servers Quorum Size Failure Tolerance 1 1 0 2 2 0 3 2 1 4 3 1 5 3 2 6 4 2 7 4 3

Slide 29

Slide 29 text

JEPSEN TESTING Partition tolerance https://github.com/aphyr/jepsen http://www.infoq.com/articles/jepsen http://blog.foundationdb.com/call-me-maybe-foundationdb-vs-jepsen

Slide 30

Slide 30 text

CONSUL AGENT Nodes & Services foo bar baz redis 01.web 02.web node service

Slide 31

Slide 31 text

CONSUL AGENT • DNS Interface • HTTP API • Health Check !

Slide 32

Slide 32 text

CONSUL AGENT DNS Interface .node.. Node Lookup:

Slide 33

Slide 33 text

CONSUL AGENT DNS Interface

Slide 34

Slide 34 text

CONSUL AGENT DNS Interface ..service. Service Lookup:

Slide 35

Slide 35 text

CONSUL AGENT DNS Interface

Slide 36

Slide 36 text

CONSUL AGENT DNS Interface

Slide 37

Slide 37 text

CONSUL AGENT DNS CACHING Stale Reads ɾࢀর͸σϑΥϧτͰLeader NodeͷΈ ɾ༗ޮԽ͢ΔͱଞͷServer Node͔ΒࢀরՄೳ ɾεϧʔϓοτ޲্ ɾmax_staleͰ࣌ؒΛઃఆ ! TTL ɾDNS TTL͸σϑΥϧτͰ0 ɾdnsmasq౳ͰΩϟογϡՄೳʹ { "dns_config": { "service_ttl": { "*": "5s", "web": "30s" } } new! (0.3.0)

Slide 38

Slide 38 text

CONSUL AGENT HTTP API /v1/kv/ key/value endpoint: RESTful: GET, PUT, DELETE

Slide 39

Slide 39 text

CONSUL AGENT HTTP API several endpoint categories: • kv - Key/Value store • agent - Agent control • catalog - Manages nodes and services • health - Manages health checks • session - Session manipulation • status - Consul system status • internal - Internal APIs. Purposely

Slide 40

Slide 40 text

CONSUL AGENT HTTP API key/value: base64 raw

Slide 41

Slide 41 text

CONSUL AGENT HTTP API /v1/agent/checks /v1/agent/services /v1/agent/members /v1/agent/self /v1/agent/join/
/v1/agent/force-leave/ /v1/agent/service/deregister/ agent endpoint: /v1/agent/register /v1/agent/deregister/ /v1/agent/pass/ /v1/agent/warn/ /v1/agent/fail/ /v1/agent/service/register

Slide 42

Slide 42 text

CONSUL AGENT HTTP API /v1/catalog/register /v1/catalog/deregister /v1/catalog/datacenters /v1/catalog/nodes catalog endpoint: /v1/catalog/services /v1/catalog/service/ /v1/catalog/node/ * supports blocking queries & consistency modes

Slide 43

Slide 43 text

CONSUL AGENT HTTP API /v1/health/node/ /v1/health/checks/ /v1/health/service/ /v1/state/ health endpoint: * supports blocking queries & consistency modes

Slide 44

Slide 44 text

CONSUL AGENT HEALTH CHECK { "service": { "check": { "script": "curl localhost > /dev/null 2>&1", "interval": "10s" } } check definition:

Slide 45

Slide 45 text

CONSUL AGENT HEALTH CHECK Exit Code 0 - Check is passing ! Exit Code 1 - Check is warning ! Any Other - Check is failing check script:

Slide 46

Slide 46 text

CONSUL AGENT HEALTH CHECK

Slide 47

Slide 47 text

BLOCKING QUERIES ɾLong PollingΛ࢖ͬͯঢ়ଶมߋΛଈ࣌ݕ஌ ! ɾԠ౴͸λΠϜΞ΢τ͔࣌ҟৗൃੜ࣌ ! ?wait=&index=

Slide 48

Slide 48 text

WEB UI

Slide 49

Slide 49 text

USE CASES ɾInternal DNS ! ɾInventory Management ! ɾFailover

Slide 50

Slide 50 text

DEMO

Slide 51

Slide 51 text

BENCHMARK GET 8 CPU Cores, 2Ghz 16GB RAM 160GB SSD disk 1Gbps NIC Requests/sec rps 0 2750 5500 8250 11000 10,469.54 2,115.61 v0.2.0 v0.3.0

Slide 52

Slide 52 text

BENCHMARK GET response time distribution # of responses 0.00 3000.00 6000.00 9000.00 12000.00 response time (s) 0.001 0.004 0.010 0.014 .018 0.023 0.035 0.051 0.067 0.083 v0.2.0 v0.3.0 8 CPU Cores, 2Ghz 16GB RAM 160GB SSD disk 1Gbps NIC

Slide 53

Slide 53 text

BENCHMARK GET 8 CPU Cores, 2Ghz 16GB RAM 160GB SSD disk 1Gbps NIC Requests/sec default consistent stale 0 2750 5500 8250 11000 10,948.2 10,245.562 10,469.54 1,986.771 1,953.555 2,115.61 v0.2.0 v0.3.0

Slide 54

Slide 54 text

BENCHMARK PUT 8 CPU Cores, 2Ghz 16GB RAM 160GB SSD disk 1Gbps NIC Requests/sec rps 0 1250 2500 3750 5000 4,091.67 1,054.031 v0.2.0 v0.3.0

Slide 55

Slide 55 text

BENCHMARK PUT response time distribution # of responses 0.00 2250.00 4500.00 6750.00 9000.00 response time (s) 0.006 0.013 0.017 0.024 0.031 0.038 0.047 0.078 0.109 0.140 0.172 v0.2.0 v0.3.0 8 CPU Cores, 2Ghz 16GB RAM 160GB SSD disk 1Gbps NIC

Slide 56

Slide 56 text

CONSUL-API Library in GO https://github.com/armon/consul-api

Slide 57

Slide 57 text

IN CLOSING consulΠΠΑʂ