ABOUT CONSUL
7/20/2014
@ijin
~In Depth~ hbstudy #60
Slide 2
Slide 2 text
ABOUT ME (@IJIN)
• Michael H. Oshita
• Japanese American
• 10+ Years in Japan
• Software/Infra/Cloud Engineer
• http://ijin.github.io
Slide 3
Slide 3 text
ABOUT ME (@IJIN)
τϥϒϧˑ͠Ύʔͨʔͣ
Slide 4
Slide 4 text
τϥϒϧˑ͠Ύʔͨʔͣ #08
http://bit.ly/torash08
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
CONSUL
Slide 7
Slide 7 text
CONSUL
HTTP APIͱDNSͰૢ࡞͕Ͱ͖ΔࢄܕΫϥελ
Slide 8
Slide 8 text
SERVICE DISCOVERY
• ؆୯ʹserviceΛొ͠ɺࣗಈͰΫϥελ্Ͱݕग़Մೳ
• DNS·ͨHTTPʹΑͬͯࢀর
Slide 9
Slide 9 text
FAILURE DETECTION
• ϔϧενΣοΫػೳ
• ࢮΜͩϊʔυͷϦΫΤετᷖճ
!
Slide 10
Slide 10 text
MULTI DATACENTER
• ෳͷσʔληϯλʔΛ·͍ͨͰ௨৴
• ଞͷDC্Ͱͷserviceͷࢀর
Slide 11
Slide 11 text
KEY/VALUE STORE
• ಈతͳઃఆใͷஔ͖ॴͱͯ͠
• Long PollingͰઃఆมߋͷଈ࣌௨
Slide 12
Slide 12 text
TECHNOLOGY
• Consensus Protocol (Raft, FSM)
In Search of an Understandable Consensus Algorithm - Stanford University
• Gossip Protocol (Serf)
SWIM: Scalable Weakly-consistent Infection-style Process Group Membership - Cornel University
• Sessions (Chuby)
The Chubby Lock Service for Loosely-Coupled Distributed Systems - Google
• TLS, Jespen Testing
!
Slide 13
Slide 13 text
rpc forwarding & gossip protocol
server
server
(leader)
server
ARCHITECTURE
client client client
Slide 14
Slide 14 text
rpc forwarding & gossip protocol
server
server
(leader)
server
ARCHITECTURE
client client client
peer set
Slide 15
Slide 15 text
rpc forwarding & gossip protocol
server
server
(leader)
server
ARCHITECTURE
client client client
gossip
Slide 16
Slide 16 text
rpc forwarding & gossip protocol
server
server
(leader)
server
ARCHITECTURE
client client client
query
forward
query query
forward
Slide 17
Slide 17 text
ARCHITECTURE
Slide 18
Slide 18 text
CONSISTENCY
Slide 19
Slide 19 text
consensus protocol using raft
server
server
(leader)
server
CONSISTENCY
server server
Slide 20
Slide 20 text
consensus protocol using raft
server
server
(leader)
server
write
CONSISTENCY
server server
Slide 21
Slide 21 text
consensus protocol using raft
server
server
(leader)
server
write
CONSISTENCY
server server
write
Slide 22
Slide 22 text
consensus protocol using raft
server
server
(leader)
server
write
CONSISTENCY
server server
write
write
Slide 23
Slide 23 text
consensus protocol using raft
server
server
(leader)
server
write
CONSISTENCY
server server
write
write
quorum[(n/2)+1] reached, commited!
CONSISTENCY
default
"?state" or "?consistent" parameter
ɾMostly Strongly Consistent
ɾStale reads possible when leader promotion
ɾFast
!
server
server
(old
leader)
server
(new
leader)
A'
A
leader lease leader lease
Slide 26
Slide 26 text
CONSISTENCY
consistent
"?state" or "?consistent" parameter
ɾStrongly Consistent
ɾLeader validates via Qurum
ɾLatency affected by Round-trip
!
server
server
(leader)
server
A A
A
leader lease
Slide 27
Slide 27 text
CONSISTENCY
stale
"?state" or "?consistent" parameter
ɾStale Reads, within ~50ms
ɾNon-Leader Servers can answer
ɾVery Fast
!
server
server
(leader)
server
A A'
A''
CONSUL AGENT
Nodes & Services
foo bar baz
redis 01.web 02.web
node
service
Slide 31
Slide 31 text
CONSUL AGENT
• DNS Interface
• HTTP API
• Health Check
!
Slide 32
Slide 32 text
CONSUL AGENT
DNS Interface
.node..
Node Lookup:
Slide 33
Slide 33 text
CONSUL AGENT
DNS Interface
Slide 34
Slide 34 text
CONSUL AGENT
DNS Interface
..service.
Service Lookup:
Slide 35
Slide 35 text
CONSUL AGENT
DNS Interface
Slide 36
Slide 36 text
CONSUL AGENT
DNS Interface
Slide 37
Slide 37 text
CONSUL AGENT
DNS CACHING
Stale Reads
ɾࢀরσϑΥϧτͰLeader NodeͷΈ
ɾ༗ޮԽ͢ΔͱଞͷServer Node͔ΒࢀরՄೳ
ɾεϧʔϓοτ্
ɾmax_staleͰ࣌ؒΛઃఆ
!
TTL
ɾDNS TTLσϑΥϧτͰ0
ɾdnsmasqͰΩϟογϡՄೳʹ
{
"dns_config": {
"service_ttl": {
"*": "5s",
"web": "30s"
}
}
new!
(0.3.0)
Slide 38
Slide 38 text
CONSUL AGENT
HTTP API
/v1/kv/
key/value endpoint:
RESTful:
GET, PUT, DELETE
Slide 39
Slide 39 text
CONSUL AGENT
HTTP API
several endpoint categories:
• kv - Key/Value store
• agent - Agent control
• catalog - Manages nodes and services
• health - Manages health checks
• session - Session manipulation
• status - Consul system status
• internal - Internal APIs. Purposely
Slide 40
Slide 40 text
CONSUL AGENT
HTTP API
key/value:
base64
raw
Slide 41
Slide 41 text
CONSUL AGENT
HTTP API
/v1/agent/checks
/v1/agent/services
/v1/agent/members
/v1/agent/self
/v1/agent/join/
/v1/agent/force-leave/
/v1/agent/service/deregister/
agent endpoint:
/v1/agent/register
/v1/agent/deregister/
/v1/agent/pass/
/v1/agent/warn/
/v1/agent/fail/
/v1/agent/service/register
Slide 42
Slide 42 text
CONSUL AGENT
HTTP API
/v1/catalog/register
/v1/catalog/deregister
/v1/catalog/datacenters
/v1/catalog/nodes
catalog endpoint:
/v1/catalog/services
/v1/catalog/service/
/v1/catalog/node/
* supports blocking queries & consistency modes
Slide 43
Slide 43 text
CONSUL AGENT
HTTP API
/v1/health/node/
/v1/health/checks/
/v1/health/service/
/v1/state/
health endpoint:
* supports blocking queries & consistency modes
USE CASES
ɾInternal DNS
!
ɾInventory Management
!
ɾFailover
Slide 50
Slide 50 text
DEMO
Slide 51
Slide 51 text
BENCHMARK
GET
8 CPU Cores, 2Ghz
16GB RAM
160GB SSD disk
1Gbps NIC
Requests/sec
rps
0 2750 5500 8250 11000
10,469.54
2,115.61
v0.2.0 v0.3.0
Slide 52
Slide 52 text
BENCHMARK
GET
response time distribution
# of responses
0.00
3000.00
6000.00
9000.00
12000.00
response time (s)
0.001 0.004 0.010 0.014 .018 0.023 0.035 0.051 0.067 0.083
v0.2.0 v0.3.0
8 CPU Cores, 2Ghz
16GB RAM
160GB SSD disk
1Gbps NIC
Slide 53
Slide 53 text
BENCHMARK
GET
8 CPU Cores, 2Ghz
16GB RAM
160GB SSD disk
1Gbps NIC
Requests/sec
default
consistent
stale
0 2750 5500 8250 11000
10,948.2
10,245.562
10,469.54
1,986.771
1,953.555
2,115.61
v0.2.0 v0.3.0
Slide 54
Slide 54 text
BENCHMARK
PUT
8 CPU Cores, 2Ghz
16GB RAM
160GB SSD disk
1Gbps NIC
Requests/sec
rps
0 1250 2500 3750 5000
4,091.67
1,054.031
v0.2.0 v0.3.0
Slide 55
Slide 55 text
BENCHMARK
PUT
response time distribution
# of responses
0.00
2250.00
4500.00
6750.00
9000.00
response time (s)
0.006 0.013 0.017 0.024 0.031 0.038 0.047 0.078 0.109 0.140 0.172
v0.2.0 v0.3.0
8 CPU Cores, 2Ghz
16GB RAM
160GB SSD disk
1Gbps NIC
Slide 56
Slide 56 text
CONSUL-API
Library in GO
https://github.com/armon/consul-api