etcd @ Hyderabad Scalability Meetup

Slide 1

Slide 1 text

@brandonphilips @coreoslinux

Slide 2

Slide 2 text

About Me CTO/CO-FOUNDER systems engineer @brandonphilips github.com/philips

Slide 3

Slide 3 text

@coreoslinux

Slide 4

Slide 4 text

ORIGINAL PHOTO (CC) BY IAN KLUFT @tectonicstack

Slide 5

Slide 5 text

https://github.com/philips/hacks/tree/master/etcd-demos

Slide 6

Slide 6 text

etcd

Slide 7

Slide 7 text

/etc distributed

Slide 8

Slide 8 text

open source software failure tolerant durable watchable exposed via HTTP runtime reconfigurable

Slide 9

Slide 9 text

Data Store API -X GET Get Wait -X PUT Put Create CAS -X DELETE Delete CAD

Slide 10

Slide 10 text

Leader Follower etcd Cluster

Slide 11

Slide 11 text

Applications locksmith

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Cluster Wide Reboot Lock 1. Need reboot to reboot? Decrement the semaphore key atomically with etcd. 2. manager.Reboot() and wait... 3. After rebooting increment the semaphore key in etcd atomically.

Slide 15

Slide 15 text

Applications vulcan, confd and skydns

Slide 16

Slide 16 text

Applications kubernetes

Slide 17

Slide 17 text

You Scheduler API Scheduler Machine(s)

Slide 18

Slide 18 text

Cluster Work Scheduling 1. Cluster API writes desired work into etcd keyspace. 2. Agents running on individual machines pick up work assigned to them. 3. Agents report where work is running and current status.

Slide 19

Slide 19 text

etcd basics clusters and bootstrapping

Slide 20

Slide 20 text

Leader Follower etcd Cluster

Slide 21

Slide 21 text

Availability In a 2F+1 cluster tolerate F machine failures

Slide 22

Slide 22 text

Available

Slide 23

Slide 23 text

Available

Slide 24

Slide 24 text

Available

Slide 25

Slide 25 text

Unavailable

Slide 26

Slide 26 text

Master Election Fast recovery (5-10*typical RTT) from temporarily unavailable

Slide 27

Slide 27 text

Available Leader Follower

Slide 28

Slide 28 text

Leader Follower Available

Slide 29

Slide 29 text

Leader Follower Temporarily Unavailable

Slide 30

Slide 30 text

Leader Follower Available

Slide 31

Slide 31 text

Durable log files, snapshots and backups

Slide 32

Slide 32 text

bootstrapping Candidate

Slide 33

Slide 33 text

GET discovery.etcd.io/new

Slide 34

Slide 34 text

discovery.etcd.io/6eadeac2 6eadeac2d

Slide 35

Slide 35 text

6eadeac2d/state CREATE

Slide 36

Slide 36 text

6eadeac2d/state Key Value Index state started 5890 n0 10.0.2.1 5891 n1 10.0.2.4 5898 ...

Slide 37

Slide 37 text

bootstrapped Leader Follower

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

Introduces gRPC backed API More efficient handling of events Non-blocking snapshots Better read performance when contention is high

Slide 40

Slide 40 text

Thanks we like pull requests github.com/coreos/etcd