Tweet
Tweet

Slide 1

Slide 1 text

riak latency distributed systems and @tsantero Wednesday, June 19, 13

Slide 2

Slide 2 text

What we talk about when we talk about Availability Wednesday, June 19, 13

Slide 3

Slide 3 text

Liveness good things eventually happen Wednesday, June 19, 13

Slide 4

Slide 4 text

User Experience Wednesday, June 19, 13

Slide 5

Slide 5 text

$$$$$$$$$$ Wednesday, June 19, 13

Slide 6

Slide 6 text

Are you Good with computers? Wednesday, June 19, 13

Slide 7

Slide 7 text

Why isn’t this website loading? Wednesday, June 19, 13

Slide 8

Slide 8 text

Things are getting Easier* Wednesday, June 19, 13

Slide 9

Slide 9 text

Users expect it to JUST WORK Wednesday, June 19, 13

Slide 10

Slide 10 text

Frontend Middleware Backend Wednesday, June 19, 13

Slide 11

Slide 11 text

Frontend Middleware Backend Frontend Frontend Middleware Middleware Wednesday, June 19, 13

Slide 12

Slide 12 text

Frontend Middleware Master Frontend Frontend Middleware Middleware S S S Wednesday, June 19, 13

Slide 13

Slide 13 text

Frontend Middleware Master Frontend Frontend Middleware Middleware S S S Wednesday, June 19, 13

Slide 14

Slide 14 text

Frontend Middleware Frontend Frontend Middleware Middleware Riak Wednesday, June 19, 13

Slide 15

Slide 15 text

everything is distributed Wednesday, June 19, 13

Slide 16

Slide 16 text

What we talk about when we talk about Distribution Wednesday, June 19, 13

Slide 17

Slide 17 text

Global Availability Wednesday, June 19, 13

Slide 18

Slide 18 text

Frontend Middleware Master Frontend Frontend Middleware Middleware Riak CDN Wednesday, June 19, 13

Slide 19

Slide 19 text

Frontend Middleware Master Frontend Frontend Middleware Middleware Riak CDN Wednesday, June 19, 13

Slide 20

Slide 20 text

3.33ms / km (+/- 5µs) Wednesday, June 19, 13

Slide 21

Slide 21 text

San Jose New York Amsterdam Frankfurt Seoul IXP Wednesday, June 19, 13

Slide 22

Slide 22 text

San Jose New York = ~ 74ms Wednesday, June 19, 13

Slide 23

Slide 23 text

San Jose Amsterdam = ~ 200ms Wednesday, June 19, 13

Slide 24

Slide 24 text

Riak Internals Wednesday, June 19, 13

Slide 25

Slide 25 text

node node node node node distributed masterless key/value Store Wednesday, June 19, 13

Slide 26

Slide 26 text

REPLICATION Wednesday, June 19, 13

Slide 27

Slide 27 text

Quorum requests N R W ead rite Wednesday, June 19, 13

Slide 28

Slide 28 text

Tail Latency Misbehaving Nodes & Wednesday, June 19, 13

Slide 29

Slide 29 text

“...widespread underestimation of the specific difficulties of size seems one of the major underlying causes of the current software failure.” --EW Dijkstra Notes on Structured Programming 1969 Wednesday, June 19, 13

Slide 30

Slide 30 text

Google’s BigTable 95th 99.9th 24ms 994ms : : Wednesday, June 19, 13

Slide 31

Slide 31 text

Reduce Tail Latencies with Redundancy Wednesday, June 19, 13

Slide 32

Slide 32 text

IF response > 10ms Send 2nd Request = ~ 99.9 : 50ms (5% increase total reqs.) Wednesday, June 19, 13

Slide 33

Slide 33 text

Overall Latency determined by latency of slowest machine Wednesday, June 19, 13

Slide 34

Slide 34 text

Redundancy in Riak? Wednesday, June 19, 13

Slide 35

Slide 35 text

Setup: basho bench Riak Riak Riak Riak Riak Wednesday, June 19, 13

Slide 36

Slide 36 text

Demo: sudo  tc  qdisc  add  dev  eth0  root  netem  delay  150ms Network: for  x  in  /sys/devices/system/cpu/*;  do (while  true  ;  do  openssl  speed;  done)  & done wait CPU: Wednesday, June 19, 13

Slide 37

Slide 37 text

demo Wednesday, June 19, 13

Slide 38

Slide 38 text

client Riak Riak Riak riak node L B Wednesday, June 19, 13

Slide 39

Slide 39 text

client Riak Riak Riak riak node L B Wednesday, June 19, 13

Slide 40

Slide 40 text

client Riak Riak Riak riak node L B Wednesday, June 19, 13

Slide 41

Slide 41 text

client Riak Riak Riak riak node L B Wednesday, June 19, 13

Slide 42

Slide 42 text

Tunables Wednesday, June 19, 13

Slide 43

Slide 43 text

I <3 Immutability Wednesday, June 19, 13

Slide 44

Slide 44 text

Write Once Read Often N W R = = = 5 N 1 notfound_ok: False Wednesday, June 19, 13

Slide 45

Slide 45 text

Write Once Read Hardly N W R = = = 3 Q Q Wednesday, June 19, 13

Slide 46

Slide 46 text

Write Once N DW R = = = 3 2 1 CRITICAL Wednesday, June 19, 13

Slide 47

Slide 47 text

RSEW? Read Someone Else’s Writes lol Wednesday, June 19, 13

Slide 48

Slide 48 text

Lots of work Remains Wednesday, June 19, 13

Slide 49

Slide 49 text

Thanks! Wednesday, June 19, 13