Growing Distributed Systems: Abril Pro Ruby

Slide 1

Slide 1 text

Growing Distributed Systems

Slide 2

Slide 2 text

Hello Bryce Kerley @bonzoesc basho

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

Distributed Systems What even are they?

Slide 10

Slide 10 text

Distributed Systems

Slide 11

Slide 11 text

Distributed Systems Always Broken

Slide 12

Slide 12 text

Distributed Systems Light Cones

Slide 13

Slide 13 text

Mutual visibility Soldier Fires Sniper Fires

Slide 14

Slide 14 text

Distributed Systems A distributed system is one in which the failure of a computer you didn’t even know existed can render your own computer unusable. Lamport, 1987

Slide 15

Slide 15 text

Distributed Systems •Rules and Theory •Inconsistency and Unavailability •Distributed Rails

Slide 16

Slide 16 text

Byzantine Generals Two armies One town

Slide 17

Slide 17 text

Byzantine Generals Horse messengers

Slide 18

Slide 18 text

Byzantine Generals Consensus on when to attack

Slide 19

Slide 19 text

Byzantine Generals No Right Answer

Slide 20

Slide 20 text

Criticality •Amazon S3, July 2008 •GitHub, Dec. 2012 •Northeast US, 2003

Slide 21

Slide 21 text

Murphy’s Law at Scale •Backblaze •Gears of War 2

Slide 22

Slide 22 text

CAP Theorem Consistency Availability Partition tolerance

Slide 23

Slide 23 text

Rules, Theory, and Practice Compromise

Slide 24

Slide 24 text

Distributed Systems •Rules and Theory •Inconsistency and Unavailability •Distributed Rails

Slide 25

Slide 25 text

CAP Theorem lose Consistency or Availability during Partitions

Slide 26

Slide 26 text

Partitions

Slide 27

Slide 27 text

Partitions

Slide 28

Slide 28 text

CP Systems Zookeeper

Slide 29

Slide 29 text

Unavailability Don’t fill requests

Slide 30

Slide 30 text

Unavailability oonbeamlabs

Slide 31

Slide 31 text

AP Systems Riak

Slide 32

Slide 32 text

Inconsistency Accept writes Perform reads

Slide 33

Slide 33 text

Inconsistency

Slide 34

Slide 34 text

Hybrid Multiple Stores

Slide 35

Slide 35 text

Hybrid Where It Counts

Slide 36

Slide 36 text

Hybrid Riak 2 Strong Consistency

Slide 37

Slide 37 text

Distributed Systems •Rules and Theory •Inconsistency and Unavailability •Distributed Rails

Slide 38

Slide 38 text

Distributed Rails You’re Already Distributed

Slide 39

Slide 39 text

Distributed Rails Rails

Slide 40

Slide 40 text

Distributed Rails Rails Web DB

Slide 41

Slide 41 text

Distributed Rails SQL Rules

Slide 42

Slide 42 text

Distributed Rails NoSQL No Rules

Slide 43

Slide 43 text

Distributed Rails Temptation

Slide 44

Slide 44 text

Temptation “We’ll put everything in Rails models.”

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

Distributed Rails Service Oriented Architecture

Slide 47

Slide 47 text

Service Oriented Extract Parts

Slide 48

Slide 48 text

Service Oriented Documented Protocols

Slide 49

Slide 49 text

Service Oriented Smaller Deployments

Slide 50

Slide 50 text

Service Oriented Simpler Organization

Slide 51

Slide 51 text

Service Oriented Break Coupling

Slide 52

Slide 52 text

Service Oriented Flexible Implementation

Slide 53

Slide 53 text

Testing Unit Tests

Slide 54

Slide 54 text

Testing Service Tests

Slide 55

Slide 55 text

Testing Acceptance Tests

Slide 56

Slide 56 text

Testing Refactoring [GitHub] with Science Wynn Netherland, Big Ruby, Feb. 2014

Slide 57

Slide 57 text

Oﬄine Apps Latency Garbage Collection Packet loss Airplane rides Local internet

Slide 58

Slide 58 text

Oﬄine Apps Smarter clients

Slide 59

Slide 59 text

Oﬄine Apps You’re Already Doing It

Slide 60

Slide 60 text

Oﬄine Strategies Conflicts

Slide 61

Slide 61 text

Oﬄine Strategies Operational Transform

Slide 62

Slide 62 text

Oﬄine Strategies Git

Slide 63

Slide 63 text

Oﬄine Strategies Vesper

Slide 64

Slide 64 text

Distributed Systems •Rules and Theory •Inconsistency and Unavailability •Distributed Rails

Slide 65

Slide 65 text

Ruby Works

Slide 66

Slide 66 text

Make Something Awesome

Slide 67

Slide 67 text

Grow When You Need To

Slide 68

Slide 68 text

Thanks Bryce Kerley @bonzoesc [email protected] http://bitly.com/aprb-dist

Slide 69

Slide 69 text

Bonus Slides

Slide 70

Slide 70 text

Consensus Algorithm Paxos 1989

Slide 71

Slide 71 text

No content

Slide 72

Slide 72 text

No content

Slide 73

Slide 73 text

Consensus Algorithm Raft 2013

Slide 74

Slide 74 text

Harvest & Yield Harvest: fraction of data Yield: probability of success

Slide 75

Slide 75 text

No content

Slide 76

Slide 76 text

NoSQL and Rails SQL ACID

Slide 77

Slide 77 text

NoSQL and Rails Atomicity Consistency Isolation Durability

Slide 78

Slide 78 text

NoSQL and Rails NoSQL BASE

Slide 79

Slide 79 text

NoSQL and Rails Basically Available Soft-state Eventually consistent

Slide 80

Slide 80 text

Oﬄine Libraries Breeze PouchDB Lawnchair

Slide 81

Slide 81 text

Must Watch Jeff Hodges “Practicalities of Productionizing Distributed Systems” youtu.be/BKqgGpAOv1w