PAT HELLAND AND ME
HOW TO BUILD STATEFUL DISTRIBUTED APPLICATIONS THAT CAN SCALE ALMOST INFINITELY
Slide 2
Slide 2 text
PAT HELLAND
AND ME
Slide 3
Slide 3 text
SEAN T. ALLEN
VP OF ENGINEERING AT WALLAROO LABS
MEMBER OF THE PONY CORE TEAM
AUTHOR OF “STORM APPLIED”
LOVER OF FRENCH STREET ART
@SEANTALLEN
@WALLAROOLABS
@PONYLANG
Slide 4
Slide 4 text
DATABASES
APPARENTLY, I LIKE TO
STICK THEM IN THINGS…
Slide 5
Slide 5 text
SOME AXIOMS…
Slide 6
Slide 6 text
TO SCALE INFINITELY,
WE HAVE TO SCALE HORIZONTALLY
Slide 7
Slide 7 text
TO SCALE INFINITELY,
WE MUST AVOID COORDINATION
Slide 8
Slide 8 text
DISTRIBUTED TRANSACTIONS ARE
A FORM OF COORDINATION
Slide 9
Slide 9 text
THEREFORE…
TO SCALE INFINITELY,
WE CAN’T USE TRANSACTIONS
Slide 10
Slide 10 text
WELCOME TO
DISTRIBUTED
SYSTEMS!
O, BY THE WAY, ALL THE
RULES HAVE CHANGED
Slide 11
Slide 11 text
WHAT IS SCALING?
Slide 12
Slide 12 text
MORE AND
MORE THINGS
BUT, THEY DON’T GET
BIGGER. THERE’S JUST…
MORE OF THEM. LOTS MORE.
Slide 13
Slide 13 text
WE SCALE
ENTITIES
ENTITIES:
LIVE ON A SINGLE MACHINE
AND ARE MANIPULATED
INDIVIDUALLY
Slide 14
Slide 14 text
WHAT IS AN ENTITY?
Slide 15
Slide 15 text
ENTITIES ARE BOUNDARIES OF
ATOMICITY
Slide 16
Slide 16 text
Bob
6 3
5
8
Alice
4 2
7
1
Slide 17
Slide 17 text
Bob
6 3
5
8
Alice
4 2
7
1
Slide 18
Slide 18 text
Bob
6 3
5
8
Alice
4
7
1
2
Slide 19
Slide 19 text
Bob
6 3
5
8
Alice
4 2
7
1
Slide 20
Slide 20 text
Bob
6 3
5
8
Alice
4 2
7
1
Slide 21
Slide 21 text
DENORMALIZE..
ALL THE THINGS!
Slide 22
Slide 22 text
TWO LAYER ARCHITECTURE
Slide 23
Slide 23 text
scale-agnostic
scale-aware
API
Slide 24
Slide 24 text
scale-agnostic
scale-aware
API
Slide 25
Slide 25 text
scale-agnostic
scale-aware
API
Slide 26
Slide 26 text
scale-agnostic
scale-aware
API
Slide 27
Slide 27 text
scale-agnostic
scale-aware
API
Slide 28
Slide 28 text
scale-agnostic
scale-aware
API
Slide 29
Slide 29 text
TO SCALE INFINITELY,
YOUR BUSINESS LOGIC HAS TO BE
INDEPENDENT OF SCALE
Slide 30
Slide 30 text
WALLAROO
SCALE INDEPENDENT
COMPUTING FOR PYTHON
Slide 31
Slide 31 text
AND IT’S NOT A DATABASE
Slide 32
Slide 32 text
ENTITIES
BUT WE CALL THEM…
“STATE OBJECTS”
Slide 33
Slide 33 text
TWO LAYER
ARCHITECTURE
BUT WE CALL IT…
“SCALE INDEPENDENCE”
Slide 34
Slide 34 text
user supplied logic
Wallaroo runtime
Wallaroo API
Slide 35
Slide 35 text
user supplied logic
Wallaroo runtime
Wallaroo API
Slide 36
Slide 36 text
user supplied logic
Wallaroo runtime
Wallaroo API
Slide 37
Slide 37 text
user supplied logic
Wallaroo runtime
Wallaroo API
Slide 38
Slide 38 text
WHAT’S HARD?
ALL OF IT? YOU’RE BUILDING
A DISTRIBUTED SYSTEMS
*FRAMEWORK*
Slide 39
Slide 39 text
CAP THEOREM
CONSISTENCY VS
AVAILABILITY…
YOU CAN’T ESCAPE IT.