Link
Embed
Share
Beginning
This slide
Copy link URL
Copy link URL
Copy iframe embed code
Copy iframe embed code
Copy javascript embed code
Copy javascript embed code
Share
Tweet
Share
Tweet
Slide 1
Slide 1 text
SCALE-INDEPENDENT PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT ANY CODE CHANGES
Slide 2
Slide 2 text
SEAN T. ALLEN VP OF ENGINEERING AT WALLAROO LABS AUTHOR OF “STORM APPLIED” @SEANTALLEN @WALLAROOLABS
Slide 3
Slide 3 text
THIS TALK IS ABOUT “SCALING”
Slide 4
Slide 4 text
“SCALING” IS ABOUT REACTING TO CHANGE
Slide 5
Slide 5 text
LET’S TALK ABOUT REACTING TO CHANGES IN DATA
Slide 6
Slide 6 text
LET’S TALK ABOUT CHANGES IN APPLICATION STATE
Slide 7
Slide 7 text
LET’S TALK ABOUT A VERY SPECIFIC THING
Slide 8
Slide 8 text
LET’S TALK ABOUT HANDLING “MORE THINGS”
Slide 9
Slide 9 text
MORE THINGS?
Slide 10
Slide 10 text
MORE CUSTOMERS
Slide 11
Slide 11 text
MORE ORDERS
Slide 12
Slide 12 text
MORE VENDORS
Slide 13
Slide 13 text
MORE THINGS
Slide 14
Slide 14 text
THAT’S THE “SCALING” WE’RE TALKING ABOUT
Slide 15
Slide 15 text
WE’RE TALKING ABOUT HORIZONTAL SCALING
Slide 16
Slide 16 text
WE’RE TALKING ABOUT PAT HELLAND’S LIFE BEYOND DISTRIBUTED TRANSACTIONS
Slide 17
Slide 17 text
SEVERAL BIG IDEAS
Slide 18
Slide 18 text
LET’S TALK ABOUT ONE BIG IDEA
Slide 19
Slide 19 text
TWO-LAYER ARCHITECTURE
Slide 20
Slide 20 text
SCALE-INDEPENDENT LAYER
Slide 21
Slide 21 text
OUR BUSINESS LOGIC LIVES IN THE SCALE-INDEPENDENT LAYER
Slide 22
Slide 22 text
SCALE-AWARE LAYER
Slide 23
Slide 23 text
CLUSTERING
Slide 24
Slide 24 text
MESSAGE ROUTING
Slide 25
Slide 25 text
GUARANTEED MESSAGE DELIVERY
Slide 26
Slide 26 text
FAULT TOLERANCE
Slide 27
Slide 27 text
AND AN API TO CONNECT YOUR SCALE-INDEPENDENT CODE TO THE SCALE-AWARE LAYER
Slide 28
Slide 28 text
AND THAT IS WHAT WE’VE BUILT AT WALLAROO LABS
Slide 29
Slide 29 text
WALLAROO SCALE-INDEPENDENT COMPUTING FOR PYTHON
Slide 30
Slide 30 text
STATEFUL DATA PROCESSING ENGINE
Slide 31
Slide 31 text
SCALE-INDEPENDENT API
Slide 32
Slide 32 text
FLOW OF DATA
Slide 33
Slide 33 text
TRANSFORMATIONS OF DATA
Slide 34
Slide 34 text
ACCUMULATED APPLICATION STATE
Slide 35
Slide 35 text
APPLICATION DEFINITION FLOW OF DATA FROM SOURCE TO SINK
Slide 36
Slide 36 text
TWO DATA PIPELINES ORDERS
Slide 37
Slide 37 text
TWO DATA PIPELINES MARKET DATA
Slide 38
Slide 38 text
DEFINE OUR SOURCES 1 PER PIPELINE
Slide 39
Slide 39 text
DEFINE OUR OPERATIONS IN THIS EXAMPLE, THERE’S 1 PER PIPELINE
Slide 40
Slide 40 text
DEFINE OUR OPERATIONS CHECK ORDER AGAINST SYMBOL DATA
Slide 41
Slide 41 text
DEFINE OUR OPERATIONS UPDATE SYMBOL DATA WITH LATEST MARKET DATA
Slide 42
Slide 42 text
DEFINE OUR SINKS 1 PER PIPELINE
Slide 43
Slide 43 text
DEFINE OUR SINKS ORDERS PIPELINE MIGHT HAVE OUTPUT
Slide 44
Slide 44 text
DEFINE OUR SINKS MARKET DATA ONLY UPDATES SYMBOL DATA- NO OUTPUT
Slide 45
Slide 45 text
SCALE-INDEPENDENT ONLY FLOW OF DATA AND OPERATIONS
Slide 46
Slide 46 text
100% SCALE-INDEPENDENT NOTHING ABOUT THE NUMBER OF PYTHON PROCESSES
Slide 47
Slide 47 text
RUN WITH ANY NUMBER OF PYTHON PROCESSES
Slide 48
Slide 48 text
WITHOUT CHANGING CODE
Slide 49
Slide 49 text
SUCH IS THE GLORY OF SCALE-INDEPENDENCE
Slide 50
Slide 50 text
THERE’S MORE BUT WE ARE PROBABLY CLOSE TO BEING OUT OF TIME
Slide 51
Slide 51 text
QUICKLY, A FEW MORE WALLAROO FEATURES
Slide 52
Slide 52 text
PLAIN OLD PYTHON FUNCTIONS
Slide 53
Slide 53 text
PLAIN OLD PYTHON DATA OBJECTS
Slide 54
Slide 54 text
USE ANY PYTHON LIBRARY
Slide 55
Slide 55 text
LEARN MORE GITHUB.COM/SEANTALLEN/ SCALE-INDEPENDENT-PYTHON