Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Scale Agnostic Python
Sean T Allen
August 17, 2017
Programming
0
140
Scale Agnostic Python
Presented to the NYC Python meetup August 17, 2017
Sean T Allen
August 17, 2017
Tweet
Share
More Decks by Sean T Allen
See All by Sean T Allen
On Deny Capabilities for Safe, Fast Actors
seantallen
0
61
Data corrupting architectures we know and love - VelocityConf San Jose 2019
seantallen
1
76
Data corrupting architectures we know and love
seantallen
0
85
Scale-Independent Python
seantallen
2
170
Pat Helland and Me: How to Build Stateful Distributed Applications That Can Scale Almost Infinitely - Velocity NY October 2018
seantallen
0
52
Adventures in Cgo Preformance
seantallen
0
460
Pat Helland and me: A talk about “Life Beyond Distributed Transactions: An Apostate’s Opinion”
seantallen
0
430
Pat Helland and me: How to build stateful distributed applications that can scale almost infinitely - Salesforce July 2018
seantallen
0
140
Pony: How I learned to stop worrying and embrace an unproven technology - OSCON 2018
seantallen
0
180
Other Decks in Programming
See All in Programming
From Java 11 to 17 and beyond
josepaumard
0
280
Improve Build Times in Less Time
zacsweers
6
2.7k
TDX22: User-Mode DB Ops
ca_peterson
3
1.1k
heyにおけるSREの大切さ~マルチプロダクト運用の「楽しさ」と「難しさ」および今後の展望~
fufuhu
3
1.4k
Node.js 最新動向 TFCon 2022
yosuke_furukawa
PRO
5
2.5k
コードの解析と言語習得の心得
jinjin33333
0
120
Securing Kafka Connect Pipelines with Client-Side Field Level Cryptography @ Kafka Summit London 2022
hpgrahsl
0
300
SPA/MPA 議論の俯瞰と 現代における設計のポイント - #tfcon 2022 フロントエンド設計
ahomu
2
1.4k
Angular's new Standalone Components: How Will They Affect My Architecture? @iJS London 2022
manfredsteyer
PRO
0
380
Keep Your Cache Always Fresh With Debezium
gunnarmorling
0
170
PublishでWebサイトを構築してみた / generate_website_with_publish
uhooi
2
110
The future of trust stores in Python
sethmlarson
0
180
Featured
See All Featured
Web development in the modern age
philhawksworth
197
9.3k
Designing for Performance
lara
596
63k
Automating Front-end Workflow
addyosmani
1351
200k
Building Adaptive Systems
keathley
25
1.1k
How To Stay Up To Date on Web Technology
chriscoyier
780
250k
The Illustrated Children's Guide to Kubernetes
chrisshort
14
34k
What the flash - Photography Introduction
edds
61
9.8k
The Cult of Friendly URLs
andyhume
68
4.7k
Documentation Writing (for coders)
carmenhchung
48
2.5k
Web Components: a chance to create the future
zenorocha
303
40k
Building Flexible Design Systems
yeseniaperezcruz
310
33k
YesSQL, Process and Tooling at Scale
rocio
157
12k
Transcript
SCALE AGNOSTIC PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT
ANY CODE CHANGES*
SEAN T. ALLEN VP OF ENGINEERING AT SENDENCE AUTHOR OF
“STORM APPLIED” @SEANTALLEN
SCALE AGNOSTIC COMPUTING
THE PROBLEM THE LITTLE APPLICATION THAT COULD… UNTIL IT COULDN’T.
Scaling an application from “little data” to “a little more
data” to “big data” is hard.
OPTIONS
Rewrite the application Rewrite to scale across multiple processes Example:
“multiprocessing” Python library
Run more versions Smart loader to load data in. Perhaps
via a job queue like Celery.
Go scale agnostic Write you application against a scale agnostic
API that runs on a scale aware platform
SENDENCE WALLAROO SCALE AGNOSTIC COMPUTING FOR PYTHON
Sendence Wallaroo Simple Scale Agnostic API Managed in-memory state Guaranteed
Message Processing
“We handle the hard infrastructure plumbing so you don’t have
to.” – The Wallaroo Team
LEARN MORE… sendence.com Sendence Engineering on Twitter: @SendenceEng Pat Helland’s
“Life Beyond Distributed Transactions”