Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scale Agnostic Python
Search
Sean T Allen
August 17, 2017
Programming
0
210
Scale Agnostic Python
Presented to the NYC Python meetup August 17, 2017
Sean T Allen
August 17, 2017
Tweet
Share
More Decks by Sean T Allen
See All by Sean T Allen
On Deny Capabilities for Safe, Fast Actors
seantallen
1
100
Data corrupting architectures we know and love - VelocityConf San Jose 2019
seantallen
1
110
Data corrupting architectures we know and love
seantallen
0
110
Scale-Independent Python
seantallen
2
220
Pat Helland and Me: How to Build Stateful Distributed Applications That Can Scale Almost Infinitely - Velocity NY October 2018
seantallen
0
84
Adventures in Cgo Preformance
seantallen
0
650
Pat Helland and me: A talk about “Life Beyond Distributed Transactions: An Apostate’s Opinion”
seantallen
0
610
Pat Helland and me: How to build stateful distributed applications that can scale almost infinitely - Salesforce July 2018
seantallen
0
170
Pony: How I learned to stop worrying and embrace an unproven technology - OSCON 2018
seantallen
0
470
Other Decks in Programming
See All in Programming
振り返れば奴(Cline)がいる
keiyagi
0
150
富山発の個人開発サービスで日本中の学校の業務を改善した話
krpk1900
4
340
知られざるDMMデータエンジニアの生態 〜かつてツチノコと呼ばれし者〜
takaha4k
3
1.1k
ASP. NET CoreにおけるWebAPIの最新情報
tomokusaba
0
310
時計仕掛けのCompose
mkeeda
1
250
chibiccをCILに移植した結果 (NGK2025S版)
kekyo
PRO
0
200
Compose でデザインと実装の差異を減らすための取り組み
oidy
1
280
チームリードになって変わったこと
isaka1022
0
160
Kubernetes History Inspector(KHI)を触ってみた
bells17
0
170
Alba: Why, How and What's So Interesting
okuramasafumi
0
240
AHC041解説
terryu16
0
560
Ruby on cygwin 2025-02
fd0
0
120
Featured
See All Featured
The Invisible Side of Design
smashingmag
299
50k
A Philosophy of Restraint
colly
203
16k
Designing for Performance
lara
604
68k
Why Our Code Smells
bkeepers
PRO
335
57k
The Art of Programming - Codeland 2020
erikaheidi
53
13k
GitHub's CSS Performance
jonrohan
1030
460k
Six Lessons from altMBA
skipperchong
27
3.6k
Statistics for Hackers
jakevdp
797
220k
Visualization
eitanlees
146
15k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
4
380
The Illustrated Children's Guide to Kubernetes
chrisshort
48
49k
Agile that works and the tools we love
rasmusluckow
328
21k
Transcript
SCALE AGNOSTIC PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT
ANY CODE CHANGES*
SEAN T. ALLEN VP OF ENGINEERING AT SENDENCE AUTHOR OF
“STORM APPLIED” @SEANTALLEN
SCALE AGNOSTIC COMPUTING
THE PROBLEM THE LITTLE APPLICATION THAT COULD… UNTIL IT COULDN’T.
Scaling an application from “little data” to “a little more
data” to “big data” is hard.
OPTIONS
Rewrite the application Rewrite to scale across multiple processes Example:
“multiprocessing” Python library
Run more versions Smart loader to load data in. Perhaps
via a job queue like Celery.
Go scale agnostic Write you application against a scale agnostic
API that runs on a scale aware platform
SENDENCE WALLAROO SCALE AGNOSTIC COMPUTING FOR PYTHON
Sendence Wallaroo Simple Scale Agnostic API Managed in-memory state Guaranteed
Message Processing
“We handle the hard infrastructure plumbing so you don’t have
to.” – The Wallaroo Team
LEARN MORE… sendence.com Sendence Engineering on Twitter: @SendenceEng Pat Helland’s
“Life Beyond Distributed Transactions”