Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scale Agnostic Python
Search
Sean T Allen
August 17, 2017
Programming
0
240
Scale Agnostic Python
Presented to the NYC Python meetup August 17, 2017
Sean T Allen
August 17, 2017
Tweet
Share
More Decks by Sean T Allen
See All by Sean T Allen
On Deny Capabilities for Safe, Fast Actors
seantallen
1
120
Data corrupting architectures we know and love - VelocityConf San Jose 2019
seantallen
1
120
Data corrupting architectures we know and love
seantallen
0
120
Scale-Independent Python
seantallen
2
240
Pat Helland and Me: How to Build Stateful Distributed Applications That Can Scale Almost Infinitely - Velocity NY October 2018
seantallen
0
96
Adventures in Cgo Preformance
seantallen
0
670
Pat Helland and me: A talk about “Life Beyond Distributed Transactions: An Apostate’s Opinion”
seantallen
0
690
Pat Helland and me: How to build stateful distributed applications that can scale almost infinitely - Salesforce July 2018
seantallen
0
180
Pony: How I learned to stop worrying and embrace an unproven technology - OSCON 2018
seantallen
0
520
Other Decks in Programming
See All in Programming
私はどうやって技術力を上げたのか
yusukebe
43
17k
大規模アプリにおけるXcode Previews実用化までの道のり
ikesyo
0
1k
複雑化したリポジトリをなんとかした話 pipenvからuvによるモノレポ構成への移行
satoshi256kbyte
1
740
Let's Write a Train Tracking Algorithm
twocentstudios
0
220
SpecKitでどこまでできる? コストはどれくらい?
leveragestech
0
480
iOSアプリの信頼性を向上させる取り組み/ios-app-improve-reliability
shino8rayu9
0
140
CI_CD「健康診断」のススメ。現場でのボトルネック特定から、健康診断を通じた組織的な改善手法
teamlab
PRO
0
160
defer f()とdefer fの挙動を 誤解していた話
kogamochiduki
2
170
dynamic!
moro
9
6k
ソフトウェア設計の実践的な考え方
masuda220
PRO
3
410
Playwrightはどのようにクロスブラウザをサポートしているのか
yotahada3
7
2.2k
NetworkXとGNNで学ぶグラフデータ分析入門〜複雑な関係性を解き明かすPythonの力〜
mhrtech
3
970
Featured
See All Featured
Imperfection Machines: The Place of Print at Facebook
scottboms
269
13k
A Tale of Four Properties
chriscoyier
160
23k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
9
570
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
960
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
45
2.5k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
The Language of Interfaces
destraynor
162
25k
Scaling GitHub
holman
463
140k
A Modern Web Designer's Workflow
chriscoyier
697
190k
The Cult of Friendly URLs
andyhume
79
6.6k
Transcript
SCALE AGNOSTIC PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT
ANY CODE CHANGES*
SEAN T. ALLEN VP OF ENGINEERING AT SENDENCE AUTHOR OF
“STORM APPLIED” @SEANTALLEN
SCALE AGNOSTIC COMPUTING
THE PROBLEM THE LITTLE APPLICATION THAT COULD… UNTIL IT COULDN’T.
Scaling an application from “little data” to “a little more
data” to “big data” is hard.
OPTIONS
Rewrite the application Rewrite to scale across multiple processes Example:
“multiprocessing” Python library
Run more versions Smart loader to load data in. Perhaps
via a job queue like Celery.
Go scale agnostic Write you application against a scale agnostic
API that runs on a scale aware platform
SENDENCE WALLAROO SCALE AGNOSTIC COMPUTING FOR PYTHON
Sendence Wallaroo Simple Scale Agnostic API Managed in-memory state Guaranteed
Message Processing
“We handle the hard infrastructure plumbing so you don’t have
to.” – The Wallaroo Team
LEARN MORE… sendence.com Sendence Engineering on Twitter: @SendenceEng Pat Helland’s
“Life Beyond Distributed Transactions”