Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scale Agnostic Python
Search
Sean T Allen
August 17, 2017
Programming
0
210
Scale Agnostic Python
Presented to the NYC Python meetup August 17, 2017
Sean T Allen
August 17, 2017
Tweet
Share
More Decks by Sean T Allen
See All by Sean T Allen
On Deny Capabilities for Safe, Fast Actors
seantallen
1
100
Data corrupting architectures we know and love - VelocityConf San Jose 2019
seantallen
1
110
Data corrupting architectures we know and love
seantallen
0
110
Scale-Independent Python
seantallen
2
220
Pat Helland and Me: How to Build Stateful Distributed Applications That Can Scale Almost Infinitely - Velocity NY October 2018
seantallen
0
84
Adventures in Cgo Preformance
seantallen
0
650
Pat Helland and me: A talk about “Life Beyond Distributed Transactions: An Apostate’s Opinion”
seantallen
0
620
Pat Helland and me: How to build stateful distributed applications that can scale almost infinitely - Salesforce July 2018
seantallen
0
170
Pony: How I learned to stop worrying and embrace an unproven technology - OSCON 2018
seantallen
0
480
Other Decks in Programming
See All in Programming
From the Wild into the Clouds - Laravel Meetup Talk
neverything
0
140
クリーンアーキテクチャから見る依存の向きの大切さ
shimabox
5
1k
Honoのおもしろいミドルウェアをみてみよう
yusukebe
1
230
Learning Kotlin with detekt
inouehi
1
110
Unity Android XR入門
sakutama_11
0
180
『テスト書いた方が開発が早いじゃん』を解き明かす #phpcon_nagoya
o0h
PRO
8
2.4k
データの整合性を保つ非同期処理アーキテクチャパターン / Async Architecture Patterns
mokuo
54
19k
Django NinjaによるAPI開発の効率化とリプレースの実践
kashewnuts
1
250
データベースのオペレーターであるCloudNativePGがStatefulSetを使わない理由に迫る
nnaka2992
0
230
Domain-Driven Transformation
hschwentner
2
1.9k
未経験でSRE、はじめました! 組織を支える役割と軌跡
curekoshimizu
1
150
15分で学ぶDuckDBの可愛い使い方 DuckDBの最近の更新
notrogue
3
510
Featured
See All Featured
Typedesign – Prime Four
hannesfritz
40
2.5k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
666
120k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
[RailsConf 2023] Rails as a piece of cake
palkan
53
5.3k
A designer walks into a library…
pauljervisheath
205
24k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
7
640
The Cult of Friendly URLs
andyhume
78
6.2k
Fontdeck: Realign not Redesign
paulrobertlloyd
83
5.4k
Large-scale JavaScript Application Architecture
addyosmani
511
110k
What’s in a name? Adding method to the madness
productmarketing
PRO
22
3.3k
jQuery: Nuts, Bolts and Bling
dougneiner
63
7.7k
Transcript
SCALE AGNOSTIC PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT
ANY CODE CHANGES*
SEAN T. ALLEN VP OF ENGINEERING AT SENDENCE AUTHOR OF
“STORM APPLIED” @SEANTALLEN
SCALE AGNOSTIC COMPUTING
THE PROBLEM THE LITTLE APPLICATION THAT COULD… UNTIL IT COULDN’T.
Scaling an application from “little data” to “a little more
data” to “big data” is hard.
OPTIONS
Rewrite the application Rewrite to scale across multiple processes Example:
“multiprocessing” Python library
Run more versions Smart loader to load data in. Perhaps
via a job queue like Celery.
Go scale agnostic Write you application against a scale agnostic
API that runs on a scale aware platform
SENDENCE WALLAROO SCALE AGNOSTIC COMPUTING FOR PYTHON
Sendence Wallaroo Simple Scale Agnostic API Managed in-memory state Guaranteed
Message Processing
“We handle the hard infrastructure plumbing so you don’t have
to.” – The Wallaroo Team
LEARN MORE… sendence.com Sendence Engineering on Twitter: @SendenceEng Pat Helland’s
“Life Beyond Distributed Transactions”