Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scale Agnostic Python
Search
Sean T Allen
August 17, 2017
Programming
0
190
Scale Agnostic Python
Presented to the NYC Python meetup August 17, 2017
Sean T Allen
August 17, 2017
Tweet
Share
More Decks by Sean T Allen
See All by Sean T Allen
On Deny Capabilities for Safe, Fast Actors
seantallen
1
93
Data corrupting architectures we know and love - VelocityConf San Jose 2019
seantallen
1
97
Data corrupting architectures we know and love
seantallen
0
100
Scale-Independent Python
seantallen
2
210
Pat Helland and Me: How to Build Stateful Distributed Applications That Can Scale Almost Infinitely - Velocity NY October 2018
seantallen
0
77
Adventures in Cgo Preformance
seantallen
0
600
Pat Helland and me: A talk about “Life Beyond Distributed Transactions: An Apostate’s Opinion”
seantallen
0
550
Pat Helland and me: How to build stateful distributed applications that can scale almost infinitely - Salesforce July 2018
seantallen
0
160
Pony: How I learned to stop worrying and embrace an unproven technology - OSCON 2018
seantallen
0
400
Other Decks in Programming
See All in Programming
CSC307 Lecture 08
javiergs
PRO
0
330
Rubyのパフォーマンスプロファイリングの改善 / Enhancing performance profiling for Ruby
osyoyu
1
410
GraphQL はいいぞ! ~Laravel で学ぶ GraphQL 入門~
azuki
1
160
「2024年版 Kotlin サーバーサイドプログラミング実践開発」の補講 〜O/Rマッパー編〜
n_takehata
2
260
Jetpack for KMP
fornewid
1
290
DMMプラットフォームにおけるTiDBの導入から運用まで
pospome
7
3k
Xcode 16のPreviewModifierと@Previewableを活用した効率的なプレビュー方法の考察
ojun9
2
160
日付と正規化
megmogmog1965
0
140
20240706_CDKConf
takuyay0ne
0
1.2k
スクラムマスターって孤独じゃないですか?
yoshitaroyoyo
1
140
Ruby メモリ管理 プログラミング
megmogmog1965
0
130
実用的かつリーズナブルな 「Azure × Gemini × LINE」~キャラクターBot 実装ライブデモ~
tomodo_ysys
1
170
Featured
See All Featured
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
277
13k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
502
140k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
23
1.9k
Scaling GitHub
holman
458
140k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
26
1.6k
Practical Orchestrator
shlominoach
185
10k
It's Worth the Effort
3n
181
27k
Designing the Hi-DPI Web
ddemaree
276
34k
Git: the NoSQL Database
bkeepers
PRO
423
64k
How STYLIGHT went responsive
nonsquared
93
5k
Teambox: Starting and Learning
jrom
130
8.6k
Put a Button on it: Removing Barriers to Going Fast.
kastner
58
3.3k
Transcript
SCALE AGNOSTIC PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT
ANY CODE CHANGES*
SEAN T. ALLEN VP OF ENGINEERING AT SENDENCE AUTHOR OF
“STORM APPLIED” @SEANTALLEN
SCALE AGNOSTIC COMPUTING
THE PROBLEM THE LITTLE APPLICATION THAT COULD… UNTIL IT COULDN’T.
Scaling an application from “little data” to “a little more
data” to “big data” is hard.
OPTIONS
Rewrite the application Rewrite to scale across multiple processes Example:
“multiprocessing” Python library
Run more versions Smart loader to load data in. Perhaps
via a job queue like Celery.
Go scale agnostic Write you application against a scale agnostic
API that runs on a scale aware platform
SENDENCE WALLAROO SCALE AGNOSTIC COMPUTING FOR PYTHON
Sendence Wallaroo Simple Scale Agnostic API Managed in-memory state Guaranteed
Message Processing
“We handle the hard infrastructure plumbing so you don’t have
to.” – The Wallaroo Team
LEARN MORE… sendence.com Sendence Engineering on Twitter: @SendenceEng Pat Helland’s
“Life Beyond Distributed Transactions”