Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scale Agnostic Python
Search
Sean T Allen
August 17, 2017
Programming
0
180
Scale Agnostic Python
Presented to the NYC Python meetup August 17, 2017
Sean T Allen
August 17, 2017
Tweet
Share
More Decks by Sean T Allen
See All by Sean T Allen
On Deny Capabilities for Safe, Fast Actors
seantallen
1
82
Data corrupting architectures we know and love - VelocityConf San Jose 2019
seantallen
1
93
Data corrupting architectures we know and love
seantallen
0
98
Scale-Independent Python
seantallen
2
210
Pat Helland and Me: How to Build Stateful Distributed Applications That Can Scale Almost Infinitely - Velocity NY October 2018
seantallen
0
75
Adventures in Cgo Preformance
seantallen
0
570
Pat Helland and me: A talk about “Life Beyond Distributed Transactions: An Apostate’s Opinion”
seantallen
0
520
Pat Helland and me: How to build stateful distributed applications that can scale almost infinitely - Salesforce July 2018
seantallen
0
160
Pony: How I learned to stop worrying and embrace an unproven technology - OSCON 2018
seantallen
0
360
Other Decks in Programming
See All in Programming
Ruby GitHub Packages
bkuhlmann
0
630
Zero Waste, Radical Magic, and Italian Graft – Quarkus Efficiency Secrets
hollycummins
0
230
PHP8.3の機能を振り返る / Review of PHP 8.3 features
seike460
PRO
1
110
0→1と1→10の狭間で Javaという技術選定を振り返る/Reflecting on the Decision to Choose Java Between Scaling from 0 to 1 and 1 to 10
jaguar_imo
2
380
AWS CDKコントリビュートTIPS / aws-cdk-contribution-tips
gotok365
2
200
新宿ダンジョンを可視化してみた
satoshi7190
2
260
Tailwind CSSを本気でカスタマイズする方法
fsubal
13
5.3k
見た目から始める生産性向上
ikumatadokoro
7
850
GitHub Actionsで泣かないためにやっておきたい設定 / Recommended GHA settings to avoid crying
pinkumohikan
3
540
ADRを一年運用してみた/adr_after_a_year
hanhan1978
7
2.4k
Snowflakeで眠ったデータを起こそう!
estie
0
120
1BRC--Nerd Sniping the Java Community
gunnarmorling
0
340
Featured
See All Featured
Web development in the modern age
philhawksworth
202
10k
Building Applications with DynamoDB
mza
88
5.6k
What’s in a name? Adding method to the madness
productmarketing
PRO
16
2.6k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
40
4.4k
4 Signs Your Business is Dying
shpigford
175
21k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
20
1.9k
What's in a price? How to price your products and services
michaelherold
237
11k
How to name files
jennybc
65
93k
Bootstrapping a Software Product
garrettdimon
PRO
302
110k
The Invisible Side of Design
smashingmag
294
49k
Designing the Hi-DPI Web
ddemaree
276
33k
Automating Front-end Workflow
addyosmani
1356
200k
Transcript
SCALE AGNOSTIC PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT
ANY CODE CHANGES*
SEAN T. ALLEN VP OF ENGINEERING AT SENDENCE AUTHOR OF
“STORM APPLIED” @SEANTALLEN
SCALE AGNOSTIC COMPUTING
THE PROBLEM THE LITTLE APPLICATION THAT COULD… UNTIL IT COULDN’T.
Scaling an application from “little data” to “a little more
data” to “big data” is hard.
OPTIONS
Rewrite the application Rewrite to scale across multiple processes Example:
“multiprocessing” Python library
Run more versions Smart loader to load data in. Perhaps
via a job queue like Celery.
Go scale agnostic Write you application against a scale agnostic
API that runs on a scale aware platform
SENDENCE WALLAROO SCALE AGNOSTIC COMPUTING FOR PYTHON
Sendence Wallaroo Simple Scale Agnostic API Managed in-memory state Guaranteed
Message Processing
“We handle the hard infrastructure plumbing so you don’t have
to.” – The Wallaroo Team
LEARN MORE… sendence.com Sendence Engineering on Twitter: @SendenceEng Pat Helland’s
“Life Beyond Distributed Transactions”