Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scale Agnostic Python

3c53e91d2a6ceb1b7f202d709f638b1b?s=47 Sean T Allen
August 17, 2017
Programming
0
140

Scale Agnostic Python

Presented to the NYC Python meetup August 17, 2017

3c53e91d2a6ceb1b7f202d709f638b1b?s=128

Sean T Allen

August 17, 2017
Tweet

More Decks by Sean T Allen

See All by Sean T Allen
On Deny Capabilities for Safe, Fast Actors
3c53e91d2a6ceb1b7f202d709f638b1b?s=48 seantallen
0
61
Data corrupting architectures we know and love - VelocityConf San Jose 2019
3c53e91d2a6ceb1b7f202d709f638b1b?s=48 seantallen
1
76
Data corrupting architectures we know and love
3c53e91d2a6ceb1b7f202d709f638b1b?s=48 seantallen
0
85
Scale-Independent Python
3c53e91d2a6ceb1b7f202d709f638b1b?s=48 seantallen
2
170
Pat Helland and Me: How to Build Stateful Distributed Applications That Can Scale Almost Infinitely - Velocity NY October 2018
3c53e91d2a6ceb1b7f202d709f638b1b?s=48 seantallen
0
52
Adventures in Cgo Preformance
3c53e91d2a6ceb1b7f202d709f638b1b?s=48 seantallen
0
460
Pat Helland and me: A talk about “Life Beyond Distributed Transactions: An Apostate’s Opinion”
3c53e91d2a6ceb1b7f202d709f638b1b?s=48 seantallen
0
430
Pat Helland and me: How to build stateful distributed applications that can scale almost infinitely - Salesforce July 2018
3c53e91d2a6ceb1b7f202d709f638b1b?s=48 seantallen
0
140
Pony: How I learned to stop worrying and embrace an unproven technology - OSCON 2018
3c53e91d2a6ceb1b7f202d709f638b1b?s=48 seantallen
0
180

Other Decks in Programming

See All in Programming
From Java 11 to 17 and beyond
F784a8de872a5d98f67ac5ccbd6683d0?s=48 josepaumard
0
280
Improve Build Times in Less Time
85a1166e93654865b4dcafdafe2b2dfd?s=48 zacsweers
6
2.7k
TDX22: User-Mode DB Ops
3cf3d4d85fad3d196c1127018a370418?s=48 ca_peterson
3
1.1k
heyにおけるSREの大切さ~マルチプロダクト運用の「楽しさ」と「難しさ」および今後の展望~
4631864606a93224804a49331d1a7dfd?s=48 fufuhu
3
1.4k
Node.js 最新動向 TFCon 2022
D76231a2114896dfcc7b79ac69558b79?s=48 yosuke_furukawa
PRO
5
2.5k
コードの解析と言語習得の心得
72dae9d4524988d16990da682116fc8a?s=48 jinjin33333
0
120
Securing Kafka Connect Pipelines with Client-Side Field Level Cryptography @ Kafka Summit London 2022
744f1c2c6cbea2ff5104b0ac512936bd?s=48 hpgrahsl
0
300
SPA/MPA 議論の俯瞰と 現代における設計のポイント - #tfcon 2022 フロントエンド設計
1644fba8a219c89987390ef4f23d06bf?s=48 ahomu
2
1.4k
Angular's new Standalone Components: How Will They Affect My Architecture? @iJS London 2022
15934fa2aa7b2ce21f091e9b7cffa856?s=48 manfredsteyer
PRO
0
380
Keep Your Cache Always Fresh With Debezium
8e25c0ca4bf25113bd9c0ccc5d118164?s=48 gunnarmorling
0
170
PublishでWebサイトを構築してみた / generate_website_with_publish
42a6a049ac8f5265f31858a9509217fb?s=48 uhooi
2
110
The future of trust stores in Python
6638acfdb9c5979d11b4312984292c55?s=48 sethmlarson
0
180

Featured

See All Featured
Web development in the modern age
465724d73fe3a92c0879fdfb43a3a6f3?s=48 philhawksworth
197
9.3k
Designing for Performance
245cee81a9c424266e5e401d844ea881?s=48 lara
596
63k
Automating Front-end Workflow
96270e4c3e5e9806cf7245475c00b275?s=48 addyosmani
1351
200k
Building Adaptive Systems
06f8b41980eb4c577fa40c41d5030c19?s=48 keathley
25
1.1k
How To Stay Up To Date on Web Technology
8081b26e05bb4354f7d65ffc34cbbd67?s=48 chriscoyier
780
250k
The Illustrated Children's Guide to Kubernetes
0438ae60cde4c7add6f9da48f28c15cc?s=48 chrisshort
14
34k
What the flash - Photography Introduction
5ce91fa49a613cbc3e20d5f96856473f?s=48 edds
61
9.8k
The Cult of Friendly URLs
Ded01cdab114abe4ec13511e4c25b9bb?s=48 andyhume
68
4.7k
Documentation Writing (for coders)
61857dafbd287b3027c4dcea9008ad3c?s=48 carmenhchung
48
2.5k
Web Components: a chance to create the future
E190023b66e2b8aa73a842b106920c93?s=48 zenorocha
303
40k
Building Flexible Design Systems
91cefdf141106fa10674aae74ce05890?s=48 yeseniaperezcruz
310
33k
YesSQL, Process and Tooling at Scale
D09fad3e36ae6841f54820be0e907f7a?s=48 rocio
157
12k

Transcript

  1. SCALE AGNOSTIC PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT

    ANY CODE CHANGES*

  2. SEAN T. ALLEN VP OF ENGINEERING AT SENDENCE AUTHOR OF

    “STORM APPLIED” @SEANTALLEN

  3. SCALE AGNOSTIC COMPUTING

  4. THE PROBLEM THE LITTLE APPLICATION THAT COULD… UNTIL IT COULDN’T.

  5. Scaling an application from “little data” to “a little more

    data” to “big data” is hard.

  6. OPTIONS

  7. Rewrite the application Rewrite to scale across multiple processes Example:

    “multiprocessing” Python library

  8. Run more versions Smart loader to load data in. Perhaps

    via a job queue like Celery.

  9. Go scale agnostic Write you application against a scale agnostic

    API that runs on a scale aware platform

  10. SENDENCE WALLAROO SCALE AGNOSTIC COMPUTING FOR PYTHON

  11. Sendence Wallaroo Simple Scale Agnostic API Managed in-memory state Guaranteed

    Message Processing

  12. “We handle the hard infrastructure plumbing so you don’t have

    to.” – The Wallaroo Team

  13. LEARN MORE… sendence.com Sendence Engineering on Twitter: @SendenceEng Pat Helland’s

    “Life Beyond Distributed Transactions”