Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scale Agnostic Python

Sean T Allen
August 17, 2017
Programming
0
140

Scale Agnostic Python

Presented to the NYC Python meetup August 17, 2017

Sean T Allen

August 17, 2017
Tweet

More Decks by Sean T Allen

See All by Sean T Allen
On Deny Capabilities for Safe, Fast Actors
seantallen
0
70
Data corrupting architectures we know and love - VelocityConf San Jose 2019
seantallen
1
84
Data corrupting architectures we know and love
seantallen
0
90
Scale-Independent Python
seantallen
2
180
Pat Helland and Me: How to Build Stateful Distributed Applications That Can Scale Almost Infinitely - Velocity NY October 2018
seantallen
0
55
Adventures in Cgo Preformance
seantallen
0
510
Pat Helland and me: A talk about “Life Beyond Distributed Transactions: An Apostate’s Opinion”
seantallen
0
470
Pat Helland and me: How to build stateful distributed applications that can scale almost infinitely - Salesforce July 2018
seantallen
0
150
Pony: How I learned to stop worrying and embrace an unproven technology - OSCON 2018
seantallen
0
250

Other Decks in Programming

See All in Programming
SREは何を目指すのか
paper2
5
1.3k
mrubyでマイコンの世界に足を踏み入れる
yuuu
0
450
Blazor WASMのアプリをMAUI Blazorに移行してみる
tomokusaba
0
140
GitHub ActionsとDeployGateで始めるAndroidアプリのCICD
yuhei_fujita
2
150
Kubernetes基盤を自律的に支えるController化の実装Tips / forkwell-202303-amsy810-k8s
masayaaoyama
5
2.8k
Coroutines Test 入門 / Android Test Night #8
star_zero
2
280
10Xにおけるdbtの事例紹介
10xinc
0
910
LayerXにおける機械学習を活用した請求書OCR機能に関する取り組み / deim2023-layerx-ai-ocr
yuya4
7
1.3k
オンサイトキックオフでロケットスタート〜貴重なリアル対面を活かすやり方と構造〜
sasakendayo
0
280
ゴーファーくんと学ぶGo言語の世界/golang-world-with-gopher
iwasiman
2
520
AT Protocol (BlueSky Social)仕様解説 ～ W3C DID仕様を添えて ～
gpsnmeajp
0
110
日常業務のカイゼンで図る開発チームへの貢献 - YAPC::Kyoto 2023
koluku
3
290

Featured

See All Featured
Reflections from 52 weeks, 52 projects
jeffersonlam
339
18k
StorybookのUI Testing Handbookを読んだ
zakiyama
8
3.4k
Why Our Code Smells
bkeepers
PRO
326
55k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
227
16k
What's new in Ruby 2.0
geeforr
336
30k
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
31
20k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
14
5.6k
Rails Girls Zürich Keynote
gr2m
87
12k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
152
13k
The Brand Is Dead. Long Live the Brand.
mthomps
48
3k
Building Your Own Lightsaber
phodgson
96
4.9k
Rebuilding a faster, lazier Slack
samanthasiow
69
7.6k

Transcript

  1. SCALE AGNOSTIC PYTHON
    HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT ANY CODE CHANGES*

    View Slide

  2. SEAN T. ALLEN
    VP OF ENGINEERING AT SENDENCE
    AUTHOR OF “STORM APPLIED”
    @SEANTALLEN

    View Slide

  3. SCALE AGNOSTIC COMPUTING

    View Slide

  4. THE PROBLEM
    THE LITTLE APPLICATION
    THAT COULD…
    UNTIL IT COULDN’T.

    View Slide

  5. Scaling an application from “little data” to “a little
    more data” to “big data” is hard.

    View Slide

  6. OPTIONS

    View Slide

  7. Rewrite the application
    Rewrite to scale across
    multiple processes

    Example: “multiprocessing”
    Python library

    View Slide

  8. Run more versions
    Smart loader to load data in.
    Perhaps via a job queue like
    Celery.

    View Slide

  9. Go scale agnostic
    Write you application against
    a scale agnostic API that runs
    on a scale aware platform

    View Slide

  10. SENDENCE
    WALLAROO
    SCALE AGNOSTIC
    COMPUTING FOR PYTHON

    View Slide

  11. Sendence Wallaroo
    Simple Scale Agnostic API

    Managed in-memory state

    Guaranteed Message Processing

    View Slide

  12. “We handle the hard infrastructure plumbing so
    you don’t have to.”
    – The Wallaroo Team

    View Slide

  13. LEARN MORE…
    sendence.com
    Sendence Engineering on Twitter: @SendenceEng
    Pat Helland’s “Life Beyond Distributed Transactions”

    View Slide