Scale Agnostic Python

Transcript

1. 1.

   SCALE AGNOSTIC PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT

   ANY CODE CHANGES*
2. 2.

   SEAN T. ALLEN VP OF ENGINEERING AT SENDENCE AUTHOR OF

   “STORM APPLIED” @SEANTALLEN
3. 3.

   SCALE AGNOSTIC COMPUTING

4. 4.

   THE PROBLEM THE LITTLE APPLICATION THAT COULD… UNTIL IT COULDN’T.

5. 5.

   Scaling an application from “little data” to “a little more

   data” to “big data” is hard.
6. 6.

   OPTIONS

7. 7.

   Rewrite the application Rewrite to scale across multiple processes Example:

   “multiprocessing” Python library
8. 8.

   Run more versions Smart loader to load data in. Perhaps

   via a job queue like Celery.
9. 9.

   Go scale agnostic Write you application against a scale agnostic

   API that runs on a scale aware platform
10. 10.

    SENDENCE WALLAROO SCALE AGNOSTIC COMPUTING FOR PYTHON

11. 11.

    Sendence Wallaroo Simple Scale Agnostic API Managed in-memory state Guaranteed

    Message Processing
12. 12.

    “We handle the hard infrastructure plumbing so you don’t have

    to.” – The Wallaroo Team
13. 13.

    LEARN MORE… sendence.com Sendence Engineering on Twitter: @SendenceEng Pat Helland’s

    “Life Beyond Distributed Transactions”