Scale Agnostic Python

Scale Agnostic Python

Presented to the NYC Python meetup August 17, 2017

3c53e91d2a6ceb1b7f202d709f638b1b?s=128

Sean T Allen

August 17, 2017
Tweet

Transcript

  1. SCALE AGNOSTIC PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT

    ANY CODE CHANGES*
  2. SEAN T. ALLEN VP OF ENGINEERING AT SENDENCE AUTHOR OF

    “STORM APPLIED” @SEANTALLEN
  3. SCALE AGNOSTIC COMPUTING

  4. THE PROBLEM THE LITTLE APPLICATION THAT COULD… UNTIL IT COULDN’T.

  5. Scaling an application from “little data” to “a little more

    data” to “big data” is hard.
  6. OPTIONS

  7. Rewrite the application Rewrite to scale across multiple processes Example:

    “multiprocessing” Python library
  8. Run more versions Smart loader to load data in. Perhaps

    via a job queue like Celery.
  9. Go scale agnostic Write you application against a scale agnostic

    API that runs on a scale aware platform
  10. SENDENCE WALLAROO SCALE AGNOSTIC COMPUTING FOR PYTHON

  11. Sendence Wallaroo Simple Scale Agnostic API Managed in-memory state Guaranteed

    Message Processing
  12. “We handle the hard infrastructure plumbing so you don’t have

    to.” – The Wallaroo Team
  13. LEARN MORE… sendence.com Sendence Engineering on Twitter: @SendenceEng Pat Helland’s

    “Life Beyond Distributed Transactions”