Slide 1

Slide 1 text

How Optimizely scaled its REST API with asyncio

Slide 2

Slide 2 text

Vinay Tota @vinaytota Senior Software Engineer Optimizely Nick DiRienzo @nickdirienzo Software Engineer Optimizely @Optimizely

Slide 3

Slide 3 text

We’ll talk about: • Optimizely’s API story • What makes an API “good” • Optimizely’s API gateway • Improving it with asyncio • Lessons learned

Slide 4

Slide 4 text

Where we started Where we started

Slide 5

Slide 5 text

Where we started

Slide 6

Slide 6 text

Optimizely Classic was an A/B Testing Solution

Slide 7

Slide 7 text

Optimizely X is an experimentation platform

Slide 8

Slide 8 text

Optimizely X Full Stack Code Based Experimentation Optimizely X Web Visual Editor Based Experimentation Experimentation Platform

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

What is happening?

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

How can we turn a good web UI into a good API?

Slide 16

Slide 16 text

What makes an API “good”?

Slide 17

Slide 17 text

Easy to use APIs What makes an API “good”?

Slide 18

Slide 18 text

More than mapping a data model to an HTTP endpoint

Slide 19

Slide 19 text

API is the UI for developers

Slide 20

Slide 20 text

TODO: Add results slide.

Slide 21

Slide 21 text

Getting Optimizely Results 1. Get Experiment data

Slide 22

Slide 22 text

Getting Optimizely Results 1. Get Experiment data 2. Get Layer data

Slide 23

Slide 23 text

Getting Optimizely Results 1. Get Experiment data 2. Get Layer data (whatever that is)

Slide 24

Slide 24 text

Getting Optimizely Results 1. Get Experiment data 2. Get Layer data (whatever that is) 3. Get Project Settings

Slide 25

Slide 25 text

Getting Optimizely Results 1. Get Experiment data 2. Get Layer data (whatever that is) 3. Get Project Settings 4. Get results data

Slide 26

Slide 26 text

1. Get Experiment data 2. Get Layer data (whatever that is) 3. Get Project Settings 4. Get results data GET /v2/experiments/:id/results Getting Optimizely Results

Slide 27

Slide 27 text

Easy to use APIs What makes an API “good”? Ensured Trust

Slide 28

Slide 28 text

Build a business on our APIs

Slide 29

Slide 29 text

How do we ensure trust? 1. No surprises

Slide 30

Slide 30 text

How do we ensure trust? 1. No surprises 2.Reliable platform

Slide 31

Slide 31 text

How do we ensure trust? 1. No surprises 2.Reliable platform 3. Performant platform

Slide 32

Slide 32 text

Trust can be made with: Interface Stability

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

Interface stability means... Beta & Stable APIs No breaking changes Versioning

Slide 35

Slide 35 text

Trust can be made with: Interface Stability Reliability

Slide 36

Slide 36 text

99.999%

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

Trust can be made with: Interface Stability Reliability Performance

Slide 39

Slide 39 text

Performance is mission critical

Slide 40

Slide 40 text

Performance can be a dealbreaker

Slide 41

Slide 41 text

Trust can be made with: Interface Stability Reliability Performance

Slide 42

Slide 42 text

Easy to use APIs What makes an API “good”? Ensured Trust Documentation

Slide 43

Slide 43 text

Write documentation

Slide 44

Slide 44 text

Write documentation

Slide 45

Slide 45 text

Generate documentation

Slide 46

Slide 46 text

Generate documentation with swagger-ui to ensure docs are... Up to Date Comprehensive Accurate

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

Easy to use APIs What makes an API “good”? Ensured Trust Documentation

Slide 49

Slide 49 text

How can we provide “good” APIs? How can we provide “good” APIs?

Slide 50

Slide 50 text

Getting Optimizely Results 1. Get Experiment data 2. Get Layer data (whatever that is) 3. Get Project Settings 4. Get results data

Slide 51

Slide 51 text

1. Get Experiment data 2. Get Layer data (whatever that is) 3. Get Project Settings 4. Get results data /v2/experiments/:id/results Getting Optimizely Results

Slide 52

Slide 52 text

We’ve got data all over Experiments Projects Results API Gateway

Slide 53

Slide 53 text

Benefits of an API gateway Operations through the API look a lot like their web UI counterparts

Slide 54

Slide 54 text

Benefits of an API gateway Operations through the API look a lot like their web UI counterparts Allows us to orchestrate APIs instead of data models

Slide 55

Slide 55 text

Benefits of an API gateway Operations through the API look a lot like their web UI counterparts Allows us to orchestrate APIs instead of data models Pulls problems away from business logic into a higher-level

Slide 56

Slide 56 text

Benefits of an API gateway Operations through the API look a lot like their web UI counterparts Allows us to orchestrate APIs instead of data models Pulls problems away from business logic into a higher-level Internal and external consumers go through the same application

Slide 57

Slide 57 text

api.optimizely.com What we’ve got

Slide 58

Slide 58 text

Python 3 + Pyramid + pyramid_swagger

Slide 59

Slide 59 text

We need to do a bunch of things in parallel...

Slide 60

Slide 60 text

Pyramid and Multiprocessing with multiprocessing.Pool() as pool, multiprocessing.Manager() as manager: response_queue = manager.Queue() try: # send our requests get back responses from the response_queue finally: pool.join() pool.terminate()

Slide 61

Slide 61 text

An Unhealthy Situation Textbook case for non blocking IO AWS Elastic Load Balancer 5XX

Slide 62

Slide 62 text

Not the Experience We Want

Slide 63

Slide 63 text

Blocking IO: “Normal” Python Application Kernel Start read() Initiate read read() returns Receive response

Slide 64

Slide 64 text

Nonblocking IO Application Kernel Start read() Initiate read, returns immediately read() Receive response poll()until data has arrived

Slide 65

Slide 65 text

Nonblocking IO Application Kernel read() Receive response poll(5) poll(5) poll(5)

Slide 66

Slide 66 text

Non Blocking IO 2002 2004 2009 2006 2008 2015 2012

Slide 67

Slide 67 text

What’s it good for? Proxy

Slide 68

Slide 68 text

What’s it good for? Push Updates/Chat

Slide 69

Slide 69 text

How do you schedule this work?

Slide 70

Slide 70 text

What framework?

Slide 71

Slide 71 text

Enter Asyncio Python 3.4

Slide 72

Slide 72 text

async def i_take_a_while(): print('slow func start') await asyncio.sleep(3) print('slow func done') async def i_run_fast(): print('fast func start') await asyncio.sleep(1) print('fast func done') async def run_me(): await asyncio.gather( i_run_fast(), i_take_a_while() ) loop = asyncio.get_event_loop() loop.run_until_complete(run_me()) loop.close() Event loop i_take_a_while i_run_fast slow func start fast func start 1 sec after start fast func done 3 sec after start slow func done

Slide 73

Slide 73 text

Blocking -> Asyncio Gunicorn Pyramid Pyramid View Switch to asyncio worker class http://docs.gunicorn.org/en/stable/design.html#async io-workers

Slide 74

Slide 74 text

Blocking -> Asyncio Gunicorn Pyramid Pyramid View Switch to aiopyramid https://github.com/housleyjk/aiopyramid

Slide 75

Slide 75 text

Blocking -> Asyncio Gunicorn Pyramid Pyramid View We can rewrite the views incrementally? Right?

Slide 76

Slide 76 text

async def i_block(): print('slow func start') time.sleep(3) print('slow func done') async def i_run_fast(): print('fast func start') await asyncio.sleep(1) print('fast func done') async def run_me(): await asyncio.gather( i_run_fast(), i_block() ) loop = asyncio.get_event_loop() loop.run_until_complete(run_me()) loop.close() Event loop i_block i_run_fast block func start fast func start 3 sec later block func done fast func done

Slide 77

Slide 77 text

Blocking and Nonblocking IO Don’t Mix

Slide 78

Slide 78 text

asyncio all the things

Slide 79

Slide 79 text

How Do You Make Big Changes?

Slide 80

Slide 80 text

Code is Social • Better make sure it actually works! • Pull Request Open 3+ weeks • Everyone on team added as a reviewer • 63 comments on Pull Request • Meetings for questions/concerns

Slide 81

Slide 81 text

Happily ever after AWS Elastic Load Balancer 5XX

Slide 82

Slide 82 text

Happily ever after 99th percentile latency

Slide 83

Slide 83 text

asyncio • asyncio from the start is ideal • But can add it later if you have to • Still early, but support will only improve

Slide 84

Slide 84 text

Good APIs are Hard • Lots of details to get right • Using the right technologies helps • Always keep iterating and improving

Slide 85

Slide 85 text

Come visit us at our booth! Try out Optimizely: optimizely.com/get-started @Optimizely

Slide 86

Slide 86 text

Q&A