2017 - Nick DiRienzo, Vinay Tota - How Optimizely Scaled its REST API with Asyncio

Description
With developers, an awesome product isn’t everything, or at least we found that out. More than a product you need a platform. But what is a platform? Learn tips and tricks about building a public API using the latest and greatest tools: OpenAPI, Python 3 and asyncio.

Abstract
With developers, an awesome product isn’t everything, or at least we found that out. More than a product, you need a platform. But what is a platform? And how do you build one alongside all the other weird old code you might have inherited?

In a perfect world, Jeff Dean would ride in on a Unicorn with stone tablets inscribed with UML diagrams of an application architecture so perfect you’d weep with joy. In the real world however that tends not to happen, so the process is a bit more iterative. Sometimes you try things that don’t work or seem silly in retrospect.

Part of our platform journey has been building a public API with our favorite hand picked technologies: OpenAPI and Python 3, especially asyncio. Building a rock solid, fast, and scalable Python application was not all rainbows and unicorns along the way. Through this process, we learned quite a bit about describing REST APIs and how modern Python concurrency mechanisms behave in the real world.

We’ll share how we used OpenAPI to power our public API and improve the developer experience for both internal engineers and external consumers.

We started off with a mix of multiprocessing and asyncio, and have since migrated completely to asyncio. Also, we’ll share a few of the things we've learned in the process, which we hope will help you if you choose to use asyncio.

Bios
Nick DiRienzo is a software engineer at Optimizely. He enjoys writing code in Python, making developers’ lives better, and eating Mission burritos.

Vinay Tota is currently a member of the Application Platform team at Optimizely. A long time ago wanted to be a theoretical physicist but now has way more fun hacking code. More recently he's worked on building high throughput low latency systems for ad serving using Python.
Speaker home page

Transcript

1. How Optimizely scaled its REST API with asyncio
2. Vinay Tota @vinaytota Senior Software Engineer Optimizely Nick DiRienzo @nickdirienzo Software Engineer Optimizely @Optimizely
3. We’ll talk about: • Optimizely’s API story • What makes an API “good” • Optimizely’s API gateway • Improving it with asyncio • Lessons learned
4. Where we started Where we started
5. Where we started
6. Optimizely Classic was an A/B Testing Solution
7. Optimizely X is an experimentation platform
8. Optimizely X Full Stack Code Based Experimentation Optimizely X Web Visual Editor Based Experimentation Experimentation Platform
11. What is happening?
15. How can we turn a good web UI into a good API?
16. What makes an API “good”?
17. Easy to use APIs What makes an API “good”?
18. More than mapping a data model to an HTTP endpoint
19. API is the UI for developers
20. TODO: Add results slide.
21. Getting Optimizely Results 1. Get Experiment data
22. Getting Optimizely Results 1. Get Experiment data 2. Get Layer data
23. Getting Optimizely Results 1. Get Experiment data 2. Get Layer data (whatever that is)
24. Getting Optimizely Results 1. Get Experiment data 2. Get Layer data (whatever that is) 3. Get Project Settings
25. Getting Optimizely Results 1. Get Experiment data 2. Get Layer data (whatever that is) 3. Get Project Settings 4. Get results data
26. 1. Get Experiment data 2. Get Layer data (whatever that is) 3. Get Project Settings 4. Get results data GET /v2/experiments/:id/results Getting Optimizely Results
27. Easy to use APIs What makes an API “good”? Ensured Trust
28. Build a business on our APIs
29. How do we ensure trust? 1. No surprises
30. How do we ensure trust? 1. No surprises 2.Reliable platform
31. How do we ensure trust? 1. No surprises 2.Reliable platform 3. Performant platform
32. Trust can be made with: Interface Stability
34. Interface stability means... Beta & Stable APIs No breaking changes Versioning
35. Trust can be made with: Interface Stability Reliability
36. 99.999%
38. Trust can be made with: Interface Stability Reliability Performance
39. Performance is mission critical
40. Performance can be a dealbreaker
41. Trust can be made with: Interface Stability Reliability Performance
42. Easy to use APIs What makes an API “good”? Ensured Trust Documentation
43. Write documentation
44. Write documentation
45. Generate documentation
46. Generate documentation with swagger-ui to ensure docs are... Up to Date Comprehensive Accurate
48. Easy to use APIs What makes an API “good”? Ensured Trust Documentation
49. How can we provide “good” APIs? How can we provide “good” APIs?
50. Getting Optimizely Results 1. Get Experiment data 2. Get Layer data (whatever that is) 3. Get Project Settings 4. Get results data
51. 1. Get Experiment data 2. Get Layer data (whatever that is) 3. Get Project Settings 4. Get results data /v2/experiments/:id/results Getting Optimizely Results
52. We’ve got data all over Experiments Projects Results API Gateway
53. Benefits of an API gateway Operations through the API look a lot like their web UI counterparts
54. Benefits of an API gateway Operations through the API look a lot like their web UI counterparts Allows us to orchestrate APIs instead of data models
55. Benefits of an API gateway Operations through the API look a lot like their web UI counterparts Allows us to orchestrate APIs instead of data models Pulls problems away from business logic into a higher-level
56. Benefits of an API gateway Operations through the API look a lot like their web UI counterparts Allows us to orchestrate APIs instead of data models Pulls problems away from business logic into a higher-level Internal and external consumers go through the same application
57. api.optimizely.com What we’ve got
58. Python 3 + Pyramid + pyramid_swagger
59. We need to do a bunch of things in parallel...
60. Pyramid and Multiprocessing with multiprocessing.Pool() as pool, multiprocessing.Manager() as manager: response_queue = manager.Queue() try: # send our requests get back responses from the response_queue finally: pool.join() pool.terminate()
61. An Unhealthy Situation Textbook case for non blocking IO AWS Elastic Load Balancer 5XX
62. Not the Experience We Want
63. Blocking IO: “Normal” Python Application Kernel Start read() Initiate read read() returns Receive response
64. Nonblocking IO Application Kernel Start read() Initiate read, returns immediately read() Receive response poll()until data has arrived
65. Nonblocking IO Application Kernel read() Receive response poll(5) poll(5) poll(5)
66. Non Blocking IO 2002 2004 2009 2006 2008 2015 2012
67. What’s it good for? Proxy
68. What’s it good for? Push Updates/Chat
69. How do you schedule this work?
70. What framework?
71. Enter Asyncio Python 3.4
72. async def i_take_a_while(): print('slow func start') await asyncio.sleep(3) print('slow func done') async def i_run_fast(): print('fast func start') await asyncio.sleep(1) print('fast func done') async def run_me(): await asyncio.gather( i_run_fast(), i_take_a_while() ) loop = asyncio.get_event_loop() loop.run_until_complete(run_me()) loop.close() Event loop i_take_a_while i_run_fast slow func start fast func start 1 sec after start fast func done 3 sec after start slow func done
73. Blocking -> Asyncio Gunicorn Pyramid Pyramid View Switch to asyncio worker class http://docs.gunicorn.org/en/stable/design.html#async io-workers
74. Blocking -> Asyncio Gunicorn Pyramid Pyramid View Switch to aiopyramid https://github.com/housleyjk/aiopyramid
75. Blocking -> Asyncio Gunicorn Pyramid Pyramid View We can rewrite the views incrementally? Right?
76. async def i_block(): print('slow func start') time.sleep(3) print('slow func done') async def i_run_fast(): print('fast func start') await asyncio.sleep(1) print('fast func done') async def run_me(): await asyncio.gather( i_run_fast(), i_block() ) loop = asyncio.get_event_loop() loop.run_until_complete(run_me()) loop.close() Event loop i_block i_run_fast block func start fast func start 3 sec later block func done fast func done
77. Blocking and Nonblocking IO Don’t Mix
78. asyncio all the things
79. How Do You Make Big Changes?
80. Code is Social • Better make sure it actually works! • Pull Request Open 3+ weeks • Everyone on team added as a reviewer • 63 comments on Pull Request • Meetings for questions/concerns
81. Happily ever after AWS Elastic Load Balancer 5XX
82. Happily ever after 99th percentile latency
83. asyncio • asyncio from the start is ideal • But can add it later if you have to • Still early, but support will only improve
84. Good APIs are Hard • Lots of details to get right • Using the right technologies helps • Always keep iterating and improving
85. Come visit us at our booth! Try out Optimizely: optimizely.com/get-started @Optimizely
86. Q&A