Beyond Python's Comfort Zone: Asynchronous Python in 2016-2017

Python was not originally created with concurrency in mind, so with rise in async in modern popular programming languages, how has Python integrated with async in 2016 and 2017 ?

Transcript

1. Beyond Python's Comfort Zone: Asynchronous Python in 2016-2017 November 2016 Alex Jerez
2. About Me • Alex Jerez • Python Programmer at ABI Research in Oyster Bay • HTML/CSS, Javascript, Java, C#, Rust • Founded Crosshatch, a social platform, in 2012, flopped
3. “Life begins at the end of your comfort zone”
4. Python is at a crossroad A Question of Concurrency Python 2.x Python 3.x
5. Python 2.x Python 3.x • Legacy • Still shipped as default for many Platforms, such as Linux (Ubuntu, Debian, Fedora) • The most common version of Python being used, but the gap is shrinking. • Support ending on 2020 (2.7) • Future-Proof • In Active Development • Library support not as good as 2.x • Async ready
6. What is Concurrency? • Multitasking - Concurrent programs handle tasks that are all in progress at the same time, but it is only necessary to work briefly and separately on each task - Doing A while waiting for B • Improved Responsive Experience from end-user • Concurrency is not Parallelism ◦ Tasks are sharing the execution thread while still appearing to be executing in parallel.
7. What is Concurrency? • Concurrency is not for all operations • CPU-Bound Applications - Your CPU is blocking you from doing more work • IO-Bound Applications - You’re waiting on something else (HTTP Request)
8. Popular libraries for Concurrency • Tornado • Twisted • Gevent • greenlets Other Methods • Threading • Multiprocessing • Subprocess • Message Queue
9. 1999: Python 1.5.2 ships with asyncore, asynchat 2001: Python 2.2 ships with Generators 2006: Python 2.5 ships with sendable ‘yield’, coroutines 2012: Python 3.3 ships with ‘yield from’ 2015: Python 3.5 ships with ‘async/await’ & asyncio
10. Generators def gen(num): if num > 1: for item in range(1,num): yield item pass
11. Coroutines @asyncio.coroutine def get_json_data(url): data = yield from get_page(url) return json.loads(data)
12. Coroutines <coroutine>.send(value) Resumes the execution and “sends” a value into the generator function. The value argument becomes the result of the current yield expression. The send() method returns the next value yielded by the generator, or raises StopIteration if the generator exits without yielding another value.
14. async/await A subtype of generators
15. asyncio Module introduced in Python 3.4 asyncio is the of a number of useful third-party python tools. • Written to tackle concurrency natively. • Added to std library • Implements a flexible interface so that it can be tied in with frameworks like Twisted and Tornado
16. asyncio • Concise, easy to use, hard to master. • New builtin types for coroutines • async for and async with • Generic, can be tied to Tornado, gevent, etc. • 2x slower than a function call, 10-100x faster than a yield • @types.coroutine turns generators into coroutines • Event loops, futures, tasks, callbacks • Libraries for sockets
17. Main asyncio functions • Asyncio.get_event_loop --> Event Loop • loop.create_task() --> Task • loop.run_until_complete(), loop.run_forever() • asyncio.gather() • loop.run_in_executor() • loop.close()
18. The Event Loop • asyncio.get_event_loop ◦ run_forever() - run loop until stop() is called. ◦ stop() - ◦ run_until_complete(Future) - run until Future is finished.
19. Scheduling Tasks • You cannot await a sleep function • asyncio.get_event_loop ◦ call_soon(callback, *args) - trigger callback asap ◦ call_later(delay, callback, *args) - delay callback trigger by delay ◦ call_at(datetime, callback, *args)
20. asyncio in debug • It’s generally a good idea to run asyncio in debug mode, especially when learning asyncio ◦ loop.set_debug(True) ◦ PYTHONASYNCIODEBUG=1
21. asyncio criticism • Armin Ronacher’s Blog Post ◦ “I don't understand Python's Asyncio” ◦ http://lucumr.pocoo.org/2016/10/30/i-dont-understand-asyncio/-
22. “Should I continue with Python 2 or Python 3?”
23. “Should I use asyncio?”
24. “My project works fine, should I move to asyncio?”
25. “Is Python ready for asynchronous programming?”
26. Conclusion
27. THANK YOU!