Python and Asynchrony

Python and Asynchrony

Presented at Jobspace on 14 October, 2016.

An introduction to asynchrony programming in Python. This presentation consists of two parts: First I introduce the idea behind asynchrony, and how it differs from more “traditional” (in Python sense) concurrent solutions.

In the second part I use code examples to walk through the development of async paradigms, introducing async through callbacks, promises, and finally coroutine, so we get a deeper understanding how the underlying plumbing works when you run an async program.

Code examples for the second part of the presentation can be found at http://bit.ly/asyncio-examples.

9dafad54b5b4f360b7aae5f482bc1c91?s=128

Tzu-ping Chung

October 14, 2016
Tweet

Transcript

  1. Python and Asynchrony

  2. http://bit.ly/asyncio-examples

  3. Quick Questions • Concurrency with Python • Threads and multi-processing

    • Single-thread asynchrony • asyncio, Twisted, Tornado, etc.
  4. Me • Call me TP • Follow @uranusjr • https://uranusjr.com

  5. None
  6. None
  7. http://macdown.uranusjr.com

  8. www. .com

  9. 10–11 June 2017 (Ծ) 7

  10. None
  11. Agenda • Introduction to asynchrony • Async the asyncio way

    • The road ahead
  12. None
  13. Waiting (wasted) Synchronous

  14. Threads

  15. Threads • More than you need • Race conditions •

    Dead-locking • Resource starvation
  16. To share, or not to share: that is the question.

  17. LET’S TALK ABOUT SOMETHING “DIFFERENT”

  18. Context Switching • First as a CPU thing • Interrupts

    • Software interrupts
  19. Multitasking • A common OS problem • More tasks than

    threads • Cooperative • Pre-emptive
  20. None
  21. Same problem?

  22. Threads

  23. Scheduling

  24. Asynchronous

  25. Sync vs. Async

  26. ASYNCIO

  27. Async How? • Callbacks • Promises • Coroutines

  28. Event Loop

  29. Launch Initialise Running Cleanup Terminate Register callbacks Receive events

  30. import asyncio # Init code goes here. loop = asyncio.get_event_loop()

    loop.run_forever()
  31. import asyncio # Init code goes here. def print_and_exit(): print('Hello

    asyncio!') asyncio.get_event_loop().stop() loop = asyncio.get_event_loop() loop.call_soon(print_and_exit) loop.run_forever()
  32. CALLBACK When the going gets rough.

  33. Pyramid of Doom $(function () { $("button").click(function (e) { $.get(URL,

    function (data) { $(".list").each(function () { $(this).click(function () { setTimeout(function () { alert("Top of the world"); }, 1000); }); }); }); }); });
  34. WELCOME TO CALLBACK HELL

  35. def send_repos_request(): ... task = ... task.add_done_callback(...) def read_repos(future): ...

    = future.result() ... task = ... task.add_done_callback(...) def send_forked_repo_requests(future): ... = future.result() ... task = ... task.add_done_callback(...) def read_forked_repos(future): ... = future.result() ... task = ... task.add_done_callback(...)
  36. Promise

  37. def main(): (send_repos_request() .then(read_repos) .then(send_forked_repo_requests, cleanup) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

  38. def main(): (send_repos_request() .then(read_repos) .then(send_forked_repo_requests, cleanup) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

  39. def main(): (send_repos_request() .then(read_repos) .then(send_forked_repo_requests, cleanup) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

  40. def main(): (send_repos_request() .then(read_repos) .then(send_forked_repo_requests, cleanup) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

  41. def main(): (send_repos_request() .then(read_repos) .catch(cleanup) .then(send_forked_repo_requests) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

  42. def main(): (send_repos_request() .then(read_repos) .catch(cleanup) .then(send_forked_repo_requests) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

  43. ???????

  44. Coroutine

  45. async def run(session): response = await send_repos_request(session) try: data =

    await read_repos(response) except APIError: return responses = await send_forked_repo_requests(session, data) datasets = await read_forked_repo_requests(responses) print_source_names(datasets)
  46. async def run(session): response = await send_repos_request(session) try: data =

    await read_repos(response) except APIError: return responses = await send_forked_repo_requests(session, data) datasets = await read_forked_repo_requests(responses) print_source_names(datasets)
  47. None
  48. Everything Has a Counterpart • async def / await •

    async with • async for (Python 3.6)
  49. class AsyncContextManager: async def __aenter__(self): await log('entering context') async def

    __aexit__(self, exc_type, exc, tb): await log('exiting context')
  50. class AsyncIterable: def __aiter__(self): return self async def __anext__(self): data

    = await ... if data: return data else: raise StopAsyncIteration
  51. Python asyncio.

  52. Wrapping Up • What is asynchrony • How asynchrony works

    • Asynchrony and asyncio
  53. Further Watching • A gentle introduction to asyncio • What

    in the World is Asyncio? • Topics of Interest (Python Asyncio) • You Might Not Want Async • and more…!
  54. None
  55. None