$30 off During Our Annual Pro Sale. View Details »

Python and Asynchrony

Python and Asynchrony

Presented at Jobspace on 14 October, 2016.

An introduction to asynchrony programming in Python. This presentation consists of two parts: First I introduce the idea behind asynchrony, and how it differs from more “traditional” (in Python sense) concurrent solutions.

In the second part I use code examples to walk through the development of async paradigms, introducing async through callbacks, promises, and finally coroutine, so we get a deeper understanding how the underlying plumbing works when you run an async program.

Code examples for the second part of the presentation can be found at http://bit.ly/asyncio-examples.

Tzu-ping Chung

October 14, 2016
Tweet

More Decks by Tzu-ping Chung

Other Decks in Programming

Transcript

  1. Python and Asynchrony

    View Slide

  2. http://bit.ly/asyncio-examples

    View Slide

  3. Quick Questions
    • Concurrency with Python
    • Threads and multi-processing
    • Single-thread asynchrony
    • asyncio, Twisted, Tornado, etc.

    View Slide

  4. Me
    • Call me TP
    • Follow @uranusjr
    • https://uranusjr.com

    View Slide

  5. View Slide

  6. View Slide

  7. http://macdown.uranusjr.com

    View Slide

  8. www. .com

    View Slide

  9. 10–11 June 2017 (Ծ)
    7

    View Slide

  10. View Slide

  11. Agenda
    • Introduction to asynchrony
    • Async the asyncio way
    • The road ahead

    View Slide

  12. View Slide

  13. Waiting (wasted)
    Synchronous

    View Slide

  14. Threads

    View Slide

  15. Threads
    • More than you need
    • Race conditions
    • Dead-locking
    • Resource starvation

    View Slide

  16. To share, or not to share:
    that is the question.

    View Slide

  17. LET’S TALK ABOUT SOMETHING
    “DIFFERENT”

    View Slide

  18. Context Switching
    • First as a CPU thing
    • Interrupts
    • Software interrupts

    View Slide

  19. Multitasking
    • A common OS problem
    • More tasks than threads
    • Cooperative
    • Pre-emptive

    View Slide

  20. View Slide

  21. Same
    problem?

    View Slide

  22. Threads

    View Slide

  23. Scheduling

    View Slide

  24. Asynchronous

    View Slide

  25. Sync vs. Async

    View Slide

  26. ASYNCIO

    View Slide

  27. Async How?
    • Callbacks
    • Promises
    • Coroutines

    View Slide

  28. Event Loop

    View Slide

  29. Launch Initialise
    Running
    Cleanup Terminate
    Register
    callbacks
    Receive
    events

    View Slide

  30. import asyncio
    # Init code goes here.
    loop = asyncio.get_event_loop()
    loop.run_forever()

    View Slide

  31. import asyncio
    # Init code goes here.
    def print_and_exit():
    print('Hello asyncio!')
    asyncio.get_event_loop().stop()
    loop = asyncio.get_event_loop()
    loop.call_soon(print_and_exit)
    loop.run_forever()

    View Slide

  32. CALLBACK
    When the going gets rough.

    View Slide

  33. Pyramid of Doom
    $(function () {
    $("button").click(function (e) {
    $.get(URL, function (data) {
    $(".list").each(function () {
    $(this).click(function () {
    setTimeout(function () {
    alert("Top of the world");
    }, 1000);
    });
    });
    });
    });
    });

    View Slide

  34. WELCOME TO
    CALLBACK HELL

    View Slide

  35. def send_repos_request():
    ...
    task = ...
    task.add_done_callback(...)
    def read_repos(future):
    ... = future.result()
    ...
    task = ...
    task.add_done_callback(...)
    def send_forked_repo_requests(future):
    ... = future.result()
    ...
    task = ...
    task.add_done_callback(...)
    def read_forked_repos(future):
    ... = future.result()
    ...
    task = ...
    task.add_done_callback(...)

    View Slide

  36. Promise

    View Slide

  37. def main():
    (send_repos_request()
    .then(read_repos)
    .then(send_forked_repo_requests, cleanup)
    .then(read_forked_repo_requests)
    .then(print_source_names)
    .done(cleanup))

    View Slide

  38. def main():
    (send_repos_request()
    .then(read_repos)
    .then(send_forked_repo_requests, cleanup)
    .then(read_forked_repo_requests)
    .then(print_source_names)
    .done(cleanup))

    View Slide

  39. def main():
    (send_repos_request()
    .then(read_repos)
    .then(send_forked_repo_requests, cleanup)
    .then(read_forked_repo_requests)
    .then(print_source_names)
    .done(cleanup))

    View Slide

  40. def main():
    (send_repos_request()
    .then(read_repos)
    .then(send_forked_repo_requests, cleanup)
    .then(read_forked_repo_requests)
    .then(print_source_names)
    .done(cleanup))

    View Slide

  41. def main():
    (send_repos_request()
    .then(read_repos)
    .catch(cleanup)
    .then(send_forked_repo_requests)
    .then(read_forked_repo_requests)
    .then(print_source_names)
    .done(cleanup))

    View Slide

  42. def main():
    (send_repos_request()
    .then(read_repos)
    .catch(cleanup)
    .then(send_forked_repo_requests)
    .then(read_forked_repo_requests)
    .then(print_source_names)
    .done(cleanup))

    View Slide

  43. ???????

    View Slide

  44. Coroutine

    View Slide

  45. async def run(session):
    response = await send_repos_request(session)
    try:
    data = await read_repos(response)
    except APIError:
    return
    responses = await send_forked_repo_requests(session, data)
    datasets = await read_forked_repo_requests(responses)
    print_source_names(datasets)

    View Slide

  46. async def run(session):
    response = await send_repos_request(session)
    try:
    data = await read_repos(response)
    except APIError:
    return
    responses = await send_forked_repo_requests(session, data)
    datasets = await read_forked_repo_requests(responses)
    print_source_names(datasets)

    View Slide

  47. View Slide

  48. Everything Has a Counterpart
    • async def / await
    • async with
    • async for (Python 3.6)

    View Slide

  49. class AsyncContextManager:
    async def __aenter__(self):
    await log('entering context')
    async def __aexit__(self, exc_type, exc, tb):
    await log('exiting context')

    View Slide

  50. class AsyncIterable:
    def __aiter__(self):
    return self
    async def __anext__(self):
    data = await ...
    if data:
    return data
    else:
    raise StopAsyncIteration

    View Slide

  51. Python
    asyncio.

    View Slide

  52. Wrapping Up
    • What is asynchrony
    • How asynchrony works
    • Asynchrony and asyncio

    View Slide

  53. Further Watching
    • A gentle introduction to asyncio
    • What in the World is Asyncio?
    • Topics of Interest (Python Asyncio)
    • You Might Not Want Async
    • and more…!

    View Slide

  54. View Slide

  55. View Slide