Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Async Python. Who's there? Knock knock.

Marc Aubé
February 28, 2020

Async Python. Who's there? Knock knock.

async, await, generators, tasks, coroutines, oh my! Let's learn how this often overlooked module can help you write concurrent code, without starting multiple threads or using multiple CPU cores. You'll leave with a better understanding of what cooperative multitasking is and how you can use it in your own projects.

Marc Aubé

February 28, 2020
Tweet

More Decks by Marc Aubé

Other Decks in Technology

Transcript

  1. Async Python. Who's
    there? Knock knock.

    View full-size slide

  2. Parallelism, concurrency,
    multi-processing, threading,
    oh my!
    !
    @maaube

    View full-size slide

  3. Parallelism: performing
    multiple operations at the
    same time.
    @maaube

    View full-size slide

  4. Concurrency: performing
    multiple operations in an
    overlapping manner.
    @maaube

    View full-size slide

  5. As David said earlier today...
    @maaube

    View full-size slide

  6. Async I/O: a style of
    concurrent programming in
    which tasks release the CPU
    during idling periods, so that
    other tasks can use it.
    @maaube

    View full-size slide

  7. Which model is the best?
    @maaube

    View full-size slide

  8. It depends...
    1. heavy computations or maths (CPU bound)
    • multi-processing
    2. network, servers, HTTP, sockets (I/O bound)
    • multi-threading
    • asyncio
    @maaube

    View full-size slide

  9. How does asyncio work?
    @maaube

    View full-size slide

  10. Chess Exhibition
    • Assumptions
    • 24 opponents
    • Polgár moves in 5 seconds
    • Opponents move in 55 seconds
    • Games average 30 move pairs
    • Each game runs for 30 minutes
    @maaube

    View full-size slide

  11. Synchronous Chess Exhibition
    • Judit plays one game at a time
    • Each game runs for 30 minutes
    • There are 24 games to play
    • 24 sequential games would take:
    24 x 30 min = 720 min = 12 hours
    @maaube

    View full-size slide

  12. Asynchronous Chess Exhibition
    • Polgár makes a move
    • While the opponent thinks, she
    moves on the second game, then
    third, ...
    • A move on all 24 games takes her:
    24 x 5 sec = 120 sec = 2 minutes
    • 24 games are completed in:
    2 min x 30 = 60 min = 1 hour!
    @maaube

    View full-size slide

  13. Asynchronous Chess Exhibition
    • Polgár makes a move
    • While the opponent thinks, she
    moves on the second game, then
    third, ...
    • A move on all 24 games takes her:
    24 x 5 sec = 120 sec = 2 minutes
    • 24 games are completed in:
    2 min x 30 = 60 min = 1 hour!
    @maaube

    View full-size slide

  14. Cooperative concurrency
    A fancy way of saying that your async functions can give
    back control to the execution loop, so that each function
    can run at the optimal time.
    @maaube

    View full-size slide

  15. Understanding computer time
    @maaube

    View full-size slide

  16. async and await
    @maaube

    View full-size slide

  17. async_hello.py executed in 1.01 seconds.
    @maaube

    View full-size slide

  18. Made-up problem time!
    @maaube

    View full-size slide

  19. Goals
    1. Crawl Confoo's Flickr account
    2. Download all pictures from photo
    stream
    3. Make a 120x120 thumbnail for
    each of them
    4. Convert it to black & white
    5. Be fast!
    !
    @maaube

    View full-size slide

  20. This function is blocking...
    @maaube

    View full-size slide

  21. Goals
    1. Crawl Confoo's Flickr account
    2. Download all pictures from photo
    stream

    3. Make a 120x120 thumbnail for
    each of them

    4. Convert it to black & white
    5. Be fast!

    @maaube

    View full-size slide

  22. Going even faster!
    @maaube

    View full-size slide

  23. GitHub repo
    @maaube

    View full-size slide

  24. Pitfalls
    •long CPU-intensive tasks
    • execute in a background thread
    • routinely release CPU with await
    asyncio.sleep(0)
    •blocking library functions
    @maaube

    View full-size slide

  25. Resources
    @maaube

    View full-size slide

  26. Questions ?
    @maaube

    View full-size slide