Async Python. Who's there? Knock knock.

5e6bcf291601ee2e0faf35b30a839cb6?s=47 Marc Aubé
February 28, 2020

Async Python. Who's there? Knock knock.

async, await, generators, tasks, coroutines, oh my! Let's learn how this often overlooked module can help you write concurrent code, without starting multiple threads or using multiple CPU cores. You'll leave with a better understanding of what cooperative multitasking is and how you can use it in your own projects.

5e6bcf291601ee2e0faf35b30a839cb6?s=128

Marc Aubé

February 28, 2020
Tweet

Transcript

  1. Async Python. Who's there? Knock knock.

  2. Parallelism, concurrency, multi-processing, threading, oh my! ! @maaube

  3. Parallelism: performing multiple operations at the same time. @maaube

  4. Concurrency: performing multiple operations in an overlapping manner. @maaube

  5. As David said earlier today... @maaube

  6. Async I/O: a style of concurrent programming in which tasks

    release the CPU during idling periods, so that other tasks can use it. @maaube
  7. Which model is the best? @maaube

  8. It depends... 1. heavy computations or maths (CPU bound) •

    multi-processing 2. network, servers, HTTP, sockets (I/O bound) • multi-threading • asyncio @maaube
  9. How does asyncio work? @maaube

  10. @maaube

  11. @maaube

  12. Chess Exhibition • Assumptions • 24 opponents • Polgár moves

    in 5 seconds • Opponents move in 55 seconds • Games average 30 move pairs • Each game runs for 30 minutes @maaube
  13. Synchronous Chess Exhibition • Judit plays one game at a

    time • Each game runs for 30 minutes • There are 24 games to play • 24 sequential games would take: 24 x 30 min = 720 min = 12 hours @maaube
  14. Asynchronous Chess Exhibition • Polgár makes a move • While

    the opponent thinks, she moves on the second game, then third, ... • A move on all 24 games takes her: 24 x 5 sec = 120 sec = 2 minutes • 24 games are completed in: 2 min x 30 = 60 min = 1 hour! @maaube
  15. Asynchronous Chess Exhibition • Polgár makes a move • While

    the opponent thinks, she moves on the second game, then third, ... • A move on all 24 games takes her: 24 x 5 sec = 120 sec = 2 minutes • 24 games are completed in: 2 min x 30 = 60 min = 1 hour! @maaube
  16. Cooperative concurrency A fancy way of saying that your async

    functions can give back control to the execution loop, so that each function can run at the optimal time. @maaube
  17. Understanding computer time @maaube

  18. async and await @maaube

  19. @maaube

  20. @maaube

  21. @maaube

  22. @maaube

  23. async_hello.py executed in 1.01 seconds. @maaube

  24. Made-up problem time! @maaube

  25. Goals 1. Crawl Confoo's Flickr account 2. Download all pictures

    from photo stream 3. Make a 120x120 thumbnail for each of them 4. Convert it to black & white 5. Be fast! ! @maaube
  26. @maaube

  27. @maaube

  28. @maaube

  29. @maaube

  30. @maaube

  31. @maaube

  32. @maaube

  33. This function is blocking... @maaube

  34. @maaube

  35. @maaube

  36. @maaube

  37. @maaube

  38. Goals 1. Crawl Confoo's Flickr account 2. Download all pictures

    from photo stream ✅ 3. Make a 120x120 thumbnail for each of them ✅ 4. Convert it to black & white 5. Be fast! ❓ @maaube
  39. @maaube

  40. Going even faster! @maaube

  41. @maaube

  42. GitHub repo @maaube

  43. Pitfalls •long CPU-intensive tasks • execute in a background thread

    • routinely release CPU with await asyncio.sleep(0) •blocking library functions @maaube
  44. Resources @maaube

  45. @maaube

  46. @maaube

  47. Questions ? @maaube