Async Python. Who's there? Knock knock.

async, await, generators, tasks, coroutines, oh my! Let's learn how this often overlooked module can help you write concurrent code, without starting multiple threads or using multiple CPU cores. You'll leave with a better understanding of what cooperative multitasking is and how you can use it in your own projects.

Transcript

1. Async Python. Who's there? Knock knock.
2. Parallelism, concurrency, multi-processing, threading, oh my! ! @maaube
3. Parallelism: performing multiple operations at the same time. @maaube
4. Concurrency: performing multiple operations in an overlapping manner. @maaube
5. As David said earlier today... @maaube
6. Async I/O: a style of concurrent programming in which tasks release the CPU during idling periods, so that other tasks can use it. @maaube
7. Which model is the best? @maaube
8. It depends... 1. heavy computations or maths (CPU bound) • multi-processing 2. network, servers, HTTP, sockets (I/O bound) • multi-threading • asyncio @maaube
9. How does asyncio work? @maaube
10. @maaube
11. @maaube
12. Chess Exhibition • Assumptions • 24 opponents • Polgár moves in 5 seconds • Opponents move in 55 seconds • Games average 30 move pairs • Each game runs for 30 minutes @maaube
13. Synchronous Chess Exhibition • Judit plays one game at a time • Each game runs for 30 minutes • There are 24 games to play • 24 sequential games would take: 24 x 30 min = 720 min = 12 hours @maaube
14. Asynchronous Chess Exhibition • Polgár makes a move • While the opponent thinks, she moves on the second game, then third, ... • A move on all 24 games takes her: 24 x 5 sec = 120 sec = 2 minutes • 24 games are completed in: 2 min x 30 = 60 min = 1 hour! @maaube
15. Asynchronous Chess Exhibition • Polgár makes a move • While the opponent thinks, she moves on the second game, then third, ... • A move on all 24 games takes her: 24 x 5 sec = 120 sec = 2 minutes • 24 games are completed in: 2 min x 30 = 60 min = 1 hour! @maaube
16. Cooperative concurrency A fancy way of saying that your async functions can give back control to the execution loop, so that each function can run at the optimal time. @maaube
17. Understanding computer time @maaube
18. async and await @maaube
19. @maaube
20. @maaube
21. @maaube
22. @maaube
23. async_hello.py executed in 1.01 seconds. @maaube
24. Made-up problem time! @maaube
25. Goals 1. Crawl Confoo's Flickr account 2. Download all pictures from photo stream 3. Make a 120x120 thumbnail for each of them 4. Convert it to black & white 5. Be fast! ! @maaube
26. @maaube
27. @maaube
28. @maaube
29. @maaube
30. @maaube
31. @maaube
32. @maaube
33. This function is blocking... @maaube
34. @maaube
35. @maaube
36. @maaube
37. @maaube
38. Goals 1. Crawl Confoo's Flickr account 2. Download all pictures from photo stream ✅ 3. Make a 120x120 thumbnail for each of them ✅ 4. Convert it to black & white 5. Be fast! ❓ @maaube
39. @maaube
40. Going even faster! @maaube
41. @maaube
42. GitHub repo @maaube
43. Pitfalls •long CPU-intensive tasks • execute in a background thread • routinely release CPU with await asyncio.sleep(0) •blocking library functions @maaube
44. Resources @maaube
45. @maaube
46. @maaube
47. Questions ? @maaube