Async Python. Who's there? Knock knock.

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Parallelism, concurrency, multi-processing, threading, oh my! ! @maaube

Slide 3

Slide 3 text

Parallelism: performing multiple operations at the same time. @maaube

Slide 4

Slide 4 text

Concurrency: performing multiple operations in an overlapping manner. @maaube

Slide 5

Slide 5 text

As David said earlier today... @maaube

Slide 6

Slide 6 text

Async I/O: a style of concurrent programming in which tasks release the CPU during idling periods, so that other tasks can use it. @maaube

Slide 7

Slide 7 text

Which model is the best? @maaube

Slide 8

Slide 8 text

It depends... 1. heavy computations or maths (CPU bound) • multi-processing 2. network, servers, HTTP, sockets (I/O bound) • multi-threading • asyncio @maaube

Slide 9

Slide 9 text

How does asyncio work? @maaube

Slide 10

Slide 10 text

@maaube

Slide 11

Slide 11 text

@maaube

Slide 12

Slide 12 text

Chess Exhibition • Assumptions • 24 opponents • Polgár moves in 5 seconds • Opponents move in 55 seconds • Games average 30 move pairs • Each game runs for 30 minutes @maaube

Slide 13

Slide 13 text

Synchronous Chess Exhibition • Judit plays one game at a time • Each game runs for 30 minutes • There are 24 games to play • 24 sequential games would take: 24 x 30 min = 720 min = 12 hours @maaube

Slide 14

Slide 14 text

Asynchronous Chess Exhibition • Polgár makes a move • While the opponent thinks, she moves on the second game, then third, ... • A move on all 24 games takes her: 24 x 5 sec = 120 sec = 2 minutes • 24 games are completed in: 2 min x 30 = 60 min = 1 hour! @maaube

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Cooperative concurrency A fancy way of saying that your async functions can give back control to the execution loop, so that each function can run at the optimal time. @maaube

Slide 17

Slide 17 text

Understanding computer time @maaube

Slide 18

Slide 18 text

async and await @maaube

Slide 19

Slide 19 text

@maaube

Slide 20

Slide 20 text

@maaube

Slide 21

Slide 21 text

@maaube

Slide 22

Slide 22 text

@maaube

Slide 23

Slide 23 text

async_hello.py executed in 1.01 seconds. @maaube

Slide 24

Slide 24 text

Made-up problem time! @maaube

Slide 25

Slide 25 text

Goals 1. Crawl Confoo's Flickr account 2. Download all pictures from photo stream 3. Make a 120x120 thumbnail for each of them 4. Convert it to black & white 5. Be fast! ! @maaube

Slide 26

Slide 26 text

@maaube

Slide 27

Slide 27 text

@maaube

Slide 28

Slide 28 text

@maaube

Slide 29

Slide 29 text

@maaube

Slide 30

Slide 30 text

@maaube

Slide 31

Slide 31 text

@maaube

Slide 32

Slide 32 text

@maaube

Slide 33

Slide 33 text

This function is blocking... @maaube

Slide 34

Slide 34 text

@maaube

Slide 35

Slide 35 text

@maaube

Slide 36

Slide 36 text

@maaube

Slide 37

Slide 37 text

@maaube

Slide 38

Slide 38 text

Goals 1. Crawl Confoo's Flickr account 2. Download all pictures from photo stream ✅ 3. Make a 120x120 thumbnail for each of them ✅ 4. Convert it to black & white 5. Be fast! ❓ @maaube

Slide 39

Slide 39 text

@maaube

Slide 40

Slide 40 text

Going even faster! @maaube

Slide 41

Slide 41 text

@maaube

Slide 42

Slide 42 text

GitHub repo @maaube

Slide 43

Slide 43 text

Pitfalls •long CPU-intensive tasks • execute in a background thread • routinely release CPU with await asyncio.sleep(0) •blocking library functions @maaube