Topics of Interest (Async)

Slide 1

Slide 1 text

Topics of Interest David Beazley (@dabeaz) November 10, 2015 PyCon Brasil

Slide 2

Slide 2 text

Bem-vindo! I'm honored to be here from Chicago

Slide 3

Slide 3 text

Where we also love our BBQ ... even in -25C

Slide 4

Slide 4 text

Overheard... "asyncio is the future of Python."

Slide 5

Slide 5 text

Background If you need to write a program to manage 10000 concurrent network connections, then asyncio is your problem.

Slide 6

Slide 6 text

Let's take a look...

Slide 7

Slide 7 text

Async/Await in C# public static async void CopyToAsync(Stream source, Stream destination) { byte[] buffer = new byte[0x1000]; int numRead; while((numRead = await source.ReadAsync(buffer, 0, buffer.Length)) > 0) { await destination.WriteAsync(buffer, 0, numRead); } } Source: http://blogs.msdn.com/b/dotnet/archive/2012/04/03/ async-in-4-5-worth-the-await.aspx

Slide 8

Slide 8 text

Javascript (ES7) async function chainAnimationsAsync(elem, animations) { let ret = null; try { for(const anim of animations) { ret = await anim(elem); } } catch(e) { /* ignore and keep going */ } return ret; } Source: http://tc39.github.io/ecmascript-asyncawait/

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

Dart import "dart:html"; main() async { var context = querySelector("canvas").context2D; var running = true; // Set false to stop game. while (running) { var time = await window.animationFrame; context.clearRect(0, 0, 500, 500); context.fillRect(time % 450, 20, 50, 50); } } Source: https://www.dartlang.org/articles/await-async/

Slide 11

Slide 11 text

Scala val futureDOY: Future[Response] = WS.url("http://api.day-of-year/today").get val futureDaysLeft: Future[Response] = WS.url("http://api.days-left/today").get val respFut = async { val dayOfYear = await(futureDOY).body val daysLeft = await(futureDaysLeft).body Ok(s"$dayOfYear: $daysLeft days left!") } Source: http://docs.scala-lang.org/sips/pending/async.html

Slide 12

Slide 12 text

Fortran ASYN FUNCTION NGCD(NA, NB) IA = NA IB = NB 1 IF (IB.NE.0) THEN ITEMP = IA IA = IB IB = AWAI MOD(ITEMP, IB) GOTO 1 END IF NGCD = IA RETURN END

Slide 13

Slide 13 text

Fortran ASYN FUNCTION NGCD(NA, NB) IA = NA IB = NB 1 IF (IB.NE.0) THEN ITEMP = IA IA = IB IB = AWAI MOD(ITEMP, IB) GOTO 1 END IF NGCD = IA RETURN END (actually I'm not so sure about Fortran)

Slide 14

Slide 14 text

Python

Slide 15

Slide 15 text

async/await is Fun!

Slide 16

Slide 16 text

asyncio is the future "Async/await is amazing, the mecca of working with asynchronous code in Javascript" - Thomas Hunter

Slide 17

Slide 17 text

a quiet moment...

Slide 18

Slide 18 text

Welcome to your Futures!!!

Slide 19

Slide 19 text

Documentation

Slide 20

Slide 20 text

Debugging

Slide 21

Slide 21 text

Reading the Source

Slide 22

Slide 22 text

Reading the Source class Future: ... def __iter__(self): if not self.done(): yield self return self.result()

Slide 23

Slide 23 text

Users "If we don't ﬁx this, we will all go insane."

Slide 24

Slide 24 text

asyncio: it ﬁts your head

Slide 25

Slide 25 text

asyncio: it ﬁts your head (Perfectly)

Slide 26

Slide 26 text

It Fits Not In Mine...

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

Help Me!!! • I get the big picture (concurrency). But... • Is it a replacement for Tornado/Twisted? • Is it a layer that everyone should use? • Is it an reference implementation of an API? • Also: How do I teach this?

Slide 29

Slide 29 text

Some History Threads Polling def handle_echo(client): while True: data = client.recv(10000) if not data: break client.sendall(data) while True: r, w, _ = select(readers, writers, [] for s in r: handle_read(s) for s in w: handle_write(s)

Slide 30

Slide 30 text

Some History Threads Polling Callbacks

Slide 31

Slide 31 text

Some History Threads Polling Callbacks Futures, Deferreds

Slide 32

Slide 32 text

Some History Threads Polling Callbacks Futures, Deferreds "Green threads?" stackless

Slide 33

Slide 33 text

Some History Threads Polling Callbacks Futures, Deferreds Generators "Green threads?" stackless

Slide 34

Slide 34 text

Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks "Green threads?" stackless

Slide 35

Slide 35 text

Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks Coroutines "Green threads?" stackless

Slide 36

Slide 36 text

Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks Coroutines yield from "Green threads?" stackless

Slide 37

Slide 37 text

Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks Coroutines yield from asyncio "Green threads?" stackless

Slide 38

Slide 38 text

Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks Coroutines yield from asyncio async/await "Green threads?" stackless

Slide 39

Slide 39 text

Defending asyncio • Think about everything going on here • It is a complicated problem • It is an important problem • But, it has a complicated history • Each part requires deep study • A lot of smart people have worked on it

Slide 40

Slide 40 text

THOUGHT I DO NOT WANT TO THINK ABOUT CALLBACKS, FUTURES, TASKS, COROUTINES, TRANSPORTS, EVENT LOOPS, ETC.

Slide 41

Slide 41 text

Nor does anyone else

Slide 42

Slide 42 text

Not THAT THIS

Slide 43

Slide 43 text

Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks Coroutines yield from asyncio async/await Is There A Shortcut?

Slide 44

Slide 44 text

Is There A Shortcut? Threads Polling Callbacks Futures, Deferreds Generators async/await

Slide 45

Slide 45 text

async/await Maybe THIS is the async future

Slide 46

Slide 46 text

An Idea Maybe Python's async should be an interface not an implementation not a library not asyncio

Slide 47

Slide 47 text

Let's Play

Slide 48

Slide 48 text

Questions • Can an async API exist independently of an async implementation? • Maybe • It seems that it should be possible • Especially if you embrace async/await

Slide 49

Slide 49 text

Historical Precedent • Python has standardized APIs before • WSGI • Database API

Slide 50

Slide 50 text

It's Interesting • The design of such an API is interesting • What operations require "await"? • Consistency in the interface • Interconnections between parts • Can you isolate it from the runtime?

Slide 51

Slide 51 text

It Will Be Better • APIs encourage experimentation • And competition • And better interoperability • APIs are better than layers

Slide 52

Slide 52 text

asyncio: Let It Go!

Slide 53

Slide 53 text

That is All • Special thanks: • Yarko Tymciurak • PyCon Brasil Organizers • Twitter: @dabeaz • Questions!