Topics of Interest (Async)

Invited Keynote. PyCon Brasil 2015. Talk includes live-coding which is best viewed on video. Conference video at https://www.youtube.com/watch?v=kkt_BtR9Kzk. Screencast at https://www.youtube.com/watch?v=lYe8W04ERnY

Transcript

1. Topics of Interest David Beazley (@dabeaz) November 10, 2015 PyCon Brasil
2. Bem-vindo! I'm honored to be here from Chicago
3. Where we also love our BBQ ... even in -25C
4. Overheard... "asyncio is the future of Python."
5. Background If you need to write a program to manage 10000 concurrent network connections, then asyncio is your problem.
6. Let's take a look...
7. Async/Await in C# public static async void CopyToAsync(Stream source, Stream destination) { byte[] buffer = new byte[0x1000]; int numRead; while((numRead = await source.ReadAsync(buffer, 0, buffer.Length)) > 0) { await destination.WriteAsync(buffer, 0, numRead); } } Source: http://blogs.msdn.com/b/dotnet/archive/2012/04/03/ async-in-4-5-worth-the-await.aspx
8. Javascript (ES7) async function chainAnimationsAsync(elem, animations) { let ret = null; try { for(const anim of animations) { ret = await anim(elem); } } catch(e) { /* ignore and keep going */ } return ret; } Source: http://tc39.github.io/ecmascript-asyncawait/
10. Dart import "dart:html"; main() async { var context = querySelector("canvas").context2D; var running = true; // Set false to stop game. while (running) { var time = await window.animationFrame; context.clearRect(0, 0, 500, 500); context.fillRect(time % 450, 20, 50, 50); } } Source: https://www.dartlang.org/articles/await-async/
11. Scala val futureDOY: Future[Response] = WS.url("http://api.day-of-year/today").get val futureDaysLeft: Future[Response] = WS.url("http://api.days-left/today").get val respFut = async { val dayOfYear = await(futureDOY).body val daysLeft = await(futureDaysLeft).body Ok(s"$dayOfYear: $daysLeft days left!") } Source: http://docs.scala-lang.org/sips/pending/async.html
12. Fortran ASYN FUNCTION NGCD(NA, NB) IA = NA IB = NB 1 IF (IB.NE.0) THEN ITEMP = IA IA = IB IB = AWAI MOD(ITEMP, IB) GOTO 1 END IF NGCD = IA RETURN END
13. Fortran ASYN FUNCTION NGCD(NA, NB) IA = NA IB = NB 1 IF (IB.NE.0) THEN ITEMP = IA IA = IB IB = AWAI MOD(ITEMP, IB) GOTO 1 END IF NGCD = IA RETURN END (actually I'm not so sure about Fortran)
14. Python
15. async/await is Fun!
16. asyncio is the future "Async/await is amazing, the mecca of working with asynchronous code in Javascript" - Thomas Hunter
17. a quiet moment...
18. Welcome to your Futures!!!
19. Documentation
20. Debugging
21. Reading the Source
22. Reading the Source class Future: ... def __iter__(self): if not self.done(): yield self return self.result()
23. Users "If we don't fix this, we will all go insane."
24. asyncio: it fits your head
25. asyncio: it fits your head (Perfectly)
26. It Fits Not In Mine...
28. Help Me!!! • I get the big picture (concurrency). But... • Is it a replacement for Tornado/Twisted? • Is it a layer that everyone should use? • Is it an reference implementation of an API? • Also: How do I teach this?
29. Some History Threads Polling def handle_echo(client): while True: data = client.recv(10000) if not data: break client.sendall(data) while True: r, w, _ = select(readers, writers, [] for s in r: handle_read(s) for s in w: handle_write(s)
30. Some History Threads Polling Callbacks
31. Some History Threads Polling Callbacks Futures, Deferreds
32. Some History Threads Polling Callbacks Futures, Deferreds "Green threads?" stackless
33. Some History Threads Polling Callbacks Futures, Deferreds Generators "Green threads?" stackless
34. Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks "Green threads?" stackless
35. Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks Coroutines "Green threads?" stackless
36. Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks Coroutines yield from "Green threads?" stackless
37. Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks Coroutines yield from asyncio "Green threads?" stackless
38. Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks Coroutines yield from asyncio async/await "Green threads?" stackless
39. Defending asyncio • Think about everything going on here • It is a complicated problem • It is an important problem • But, it has a complicated history • Each part requires deep study • A lot of smart people have worked on it
40. THOUGHT I DO NOT WANT TO THINK ABOUT CALLBACKS, FUTURES, TASKS, COROUTINES, TRANSPORTS, EVENT LOOPS, ETC.
41. Nor does anyone else
42. Not THAT THIS
43. Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks Coroutines yield from asyncio async/await Is There A Shortcut?
44. Is There A Shortcut? Threads Polling Callbacks Futures, Deferreds Generators async/await
45. async/await Maybe THIS is the async future
46. An Idea Maybe Python's async should be an interface not an implementation not a library not asyncio
47. Let's Play
48. Questions • Can an async API exist independently of an async implementation? • Maybe • It seems that it should be possible • Especially if you embrace async/await
49. Historical Precedent • Python has standardized APIs before • WSGI • Database API
50. It's Interesting • The design of such an API is interesting • What operations require "await"? • Consistency in the interface • Interconnections between parts • Can you isolate it from the runtime?
51. It Will Be Better • APIs encourage experimentation • And competition • And better interoperability • APIs are better than layers
52. asyncio: Let It Go!
53. That is All • Special thanks: • Yarko Tymciurak • PyCon Brasil Organizers • Twitter: @dabeaz • Questions!