Topics of Interest (Async)

Topics of Interest (Async)

Invited Keynote. PyCon Brasil 2015. Talk includes live-coding which is best viewed on video. Conference video at https://www.youtube.com/watch?v=kkt_BtR9Kzk. Screencast at https://www.youtube.com/watch?v=lYe8W04ERnY

70c42f4cf225f1455a7e01379bbd4d48?s=128

David Beazley

November 10, 2015
Tweet

Transcript

  1. Topics of Interest David Beazley (@dabeaz) November 10, 2015 PyCon

    Brasil
  2. Bem-vindo! I'm honored to be here from Chicago

  3. Where we also love our BBQ ... even in -25C

  4. Overheard... "asyncio is the future of Python."

  5. Background If you need to write a program to manage

    10000 concurrent network connections, then asyncio is your problem.
  6. Let's take a look...

  7. Async/Await in C# public static async void CopyToAsync(Stream source, Stream

    destination) { byte[] buffer = new byte[0x1000]; int numRead; while((numRead = await source.ReadAsync(buffer, 0, buffer.Length)) > 0) { await destination.WriteAsync(buffer, 0, numRead); } } Source: http://blogs.msdn.com/b/dotnet/archive/2012/04/03/ async-in-4-5-worth-the-await.aspx
  8. Javascript (ES7) async function chainAnimationsAsync(elem, animations) { let ret =

    null; try { for(const anim of animations) { ret = await anim(elem); } } catch(e) { /* ignore and keep going */ } return ret; } Source: http://tc39.github.io/ecmascript-asyncawait/
  9. None
  10. Dart import "dart:html"; main() async { var context = querySelector("canvas").context2D;

    var running = true; // Set false to stop game. while (running) { var time = await window.animationFrame; context.clearRect(0, 0, 500, 500); context.fillRect(time % 450, 20, 50, 50); } } Source: https://www.dartlang.org/articles/await-async/
  11. Scala val futureDOY: Future[Response] = WS.url("http://api.day-of-year/today").get val futureDaysLeft: Future[Response] =

    WS.url("http://api.days-left/today").get val respFut = async { val dayOfYear = await(futureDOY).body val daysLeft = await(futureDaysLeft).body Ok(s"$dayOfYear: $daysLeft days left!") } Source: http://docs.scala-lang.org/sips/pending/async.html
  12. Fortran ASYN FUNCTION NGCD(NA, NB) IA = NA IB =

    NB 1 IF (IB.NE.0) THEN ITEMP = IA IA = IB IB = AWAI MOD(ITEMP, IB) GOTO 1 END IF NGCD = IA RETURN END
  13. Fortran ASYN FUNCTION NGCD(NA, NB) IA = NA IB =

    NB 1 IF (IB.NE.0) THEN ITEMP = IA IA = IB IB = AWAI MOD(ITEMP, IB) GOTO 1 END IF NGCD = IA RETURN END (actually I'm not so sure about Fortran)
  14. Python

  15. async/await is Fun!

  16. asyncio is the future "Async/await is amazing, the mecca of

    working with asynchronous code in Javascript" - Thomas Hunter
  17. a quiet moment...

  18. Welcome to your Futures!!!

  19. Documentation

  20. Debugging

  21. Reading the Source

  22. Reading the Source class Future: ... def __iter__(self): if not

    self.done(): yield self return self.result()
  23. Users "If we don't fix this, we will all go

    insane."
  24. asyncio: it fits your head

  25. asyncio: it fits your head (Perfectly)

  26. It Fits Not In Mine...

  27. None
  28. Help Me!!! • I get the big picture (concurrency). But...

    • Is it a replacement for Tornado/Twisted? • Is it a layer that everyone should use? • Is it an reference implementation of an API? • Also: How do I teach this?
  29. Some History Threads Polling def handle_echo(client): while True: data =

    client.recv(10000) if not data: break client.sendall(data) while True: r, w, _ = select(readers, writers, [] for s in r: handle_read(s) for s in w: handle_write(s)
  30. Some History Threads Polling Callbacks

  31. Some History Threads Polling Callbacks Futures, Deferreds

  32. Some History Threads Polling Callbacks Futures, Deferreds "Green threads?" stackless

  33. Some History Threads Polling Callbacks Futures, Deferreds Generators "Green threads?"

    stackless
  34. Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks

    "Green threads?" stackless
  35. Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks

    Coroutines "Green threads?" stackless
  36. Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks

    Coroutines yield from "Green threads?" stackless
  37. Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks

    Coroutines yield from asyncio "Green threads?" stackless
  38. Some History Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks

    Coroutines yield from asyncio async/await "Green threads?" stackless
  39. Defending asyncio • Think about everything going on here •

    It is a complicated problem • It is an important problem • But, it has a complicated history • Each part requires deep study • A lot of smart people have worked on it
  40. THOUGHT I DO NOT WANT TO THINK ABOUT CALLBACKS, FUTURES,

    TASKS, COROUTINES, TRANSPORTS, EVENT LOOPS, ETC.
  41. Nor does anyone else

  42. Not THAT THIS

  43. Threads Polling Callbacks Futures, Deferreds Generators Inlined Callbacks Coroutines yield

    from asyncio async/await Is There A Shortcut?
  44. Is There A Shortcut? Threads Polling Callbacks Futures, Deferreds Generators

    async/await
  45. async/await Maybe THIS is the async future

  46. An Idea Maybe Python's async should be an interface not

    an implementation not a library not asyncio
  47. Let's Play

  48. Questions • Can an async API exist independently of an

    async implementation? • Maybe • It seems that it should be possible • Especially if you embrace async/await
  49. Historical Precedent • Python has standardized APIs before • WSGI

    • Database API
  50. It's Interesting • The design of such an API is

    interesting • What operations require "await"? • Consistency in the interface • Interconnections between parts • Can you isolate it from the runtime?
  51. It Will Be Better • APIs encourage experimentation • And

    competition • And better interoperability • APIs are better than layers
  52. asyncio: Let It Go!

  53. That is All • Special thanks: • Yarko Tymciurak •

    PyCon Brasil Organizers • Twitter: @dabeaz • Questions!