Topics of Interest (Async)

Invited Keynote. PyCon Brasil 2015. Talk includes live-coding which is best viewed on video. Conference video at https://www.youtube.com/watch?v=kkt_BtR9Kzk. Screencast at https://www.youtube.com/watch?v=lYe8W04ERnY

Transcript

1. Topics of Interest
   David Beazley (@dabeaz)
   November 10, 2015
   PyCon Brasil
   

   View Slide

2. Bem-vindo!
   I'm honored to be here from Chicago
   

   View Slide

3. Where we also
   love our BBQ
   ... even in -25C
   

   View Slide

4. Overheard...
   "asyncio is the future of Python."
   

   View Slide

5. Background
   If you need to write a program to manage 10000
   concurrent network connections, then asyncio is
   your problem.
   

   View Slide

6. Let's take a look...
   

   View Slide

7. Async/Await in C#
   public static async void CopyToAsync(Stream source,
   Stream destination)
   {
   byte[] buffer = new byte[0x1000];
   int numRead;
   while((numRead = await source.ReadAsync(buffer, 0,
   buffer.Length)) > 0)
   {
   await destination.WriteAsync(buffer, 0, numRead);
   }
   }
   Source: http://blogs.msdn.com/b/dotnet/archive/2012/04/03/
   async-in-4-5-worth-the-await.aspx
   

   View Slide

8. Javascript (ES7)
   async function chainAnimationsAsync(elem, animations)
   {
   let ret = null;
   try {
   for(const anim of animations) {
   ret = await anim(elem);
   }
   } catch(e) { /* ignore and keep going */ }
   return ret;
   }
   Source: http://tc39.github.io/ecmascript-asyncawait/
   

   View Slide

9. View Slide

10. Dart
    import "dart:html";
    main() async {
    var context = querySelector("canvas").context2D;
    var running = true; // Set false to stop game.
    while (running) {
    var time = await window.animationFrame;
    context.clearRect(0, 0, 500, 500);
    context.fillRect(time % 450, 20, 50, 50);
    }
    }
    Source: https://www.dartlang.org/articles/await-async/
    

    View Slide

11. Scala
    val futureDOY: Future[Response] =
    WS.url("http://api.day-of-year/today").get
    val futureDaysLeft: Future[Response] =
    WS.url("http://api.days-left/today").get
    val respFut = async {
    val dayOfYear = await(futureDOY).body
    val daysLeft = await(futureDaysLeft).body
    Ok(s"$dayOfYear: $daysLeft days left!")
    }
    Source: http://docs.scala-lang.org/sips/pending/async.html
    

    View Slide

12. Fortran
    ASYN FUNCTION NGCD(NA, NB)
    IA = NA
    IB = NB
    1 IF (IB.NE.0) THEN
    ITEMP = IA
    IA = IB
    IB = AWAI MOD(ITEMP, IB)
    GOTO 1
    END IF
    NGCD = IA
    RETURN
    END
    

    View Slide

13. Fortran
    ASYN FUNCTION NGCD(NA, NB)
    IA = NA
    IB = NB
    1 IF (IB.NE.0) THEN
    ITEMP = IA
    IA = IB
    IB = AWAI MOD(ITEMP, IB)
    GOTO 1
    END IF
    NGCD = IA
    RETURN
    END
    (actually I'm not so sure about Fortran)
    

    View Slide

14. Python
    

    View Slide

15. async/await is Fun!
    

    View Slide

16. asyncio is the future
    "Async/await is amazing, the
    mecca of working with
    asynchronous code in
    Javascript" - Thomas Hunter
    

    View Slide

17. a quiet moment...
    

    View Slide

18. Welcome to your Futures!!!
    

    View Slide

19. Documentation
    

    View Slide

20. Debugging
    

    View Slide

21. Reading the Source
    

    View Slide

22. Reading the Source
    class Future:
    ...
    def __iter__(self):
    if not self.done():
    yield self
    return self.result()
    

    View Slide

23. Users
    "If we don't fix this, we will all go insane."
    

    View Slide

24. asyncio: it fits your head
    

    View Slide

25. asyncio: it fits your head
    (Perfectly)
    

    View Slide

26. It Fits Not In Mine...
    

    View Slide

27. View Slide

28. Help Me!!!
    • I get the big picture (concurrency). But...
    • Is it a replacement for Tornado/Twisted?
    • Is it a layer that everyone should use?
    • Is it an reference implementation of an API?
    • Also: How do I teach this?
    

    View Slide

29. Some History
    Threads
    Polling
    def handle_echo(client):
    while True:
    data = client.recv(10000)
    if not data:
    break
    client.sendall(data)
    while True:
    r, w, _ = select(readers, writers, []
    for s in r:
    handle_read(s)
    for s in w:
    handle_write(s)
    

    View Slide

30. Some History
    Threads
    Polling
    Callbacks
    

    View Slide

31. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    

    View Slide

32. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    "Green
    threads?"
    stackless
    

    View Slide

33. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    "Green
    threads?"
    stackless
    

    View Slide

34. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    "Green
    threads?"
    stackless
    

    View Slide

35. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    Coroutines
    "Green
    threads?"
    stackless
    

    View Slide

36. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    Coroutines
    yield from
    "Green
    threads?"
    stackless
    

    View Slide

37. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    Coroutines
    yield from
    asyncio
    "Green
    threads?"
    stackless
    

    View Slide

38. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    Coroutines
    yield from
    asyncio
    async/await
    "Green
    threads?"
    stackless
    

    View Slide

39. Defending asyncio
    • Think about everything going on here
    • It is a complicated problem
    • It is an important problem
    • But, it has a complicated history
    • Each part requires deep study
    • A lot of smart people have worked on it
    

    View Slide

40. THOUGHT
    I DO NOT WANT TO THINK ABOUT
    CALLBACKS, FUTURES, TASKS, COROUTINES,
    TRANSPORTS, EVENT LOOPS, ETC.
    

    View Slide

41. Nor does anyone else
    

    View Slide

42. Not THAT
    THIS
    

    View Slide

43. Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    Coroutines
    yield from
    asyncio
    async/await
    Is There A Shortcut?
    

    View Slide

44. Is There A Shortcut?
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    async/await
    

    View Slide

45. async/await
    Maybe THIS is the
    async future
    

    View Slide

46. An Idea
    Maybe Python's async should be an interface
    not an implementation
    not a library
    not asyncio
    

    View Slide

47. Let's Play
    

    View Slide

48. Questions
    • Can an async API exist independently of an
    async implementation?
    • Maybe
    • It seems that it should be possible
    • Especially if you embrace async/await
    

    View Slide

49. Historical Precedent
    • Python has standardized APIs before
    • WSGI
    • Database API
    

    View Slide

50. It's Interesting
    • The design of such an API is interesting
    • What operations require "await"?
    • Consistency in the interface
    • Interconnections between parts
    • Can you isolate it from the runtime?
    

    View Slide

51. It Will Be Better
    • APIs encourage experimentation
    • And competition
    • And better interoperability
    • APIs are better than layers
    

    View Slide

52. asyncio: Let It Go!
    

    View Slide

53. That is All
    • Special thanks:
    • Yarko Tymciurak
    • PyCon Brasil Organizers
    • Twitter: @dabeaz
    • Questions!
    

    View Slide