$30 off During Our Annual Pro Sale. View Details »

Topics of Interest (Async)

Topics of Interest (Async)

Invited Keynote. PyCon Brasil 2015. Talk includes live-coding which is best viewed on video. Conference video at https://www.youtube.com/watch?v=kkt_BtR9Kzk. Screencast at https://www.youtube.com/watch?v=lYe8W04ERnY

David Beazley

November 10, 2015
Tweet

More Decks by David Beazley

Other Decks in Programming

Transcript

  1. Topics of Interest
    David Beazley (@dabeaz)
    November 10, 2015
    PyCon Brasil

    View Slide

  2. Bem-vindo!
    I'm honored to be here from Chicago

    View Slide

  3. Where we also
    love our BBQ
    ... even in -25C

    View Slide

  4. Overheard...
    "asyncio is the future of Python."

    View Slide

  5. Background
    If you need to write a program to manage 10000
    concurrent network connections, then asyncio is
    your problem.

    View Slide

  6. Let's take a look...

    View Slide

  7. Async/Await in C#
    public static async void CopyToAsync(Stream source,
    Stream destination)
    {
    byte[] buffer = new byte[0x1000];
    int numRead;
    while((numRead = await source.ReadAsync(buffer, 0,
    buffer.Length)) > 0)
    {
    await destination.WriteAsync(buffer, 0, numRead);
    }
    }
    Source: http://blogs.msdn.com/b/dotnet/archive/2012/04/03/
    async-in-4-5-worth-the-await.aspx

    View Slide

  8. Javascript (ES7)
    async function chainAnimationsAsync(elem, animations)
    {
    let ret = null;
    try {
    for(const anim of animations) {
    ret = await anim(elem);
    }
    } catch(e) { /* ignore and keep going */ }
    return ret;
    }
    Source: http://tc39.github.io/ecmascript-asyncawait/

    View Slide

  9. View Slide

  10. Dart
    import "dart:html";
    main() async {
    var context = querySelector("canvas").context2D;
    var running = true; // Set false to stop game.
    while (running) {
    var time = await window.animationFrame;
    context.clearRect(0, 0, 500, 500);
    context.fillRect(time % 450, 20, 50, 50);
    }
    }
    Source: https://www.dartlang.org/articles/await-async/

    View Slide

  11. Scala
    val futureDOY: Future[Response] =
    WS.url("http://api.day-of-year/today").get
    val futureDaysLeft: Future[Response] =
    WS.url("http://api.days-left/today").get
    val respFut = async {
    val dayOfYear = await(futureDOY).body
    val daysLeft = await(futureDaysLeft).body
    Ok(s"$dayOfYear: $daysLeft days left!")
    }
    Source: http://docs.scala-lang.org/sips/pending/async.html

    View Slide

  12. Fortran
    ASYN FUNCTION NGCD(NA, NB)
    IA = NA
    IB = NB
    1 IF (IB.NE.0) THEN
    ITEMP = IA
    IA = IB
    IB = AWAI MOD(ITEMP, IB)
    GOTO 1
    END IF
    NGCD = IA
    RETURN
    END

    View Slide

  13. Fortran
    ASYN FUNCTION NGCD(NA, NB)
    IA = NA
    IB = NB
    1 IF (IB.NE.0) THEN
    ITEMP = IA
    IA = IB
    IB = AWAI MOD(ITEMP, IB)
    GOTO 1
    END IF
    NGCD = IA
    RETURN
    END
    (actually I'm not so sure about Fortran)

    View Slide

  14. Python

    View Slide

  15. async/await is Fun!

    View Slide

  16. asyncio is the future
    "Async/await is amazing, the
    mecca of working with
    asynchronous code in
    Javascript" - Thomas Hunter

    View Slide

  17. a quiet moment...

    View Slide

  18. Welcome to your Futures!!!

    View Slide

  19. Documentation

    View Slide

  20. Debugging

    View Slide

  21. Reading the Source

    View Slide

  22. Reading the Source
    class Future:
    ...
    def __iter__(self):
    if not self.done():
    yield self
    return self.result()

    View Slide

  23. Users
    "If we don't fix this, we will all go insane."

    View Slide

  24. asyncio: it fits your head

    View Slide

  25. asyncio: it fits your head
    (Perfectly)

    View Slide

  26. It Fits Not In Mine...

    View Slide

  27. View Slide

  28. Help Me!!!
    • I get the big picture (concurrency). But...
    • Is it a replacement for Tornado/Twisted?
    • Is it a layer that everyone should use?
    • Is it an reference implementation of an API?
    • Also: How do I teach this?

    View Slide

  29. Some History
    Threads
    Polling
    def handle_echo(client):
    while True:
    data = client.recv(10000)
    if not data:
    break
    client.sendall(data)
    while True:
    r, w, _ = select(readers, writers, []
    for s in r:
    handle_read(s)
    for s in w:
    handle_write(s)

    View Slide

  30. Some History
    Threads
    Polling
    Callbacks

    View Slide

  31. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds

    View Slide

  32. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    "Green
    threads?"
    stackless

    View Slide

  33. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    "Green
    threads?"
    stackless

    View Slide

  34. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    "Green
    threads?"
    stackless

    View Slide

  35. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    Coroutines
    "Green
    threads?"
    stackless

    View Slide

  36. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    Coroutines
    yield from
    "Green
    threads?"
    stackless

    View Slide

  37. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    Coroutines
    yield from
    asyncio
    "Green
    threads?"
    stackless

    View Slide

  38. Some History
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    Coroutines
    yield from
    asyncio
    async/await
    "Green
    threads?"
    stackless

    View Slide

  39. Defending asyncio
    • Think about everything going on here
    • It is a complicated problem
    • It is an important problem
    • But, it has a complicated history
    • Each part requires deep study
    • A lot of smart people have worked on it

    View Slide

  40. THOUGHT
    I DO NOT WANT TO THINK ABOUT
    CALLBACKS, FUTURES, TASKS, COROUTINES,
    TRANSPORTS, EVENT LOOPS, ETC.

    View Slide

  41. Nor does anyone else

    View Slide

  42. Not THAT
    THIS

    View Slide

  43. Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    Inlined Callbacks
    Coroutines
    yield from
    asyncio
    async/await
    Is There A Shortcut?

    View Slide

  44. Is There A Shortcut?
    Threads
    Polling
    Callbacks Futures,
    Deferreds
    Generators
    async/await

    View Slide

  45. async/await
    Maybe THIS is the
    async future

    View Slide

  46. An Idea
    Maybe Python's async should be an interface
    not an implementation
    not a library
    not asyncio

    View Slide

  47. Let's Play

    View Slide

  48. Questions
    • Can an async API exist independently of an
    async implementation?
    • Maybe
    • It seems that it should be possible
    • Especially if you embrace async/await

    View Slide

  49. Historical Precedent
    • Python has standardized APIs before
    • WSGI
    • Database API

    View Slide

  50. It's Interesting
    • The design of such an API is interesting
    • What operations require "await"?
    • Consistency in the interface
    • Interconnections between parts
    • Can you isolate it from the runtime?

    View Slide

  51. It Will Be Better
    • APIs encourage experimentation
    • And competition
    • And better interoperability
    • APIs are better than layers

    View Slide

  52. asyncio: Let It Go!

    View Slide

  53. That is All
    • Special thanks:
    • Yarko Tymciurak
    • PyCon Brasil Organizers
    • Twitter: @dabeaz
    • Questions!

    View Slide