[PyCon KR 2017] aiohttp in Production

PyCon Korea 2017 [aiohttp in Production](https://www.pycon.kr/2017/program/133)

Transcript

1. aiohttp in Production
   Sunghyun Hwang
   (2017-08-13)
   

   View Slide

2. Disclaimer
   • Requires Python 3.3+
   • Assumes general awareness of
   • Generators/coroutines
   • AsyncIO concepts
   • Async/await syntax
   • Common programming patterns
   

   View Slide

3. View Slide

4. View Slide

5. aiohttp: Asynchronous
   HTTP Client/Server
   

   View Slide

6. aiohttp: Asynchronous
   HTTP Client/Server
   

   View Slide

7. How aiohttp works
   

   View Slide

8. PEP 3156 — Asynchronous IO Support
   Rebooted: the "asyncio" Module
   Guido van Rossum (2012-12-12)
   

   View Slide

9. This module provides infrastructure for
   writing single-threaded concurrent code
   using coroutines, multiplexing I/O
   access over sockets...
   

   View Slide

10. This module provides infrastructure for
    writing single-threaded concurrent code
    using coroutines, multiplexing I/O
    access over sockets...
    

    View Slide

11. This module provides infrastructure for
    writing single-threaded concurrent code
    using coroutines, multiplexing I/O
    access over sockets...
    

    View Slide

12. This module provides infrastructure for
    writing single-threaded concurrent code
    using coroutines, multiplexing I/O
    access over sockets...
    

    View Slide

13. app = Application()
    async def hello(request):
    return Response(text='Hello, Pycon!')
    app.router.add_get('/', hello)
    run_app(app, host='0.0.0.0', port=8080)
    

    View Slide

14. ======== Running on http://0.0.0.0:8080 ========
    (Press CTRL+C to quit)
    

    View Slide

15. app = Application()
    async def hello(request):
    return Response(text='Hello, Pycon!')
    app.router.add_get('/', hello)
    run_app(app, host='0.0.0.0', port=8080)
    

    View Slide

16. class Application(MutableMapping):
    def __init__(self, *,
    logger=web_logger,
    router=None,
    middlewares=(),
    handler_args=None,
    client_max_size=1024**2,
    secure_proxy_ssl_header=None,
    loop=None,
    debug=...)
    aiohttp/web.py
    

    View Slide

17. app = Application()
    async def hello(request):
    return Response(text='Hello, Pycon!')
    app.router.add_get('/', hello)
    run_app(app, host='0.0.0.0', port=8080)
    

    View Slide

18. app = Application()
    async def hello(request):
    return Response(text='Hello, Pycon!')
    app.router.add_get('/', hello)
    run_app(app, host='0.0.0.0', port=8080)
    app = Application()
    async def hello(request):
    return Response(text='Hello, Pycon!')
    app.router.add_get('/', hello)
    run_app(app, host='0.0.0.0', port=8080)
    

    View Slide

19. aiohttp/web.py
    try:
    print("======== Running on {} ========\n"
    "(Press CTRL+C to quit)".format(', '.join(uris)))
    loop.run_forever()
    except (GracefulExit, KeyboardInterrupt):
    pass
    

    View Slide

20. aiohttp/web.py
    try:
    print("======== Running on {} ========\n"
    "(Press CTRL+C to quit)".format(', '.join(uris)))
    loop.run_forever()
    except (GracefulExit, KeyboardInterrupt):
    pass
    try:
    print("======== Running on {} ========\n"
    "(Press CTRL+C to quit)".format(', '.join(uris)))
    loop.run_forever()
    except (GracefulExit, KeyboardInterrupt):
    pass
    

    View Slide

21. aiohttp/web.py
    def run_app(app, *, host=None, port=None, ..., loop=None):
    """Run an app locally"""
    if loop is None:
    loop = asyncio.get_event_loop()
    

    View Slide

22. aiohttp/web.py
    def run_app(app, *, host=None, port=None, ..., loop=None):
    """Run an app locally"""
    if loop is None:
    loop = asyncio.get_event_loop()
    def run_app(app, *, host=None, port=None, ..., loop=None):
    """Run an app locally"""
    if loop is None:
    loop = asyncio.get_event_loop()
    

    View Slide

23. aiohttp/web.py
    def run_app(app, *, host=None, port=None, ..., loop=None):
    """Run an app locally"""
    if loop is None:
    loop = asyncio.get_event_loop()
    def run_app(app, *, host=None, port=None, ..., loop=None):
    """Run an app locally"""
    if loop is None:
    loop = asyncio.get_event_loop()
    

    View Slide

24. Event Loop
    In computer science, the event loop is a
    programming construct that waits for and
    dispatches events or messages in a program.
    — Wikipedia
    

    View Slide

25. Event Loop
    In computer science, the event loop is a
    programming construct that waits for and
    dispatches events or messages in a program.
    — Wikipedia
    

    View Slide

26. Event Loop
    In computer science, the event loop is a
    programming construct that waits for and
    dispatches events or messages in a program.
    — Wikipedia
    

    View Slide

27. Event Loop
    In computer science, the event loop is a
    programming construct that waits for and
    dispatches events or messages in a program.
    — Wikipedia
    

    View Slide

28. Event Loop
    In computer science, the event loop is a
    programming construct that waits for and
    dispatches events or messages in a program.
    — Wikipedia
    

    View Slide

29. Event Loop
    while there are still events to process:
    e = get the next event
    if there is a callback associated with e:
    call the callback
    — [An Introduction to libuv](https://nikhilm.github.io/uvbook/basics.html)
    

    View Slide

30. class _RunningLoop(threading.local):
    _loop = None
    _pid = None
    _running_loop = _RunningLoop()
    def _get_running_loop():
    running_loop = _running_loop._loop
    if running_loop is not None and _running_loop._pid == os.getpid():
    return running_loop
    asyncio/events.py
    

    View Slide

31. class _RunningLoop(threading.local):
    _loop = None
    _pid = None
    _running_loop = _RunningLoop()
    def _get_running_loop():
    running_loop = _running_loop._loop
    if running_loop is not None and _running_loop._pid == os.getpid():
    return running_loop
    asyncio/events.py
    class _RunningLoop(threading.local):
    _loop = None
    _pid = None
    _running_loop = _RunningLoop()
    def _get_running_loop():
    running_loop = _running_loop._loop
    if running_loop is not None and _running_loop._pid == os.getpid():
    return running_loop
    

    View Slide

32. class _RunningLoop(threading.local):
    _loop = None
    _pid = None
    _running_loop = _RunningLoop()
    def _get_running_loop():
    running_loop = _running_loop._loop
    if running_loop is not None and _running_loop._pid == os.getpid():
    return running_loop
    asyncio/events.py
    class _RunningLoop(threading.local):
    _loop = None
    _pid = None
    _running_loop = _RunningLoop()
    def _get_running_loop():
    running_loop = _running_loop._loop
    if running_loop is not None and _running_loop._pid == os.getpid():
    return running_loop
    

    View Slide

33. asyncio/events.py
    class AbstractEventLoop:
    """Abstract event loop."""
    # Running and stopping the event loop.
    def run_forever(self):
    """Run the event loop until stop() is called."""
    raise NotImplementedError
    def run_until_complete(self, future):
    """Run the event loop until a Future is done.
    Return the Future's result, or raise its exception.
    """
    raise NotImplementedError
    def is_running(self):
    """Return whether the event loop is currently running."""
    raise NotImplementedError
    

    View Slide

34. asyncio/events.py
    class AbstractEventLoop:
    """Abstract event loop."""
    # Running and stopping the event loop.
    def run_forever(self):
    """Run the event loop until stop() is called."""
    raise NotImplementedError
    def run_until_complete(self, future):
    """Run the event loop until a Future is done.
    Return the Future's result, or raise its exception.
    """
    raise NotImplementedError
    def is_running(self):
    """Return whether the event loop is currently running."""
    raise NotImplementedError
    class AbstractEventLoop:
    """Abstract event loop."""
    # Running and stopping the event loop.
    def run_forever(self):
    """Run the event loop until stop() is called."""
    raise NotImplementedError
    def run_until_complete(self, future):
    """Run the event loop until a Future is done.
    Return the Future's result, or raise its exception.
    """
    raise NotImplementedError
    def is_running(self):
    """Return whether the event loop is currently running."""
    raise NotImplementedError
    

    View Slide

35. asyncio/events.py
    class AbstractEventLoop:
    """Abstract event loop."""
    # Running and stopping the event loop.
    def run_forever(self):
    """Run the event loop until stop() is called."""
    raise NotImplementedError
    def run_until_complete(self, future):
    """Run the event loop until a Future is done.
    Return the Future's result, or raise its exception.
    """
    raise NotImplementedError
    def is_running(self):
    """Return whether the event loop is currently running."""
    raise NotImplementedError
    class AbstractEventLoop:
    """Abstract event loop."""
    # Running and stopping the event loop.
    def run_forever(self):
    """Run the event loop until stop() is called."""
    raise NotImplementedError
    def run_until_complete(self, future):
    """Run the event loop until a Future is done.
    Return the Future's result, or raise its exception.
    """
    raise NotImplementedError
    def is_running(self):
    """Return whether the event loop is currently running."""
    raise NotImplementedError
    

    View Slide

36. asyncio/events.py
    class AbstractEventLoop:
    """Abstract event loop."""
    # Running and stopping the event loop.
    def run_forever(self):
    """Run the event loop until stop() is called."""
    raise NotImplementedError
    def run_until_complete(self, future):
    """Run the event loop until a Future is done.
    Return the Future's result, or raise its exception.
    """
    raise NotImplementedError
    def is_running(self):
    """Return whether the event loop is currently running."""
    raise NotImplementedError
    class AbstractEventLoop:
    """Abstract event loop."""
    # Running and stopping the event loop.
    def run_forever(self):
    """Run the event loop until stop() is called."""
    raise NotImplementedError
    def run_until_complete(self, future):
    """Run the event loop until a Future is done.
    Return the Future's result, or raise its exception.
    """
    raise NotImplementedError
    def is_running(self):
    """Return whether the event loop is currently running."""
    raise NotImplementedError
    

    View Slide

37. asyncio/futures.py
    class Future:
    _state = _PENDING
    _result = None
    _exception = None
    

    View Slide

38. asyncio/futures.py
    class Future:
    _state = _FINISHED
    _result = None
    _exception = None
    

    View Slide

39. asyncio/futures.py
    class Future:
    _state = _CANCELLED
    _result = None
    _exception = None
    

    View Slide

40. asyncio/events.py
    class AbstractEventLoop:
    """Abstract event loop."""
    # Running and stopping the event loop.
    def run_forever(self):
    """Run the event loop until stop() is called."""
    raise NotImplementedError
    def run_until_complete(self, future):
    """Run the event loop until a Future is done.
    Return the Future's result, or raise its exception.
    """
    raise NotImplementedError
    def is_running(self):
    """Return whether the event loop is currently running."""
    raise NotImplementedError
    class AbstractEventLoop:
    """Abstract event loop."""
    # Running and stopping the event loop.
    def run_forever(self):
    """Run the event loop until stop() is called."""
    raise NotImplementedError
    def run_until_complete(self, future):
    """Run the event loop until a Future is done.
    Return the Future's result, or raise its exception.
    """
    raise NotImplementedError
    def is_running(self):
    """Return whether the event loop is currently running."""
    raise NotImplementedError
    

    View Slide

41. asyncio/base_events.py
    def is_running(self):
    """Returns True if the event loop is running."""
    return (self._thread_id is not None)
    

    View Slide

42. asyncio/base_events.py
    def run_until_complete(self, future):
    ...
    future.add_done_callback(_run_until_complete_cb)
    try:
    self.run_forever()
    except:
    if new_task and future.done() and not future.cancelled():
    future.exception()
    raise
    finally:
    future.remove_done_callback(_run_until_complete_cb)
    ...
    

    View Slide

43. asyncio/base_events.py
    def run_until_complete(self, future):
    ...
    future.add_done_callback(_run_until_complete_cb)
    try:
    self.run_forever()
    except:
    if new_task and future.done() and not future.cancelled():
    future.exception()
    raise
    finally:
    future.remove_done_callback(_run_until_complete_cb)
    ...
    def run_until_complete(self, future):
    ...
    future.add_done_callback(_run_until_complete_cb)
    try:
    self.run_forever()
    except:
    if new_task and future.done() and not future.cancelled():
    future.exception()
    raise
    finally:
    future.remove_done_callback(_run_until_complete_cb)
    ...
    

    View Slide

44. asyncio/base_events.py
    def run_until_complete(self, future):
    ...
    future.add_done_callback(_run_until_complete_cb)
    try:
    self.run_forever()
    except:
    if new_task and future.done() and not future.cancelled():
    future.exception()
    raise
    finally:
    future.remove_done_callback(_run_until_complete_cb)
    ...
    def run_until_complete(self, future):
    ...
    future.add_done_callback(_run_until_complete_cb)
    try:
    self.run_forever()
    except:
    if new_task and future.done() and not future.cancelled():
    future.exception()
    raise
    finally:
    future.remove_done_callback(_run_until_complete_cb)
    ...
    

    View Slide

45. asyncio/base_events.py
    def _run_until_complete_cb(fut):
    exc = fut._exception
    ...
    fut._loop.stop()
    

    View Slide

46. asyncio/base_events.py
    def _run_until_complete_cb(fut):
    exc = fut._exception
    ...
    fut._loop.stop()
    def _run_until_complete_cb(fut):
    exc = fut._exception
    ...
    fut._loop.stop()
    

    View Slide

47. asyncio/base_events.py
    def run_forever(self):
    """Run until stop() is called."""
    self._check_closed()
    ...
    self._thread_id = threading.get_ident()
    ...
    try:
    events._set_running_loop(self)
    while True:
    self._run_once()
    if self._stopping:
    break
    finally:
    self._stopping = False
    self._thread_id = None
    events._set_running_loop(None)
    

    View Slide

48. asyncio/base_events.py
    def run_forever(self):
    """Run until stop() is called."""
    self._check_closed()
    ...
    self._thread_id = threading.get_ident()
    ...
    try:
    events._set_running_loop(self)
    while True:
    self._run_once()
    if self._stopping:
    break
    finally:
    self._stopping = False
    self._thread_id = None
    events._set_running_loop(None)
    def run_forever(self):
    """Run until stop() is called."""
    self._check_closed()
    ...
    self._thread_id = threading.get_ident()
    ...
    try:
    events._set_running_loop(self)
    while True:
    self._run_once()
    if self._stopping:
    break
    finally:
    self._stopping = False
    self._thread_id = None
    events._set_running_loop(None)
    

    View Slide

49. asyncio/base_events.py
    def _run_once(self):
    """Run one full iteration of the event loop.
    This calls all currently ready callbacks, polls for I/O,
    schedules the resulting callbacks, and finally schedules
    'call_later' callbacks.
    """
    

    View Slide

50. asyncio/base_events.py
    def _run_once(self):
    """Run one full iteration of the event loop.
    This calls all currently ready callbacks, polls for I/O,
    schedules the resulting callbacks, and finally schedules
    'call_later' callbacks.
    """
    

    View Slide

51. asyncio/base_events.py
    def _run_once(self):
    """Run one full iteration of the event loop.
    This calls all currently ready callbacks, polls for I/O,
    schedules the resulting callbacks, and finally schedules
    'call_later' callbacks.
    """
    

    View Slide

52. asyncio/base_events.py
    def _run_once(self):
    """Run one full iteration of the event loop.
    This calls all currently ready callbacks, polls for I/O,
    schedules the resulting callbacks, and finally schedules
    'call_later' callbacks.
    """
    

    View Slide

53. asyncio/base_events.py
    def _run_once(self):
    """Run one full iteration of the event loop.
    This calls all currently ready callbacks, polls for I/O,
    schedules the resulting callbacks, and finally schedules
    'call_later' callbacks.
    """
    

    View Slide

54. asyncio/base_events.py
    def _run_once(self):
    ...
    ntodo = len(self._ready)
    for i in range(ntodo):
    handle = self._ready.popleft()
    if handle._cancelled:
    continue
    if self._debug:
    try:
    self._current_handle = handle
    t0 = self.time()
    handle._run()
    dt = self.time() - t0
    if dt >= self.slow_callback_duration:
    logger.warning('Executing %s took %.3f seconds',
    _format_handle(handle), dt)
    finally:
    self._current_handle = None
    else:
    handle._run()
    

    View Slide

55. asyncio/base_events.py
    def _run_once(self):
    ...
    ntodo = len(self._ready)
    for i in range(ntodo):
    handle = self._ready.popleft()
    if handle._cancelled:
    continue
    if self._debug:
    try:
    self._current_handle = handle
    t0 = self.time()
    handle._run()
    dt = self.time() - t0
    if dt >= self.slow_callback_duration:
    logger.warning('Executing %s took %.3f seconds',
    _format_handle(handle), dt)
    finally:
    self._current_handle = None
    else:
    handle._run()
    def _run_once(self):
    ...
    ntodo = len(self._ready)
    for i in range(ntodo):
    handle = self._ready.popleft()
    if handle._cancelled:
    continue
    if self._debug:
    try:
    self._current_handle = handle
    t0 = self.time()
    handle._run()
    dt = self.time() - t0
    if dt >= self.slow_callback_duration:
    logger.warning('Executing %s took %.3f seconds',
    _format_handle(handle), dt)
    finally:
    self._current_handle = None
    else:
    handle._run()
    

    View Slide

56. asyncio/base_events.py
    def _run_once(self):
    ...
    ntodo = len(self._ready)
    for i in range(ntodo):
    handle = self._ready.popleft()
    if handle._cancelled:
    continue
    if self._debug:
    try:
    self._current_handle = handle
    t0 = self.time()
    handle._run()
    dt = self.time() - t0
    if dt >= self.slow_callback_duration:
    logger.warning('Executing %s took %.3f seconds',
    _format_handle(handle), dt)
    finally:
    self._current_handle = None
    else:
    handle._run()
    def _run_once(self):
    ...
    ntodo = len(self._ready)
    for i in range(ntodo):
    handle = self._ready.popleft()
    if handle._cancelled:
    continue
    if self._debug:
    try:
    self._current_handle = handle
    t0 = self.time()
    handle._run()
    dt = self.time() - t0
    if dt >= self.slow_callback_duration:
    logger.warning('Executing %s took %.3f seconds',
    _format_handle(handle), dt)
    finally:
    self._current_handle = None
    else:
    handle._run()
    

    View Slide

57. asyncio/base_events.py
    def _run_once(self):
    ...
    ntodo = len(self._ready)
    for i in range(ntodo):
    handle = self._ready.popleft()
    if handle._cancelled:
    continue
    if self._debug:
    try:
    self._current_handle = handle
    t0 = self.time()
    handle._run()
    dt = self.time() - t0
    if dt >= self.slow_callback_duration:
    logger.warning('Executing %s took %.3f seconds',
    _format_handle(handle), dt)
    finally:
    self._current_handle = None
    else:
    handle._run()
    def _run_once(self):
    ...
    ntodo = len(self._ready)
    for i in range(ntodo):
    handle = self._ready.popleft()
    if handle._cancelled:
    continue
    if self._debug:
    try:
    self._current_handle = handle
    t0 = self.time()
    handle._run()
    dt = self.time() - t0
    if dt >= self.slow_callback_duration:
    logger.warning('Executing %s took %.3f seconds',
    _format_handle(handle), dt)
    finally:
    self._current_handle = None
    else:
    handle._run()
    

    View Slide

58. aiohttp/web.py
    try:
    print("======== Running on {} ========\n"
    "(Press CTRL+C to quit)".format(', '.join(uris)))
    loop.run_forever()
    except (GracefulExit, KeyboardInterrupt):
    pass
    try:
    print("======== Running on {} ========\n"
    "(Press CTRL+C to quit)".format(', '.join(uris)))
    loop.run_forever()
    except (GracefulExit, KeyboardInterrupt):
    pass
    

    View Slide

59. aiohttp/web.py
    server_creations, uris = _make_server_creators(
    handler,
    loop=loop, ssl_context=ssl_context,
    host=host, port=port, path=path, sock=sock,
    backlog=backlog)
    servers = loop.run_until_complete(
    asyncio.gather(*server_creations, loop=loop)
    )
    

    View Slide

60. aiohttp/web.py
    server_creations, uris = _make_server_creators(
    handler,
    loop=loop, ssl_context=ssl_context,
    host=host, port=port, path=path, sock=sock,
    backlog=backlog)
    servers = loop.run_until_complete(
    asyncio.gather(*server_creations, loop=loop)
    )
    server_creations, uris = _make_server_creators(
    handler,
    loop=loop, ssl_context=ssl_context,
    host=host, port=port, path=path, sock=sock,
    backlog=backlog)
    servers = loop.run_until_complete(
    asyncio.gather(*server_creations, loop=loop)
    )
    

    View Slide

61. aiohttp/web.py
    def _make_server_creators(handler, *, loop, ssl_context,
    host, port, path, sock, backlog):
    host_binding = hosts[0] if len(hosts) == 1 else hosts
    server_creations.append(
    loop.create_server(handler, host_binding, port, ...)
    )
    

    View Slide

62. aiohttp/web.py
    def _make_server_creators(handler, *, loop, ssl_context,
    host, port, path, sock, backlog):
    host_binding = hosts[0] if len(hosts) == 1 else hosts
    server_creations.append(
    loop.create_server(handler, host_binding, port, ...)
    )
    def _make_server_creators(handler, *, loop, ssl_context,
    host, port, path, sock, backlog):
    host_binding = hosts[0] if len(hosts) == 1 else hosts
    server_creations.append(
    loop.create_server(handler, host_binding, port, ...)
    )
    

    View Slide

63. asyncio/base_events.py
    @coroutine
    def create_server(self, protocol_factory, host=None, port=None,
    *,
    family=socket.AF_UNSPEC,
    flags=socket.AI_PASSIVE,
    sock=None,
    backlog=100,
    ssl=None,
    reuse_address=None,
    reuse_port=None):
    ...
    for sock in sockets:
    sock.listen(backlog)
    sock.setblocking(False)
    self._start_serving(protocol_factory, sock, ssl, server, backlog)
    

    View Slide

64. asyncio/base_events.py
    @coroutine
    def create_server(self, protocol_factory, host=None, port=None,
    *,
    family=socket.AF_UNSPEC,
    flags=socket.AI_PASSIVE,
    sock=None,
    backlog=100,
    ssl=None,
    reuse_address=None,
    reuse_port=None):
    ...
    for sock in sockets:
    sock.listen(backlog)
    sock.setblocking(False)
    self._start_serving(protocol_factory, sock, ssl, server, backlog)
    @coroutine
    def create_server(self, protocol_factory, host=None, port=None,
    *,
    family=socket.AF_UNSPEC,
    flags=socket.AI_PASSIVE,
    sock=None,
    backlog=100,
    ssl=None,
    reuse_address=None,
    reuse_port=None):
    ...
    for sock in sockets:
    sock.listen(backlog)
    sock.setblocking(False)
    self._start_serving(protocol_factory, sock, ssl, server, backlog)
    

    View Slide

65. asyncio/base_events.py
    @coroutine
    def create_server(self, protocol_factory, host=None, port=None,
    *,
    family=socket.AF_UNSPEC,
    flags=socket.AI_PASSIVE,
    sock=None,
    backlog=100,
    ssl=None,
    reuse_address=None,
    reuse_port=None):
    ...
    for sock in sockets:
    sock.listen(backlog)
    sock.setblocking(False)
    self._start_serving(protocol_factory, sock, ssl, server, backlog)
    @coroutine
    def create_server(self, protocol_factory, host=None, port=None,
    *,
    family=socket.AF_UNSPEC,
    flags=socket.AI_PASSIVE,
    sock=None,
    backlog=100,
    ssl=None,
    reuse_address=None,
    reuse_port=None):
    ...
    for sock in sockets:
    sock.listen(backlog)
    sock.setblocking(False)
    self._start_serving(protocol_factory, sock, ssl, server, backlog)
    

    View Slide

66. aiohttp/web.py
    server_creations, uris = _make_server_creators(
    handler,
    loop=loop, ssl_context=ssl_context,
    host=host, port=port, path=path, sock=sock,
    backlog=backlog)
    servers = loop.run_until_complete(
    asyncio.gather(*server_creations, loop=loop)
    )
    server_creations, uris = _make_server_creators(
    handler,
    loop=loop, ssl_context=ssl_context,
    host=host, port=port, path=path, sock=sock,
    backlog=backlog)
    servers = loop.run_until_complete(
    asyncio.gather(*server_creations, loop=loop)
    )
    

    View Slide

67. aiohttp/web.py
    handler = app.make_handler(
    loop=loop,
    access_log=access_log,
    **make_handler_kwargs
    )
    

    View Slide

68. aiohttp/web.py
    def make_handler(self, *, loop=None,
    secure_proxy_ssl_header=None, **kwargs):
    self._set_loop(loop)
    ...
    return Server(self._handle, request_factory=self._make_request,
    loop=self.loop, **kwargs)
    

    View Slide

69. aiohttp/web.py
    def make_handler(self, *, loop=None,
    secure_proxy_ssl_header=None, **kwargs):
    self._set_loop(loop)
    ...
    return Server(self._handle, request_factory=self._make_request,
    loop=self.loop, **kwargs)
    def make_handler(self, *, loop=None,
    secure_proxy_ssl_header=None, **kwargs):
    self._set_loop(loop)
    ...
    return Server(self._handle, request_factory=self._make_request,
    loop=self.loop, **kwargs)
    

    View Slide

70. aiohttp/web_server.py
    class Server:
    ...
    @asyncio.coroutine
    def shutdown(self, timeout=None):
    coros = [conn.shutdown(timeout) for conn in self._connections]
    yield from asyncio.gather(*coros, loop=self._loop)
    self._connections.clear()
    self.time_service.close()
    def __call__(self):
    return RequestHandler(self, loop=self._loop, **self._kwargs)
    

    View Slide

71. aiohttp/web_server.py
    class Server:
    ...
    @asyncio.coroutine
    def shutdown(self, timeout=None):
    coros = [conn.shutdown(timeout) for conn in self._connections]
    yield from asyncio.gather(*coros, loop=self._loop)
    self._connections.clear()
    self.time_service.close()
    def __call__(self):
    return RequestHandler(self, loop=self._loop, **self._kwargs)
    class Server:
    ...
    @asyncio.coroutine
    def shutdown(self, timeout=None):
    coros = [conn.shutdown(timeout) for conn in self._connections]
    yield from asyncio.gather(*coros, loop=self._loop)
    self._connections.clear()
    self.time_service.close()
    def __call__(self):
    return RequestHandler(self, loop=self._loop, **self._kwargs)
    

    View Slide

72. aiohttp/web_protocol.py
    class RequestHandler(..., asyncio.Protocol):
    

    View Slide

73. aiohttp/web_protocol.py
    class RequestHandler(..., asyncio.Protocol):
    class RequestHandler(..., asyncio.Protocol):
    

    View Slide

74. asyncio/protocols.py
    class Protocol(BaseProtocol):
    """Interface for stream protocol.
    ...
    State machine of calls:
    start -> CM [-> DR*] [-> ER?] -> CL -> end
    * CM: connection_made()
    * DR: data_received()
    * ER: eof_received()
    * CL: connection_lost()
    """
    def data_received(self, data):
    pass
    def eof_received(self):
    pass
    

    View Slide

75. asyncio/protocols.py
    class Protocol(BaseProtocol):
    """Interface for stream protocol.
    ...
    State machine of calls:
    start -> CM [-> DR*] [-> ER?] -> CL -> end
    * CM: connection_made()
    * DR: data_received()
    * ER: eof_received()
    * CL: connection_lost()
    """
    def data_received(self, data):
    pass
    def eof_received(self):
    pass
    class Protocol(BaseProtocol):
    """Interface for stream protocol.
    ...
    State machine of calls:
    start -> CM [-> DR*] [-> ER?] -> CL -> end
    * CM: connection_made()
    * DR: data_received()
    * ER: eof_received()
    * CL: connection_lost()
    """
    def data_received(self, data):
    pass
    def eof_received(self):
    pass
    

    View Slide

76. asyncio/protocols.py
    class Protocol(BaseProtocol):
    """Interface for stream protocol.
    ...
    State machine of calls:
    start -> CM [-> DR*] [-> ER?] -> CL -> end
    * CM: connection_made()
    * DR: data_received()
    * ER: eof_received()
    * CL: connection_lost()
    """
    def data_received(self, data):
    pass
    def eof_received(self):
    pass
    class Protocol(BaseProtocol):
    """Interface for stream protocol.
    ...
    State machine of calls:
    start -> CM [-> DR*] [-> ER?] -> CL -> end
    * CM: connection_made()
    * DR: data_received()
    * ER: eof_received()
    * CL: connection_lost()
    """
    def data_received(self, data):
    pass
    def eof_received(self):
    pass
    

    View Slide

77. asyncio/protocols.py
    class Protocol(BaseProtocol):
    """Interface for stream protocol.
    ...
    State machine of calls:
    start -> CM [-> DR*] [-> ER?] -> CL -> end
    * CM: connection_made()
    * DR: data_received()
    * ER: eof_received()
    * CL: connection_lost()
    """
    def data_received(self, data):
    pass
    def eof_received(self):
    pass
    class Protocol(BaseProtocol):
    """Interface for stream protocol.
    ...
    State machine of calls:
    start -> CM [-> DR*] [-> ER?] -> CL -> end
    * CM: connection_made()
    * DR: data_received()
    * ER: eof_received()
    * CL: connection_lost()
    """
    def data_received(self, data):
    pass
    def eof_received(self):
    pass
    

    View Slide

78. aiohttp/web_protocol.py
    class RequestHandler(..., asyncio.Protocol):
    def data_received(self, data):
    ...
    messages, upgraded, tail = self._request_parser.feed_data(data)
    

    View Slide

79. aiohttp server patterns
    

    View Slide

80. app = Application()
    app['conf'] = configuration
    app['db'] = db
    app.on_startup.append(init_db)
    app.on_cleanup.append(close_db)
    app.router.add_get('/', index_handler)
    app.router.add_get(r'/resources/{resource_id:\d+}', resource_handler)
    app.middlewares.append(error_handling_middleware)
    

    View Slide

81. app = Application()
    app['conf'] = configuration
    app['db'] = db
    app.on_startup.append(init_db)
    app.on_cleanup.append(close_db)
    app.router.add_get('/', index_handler)
    app.router.add_get(r'/resources/{resource_id:\d+}', resource_handler)
    app.middlewares.append(error_handling_middleware)
    app = Application()
    app['conf'] = configuration
    app['db'] = db
    app.on_startup.append(init_db)
    app.on_cleanup.append(close_db)
    app.router.add_get('/', index_handler)
    app.router.add_get(r'/resources/{resource_id:\d+}', resource_handler)
    app.middlewares.append(error_handling_middleware)
    

    View Slide

82. app = Application()
    app['conf'] = configuration
    app['db'] = db
    app.on_startup.append(init_db)
    app.on_cleanup.append(close_db)
    app.router.add_get('/', index_handler)
    app.router.add_get(r'/resources/{resource_id:\d+}', resource_handler)
    app.middlewares.append(error_handling_middleware)
    app = Application()
    app['conf'] = configuration
    app['db'] = db
    app.on_startup.append(init_db)
    app.on_cleanup.append(close_db)
    app.router.add_get('/', index_handler)
    app.router.add_get(r'/resources/{resource_id:\d+}', resource_handler)
    app.middlewares.append(error_handling_middleware)
    

    View Slide

83. app = Application()
    app['conf'] = configuration
    app['db'] = db
    app.on_startup.append(init_db)
    app.on_cleanup.append(close_db)
    app.router.add_get('/', index_handler)
    app.router.add_get(r'/resources/{resource_id:\d+}', resource_handler)
    app.middlewares.append(error_handling_middleware)
    app = Application()
    app['conf'] = configuration
    app['db'] = db
    app.on_startup.append(init_db)
    app.on_cleanup.append(close_db)
    app.router.add_get('/', index_handler)
    app.router.add_get(r'/resources/{resource_id:\d+}', resource_handler)
    app.middlewares.append(error_handling_middleware)
    

    View Slide

84. app = Application()
    app['conf'] = configuration
    app['db'] = db
    app.on_startup.append(init_db)
    app.on_cleanup.append(close_db)
    app.router.add_get('/', index_handler)
    app.router.add_get(r'/resources/{resource_id:\d+}', resource_handler)
    app.middlewares.append(error_handling_middleware)
    app = Application()
    app['conf'] = configuration
    app['db'] = db
    app.on_startup.append(init_db)
    app.on_cleanup.append(close_db)
    app.router.add_get('/', index_handler)
    app.router.add_get(r'/resources/{resource_id:\d+}', resource_handler)
    app.middlewares.append(error_handling_middleware)
    

    View Slide

85. My one year with aiohttp
    

    View Slide

86. 1. Performance
    import pymongo
    from flask import Flask
    app = Flask(__name__)
    client = pymongo.MongoClient('localhost', 27017)
    db = client.test
    @app.route('/')
    def hello():
    return 'Hello, Pycon!'
    @app.route('/fetch')
    def fetch():
    document = db.greetings.find_one()
    return document['pycon']
    if __name__ == '__main__':
    app.run(debug=False)
    

    View Slide

87. 1. Performance
    import motor.motor_asyncio
    from aiohttp.web import Application, Response, run_app
    app = Application()
    client = motor.motor_asyncio.AsyncIOMotorClient()
    app['db'] = client.test
    async def hello(request):
    return Response(text='Hello, Pycon!')
    async def fetch(request):
    db = request.app['db']
    document = await db.greetings.find_one()
    return Response(text=document['pycon'])
    app.router.add_get('/', hello)
    app.router.add_get('/fetch', fetch)
    if __name__ == '__main__':
    run_app(app)
    

    View Slide

88. 1. Performance
    $ wrk -t12 -c400 -d10s flask
    Running 10s test @ flask
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 75.59ms 11.75ms 135.98ms 78.48%
    Req/Sec 139.45 59.91 323.00 64.60%
    15775 requests in 10.09s, 2.51MB read
    Socket errors: connect 0, read 870, write 7, timeout 0
    Requests/sec: 1562.90
    Transfer/sec: 254.89KB
    $ wrk -t12 -c400 -d10s aiohttp
    Running 10s test @ aiohttp
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 56.36ms 9.80ms 133.68ms 70.69%
    Req/Sec 539.28 95.55 0.93k 74.25%
    64714 requests in 10.08s, 10.12MB read
    Socket errors: connect 0, read 257, write 0, timeout 0
    Requests/sec: 6423.19
    Transfer/sec: 1.00MB
    $ wrk -t12 -c400 -d10s flask
    Running 10s test @ flask
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 75.59ms 11.75ms 135.98ms 78.48%
    Req/Sec 139.45 59.91 323.00 64.60%
    15775 requests in 10.09s, 2.51MB read
    Socket errors: connect 0, read 870, write 7, timeout 0
    Requests/sec: 1562.90
    Transfer/sec: 254.89KB
    $ wrk -t12 -c400 -d10s aiohttp
    Running 10s test @ aiohttp
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 56.36ms 9.80ms 133.68ms 70.69%
    Req/Sec 539.28 95.55 0.93k 74.25%
    64714 requests in 10.08s, 10.12MB read
    Socket errors: connect 0, read 257, write 0, timeout 0
    Requests/sec: 6423.19
    Transfer/sec: 1.00MB
    

    View Slide

89. $ wrk -t12 -c400 -d10s flask
    Running 10s test @ flask
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 75.59ms 11.75ms 135.98ms 78.48%
    Req/Sec 139.45 59.91 323.00 64.60%
    15775 requests in 10.09s, 2.51MB read
    Socket errors: connect 0, read 870, write 7, timeout 0
    Requests/sec: 1562.90
    Transfer/sec: 254.89KB
    $ wrk -t12 -c400 -d10s aiohttp
    Running 10s test @ aiohttp
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 56.36ms 9.80ms 133.68ms 70.69%
    Req/Sec 539.28 95.55 0.93k 74.25%
    64714 requests in 10.08s, 10.12MB read
    Socket errors: connect 0, read 257, write 0, timeout 0
    Requests/sec: 6423.19
    Transfer/sec: 1.00MB
    $ wrk -t12 -c400 -d10s flask
    Running 10s test @ flask
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 75.59ms 11.75ms 135.98ms 78.48%
    Req/Sec 139.45 59.91 323.00 64.60%
    15775 requests in 10.09s, 2.51MB read
    Socket errors: connect 0, read 870, write 7, timeout 0
    Requests/sec: 1562.90
    Transfer/sec: 254.89KB
    $ wrk -t12 -c400 -d10s aiohttp
    Running 10s test @ aiohttp
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 56.36ms 9.80ms 133.68ms 70.69%
    Req/Sec 539.28 95.55 0.93k 74.25%
    64714 requests in 10.08s, 10.12MB read
    Socket errors: connect 0, read 257, write 0, timeout 0
    Requests/sec: 6423.19
    Transfer/sec: 1.00MB
    1. Performance
    

    View Slide

90. 1. Performance (w/ MongoDB)
    $ wrk -t12 -c400 -d10s flask/fetch
    Running 10s test @ flask/fetch
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 118.74ms 10.27ms 147.84ms 89.06%
    Req/Sec 89.69 40.24 200.00 63.36%
    10785 requests in 10.09s, 1.72MB read
    Socket errors: connect 0, read 1039, write 6, timeout 0
    Requests/sec: 1068.89
    Transfer/sec: 174.32KB
    $ wrk -t12 -c400 -d10s aiohttp/fetch
    Running 10s test @ aiohttp/fetch
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 178.17ms 31.92ms 321.67ms 66.33%
    Req/Sec 169.80 79.86 333.00 57.36%
    20344 requests in 10.09s, 3.18MB read
    Socket errors: connect 0, read 253, write 0, timeout 0
    Requests/sec: 2015.68
    Transfer/sec: 322.82KB
    $ wrk -t12 -c400 -d10s flask/fetch
    Running 10s test @ flask/fetch
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 118.74ms 10.27ms 147.84ms 89.06%
    Req/Sec 89.69 40.24 200.00 63.36%
    10785 requests in 10.09s, 1.72MB read
    Socket errors: connect 0, read 1039, write 6, timeout 0
    Requests/sec: 1068.89
    Transfer/sec: 174.32KB
    $ wrk -t12 -c400 -d10s aiohttp/fetch
    Running 10s test @ aiohttp/fetch
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 178.17ms 31.92ms 321.67ms 66.33%
    Req/Sec 169.80 79.86 333.00 57.36%
    20344 requests in 10.09s, 3.18MB read
    Socket errors: connect 0, read 253, write 0, timeout 0
    Requests/sec: 2015.68
    Transfer/sec: 322.82KB
    

    View Slide

91. 1. Performance (w/ MongoDB)
    $ wrk -t12 -c400 -d10s flask/fetch
    Running 10s test @ flask/fetch
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 118.74ms 10.27ms 147.84ms 89.06%
    Req/Sec 89.69 40.24 200.00 63.36%
    10785 requests in 10.09s, 1.72MB read
    Socket errors: connect 0, read 1039, write 6, timeout 0
    Requests/sec: 1068.89
    Transfer/sec: 174.32KB
    $ wrk -t12 -c400 -d10s aiohttp/fetch
    Running 10s test @ aiohttp/fetch
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 178.17ms 31.92ms 321.67ms 66.33%
    Req/Sec 169.80 79.86 333.00 57.36%
    20344 requests in 10.09s, 3.18MB read
    Socket errors: connect 0, read 253, write 0, timeout 0
    Requests/sec: 2015.68
    Transfer/sec: 322.82KB
    $ wrk -t12 -c400 -d10s flask/fetch
    Running 10s test @ flask/fetch
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 118.74ms 10.27ms 147.84ms 89.06%
    Req/Sec 89.69 40.24 200.00 63.36%
    10785 requests in 10.09s, 1.72MB read
    Socket errors: connect 0, read 1039, write 6, timeout 0
    Requests/sec: 1068.89
    Transfer/sec: 174.32KB
    $ wrk -t12 -c400 -d10s aiohttp/fetch
    Running 10s test @ aiohttp/fetch
    12 threads and 400 connections
    Thread Stats Avg Stdev Max +/- Stdev
    Latency 178.17ms 31.92ms 321.67ms 66.33%
    Req/Sec 169.80 79.86 333.00 57.36%
    20344 requests in 10.09s, 3.18MB read
    Socket errors: connect 0, read 253, write 0, timeout 0
    Requests/sec: 2015.68
    Transfer/sec: 322.82KB
    

    View Slide

92. 2. Design
    • No global state or context
    • Separation between handlers and URLs
    • HTTP exception by return not raise
    • Well written code
    

    View Slide

93. 2. Design
    • No global state or context
    • Separation between handlers and URLs
    • HTTP exception by return not raise
    • Well written code
    

    View Slide

94. 2. Design
    • No global state or context
    • Separation between handlers and URLs
    • HTTP exception by return not raise
    • Well written code
    

    View Slide

95. 2. Design
    • No global state or context
    • Separation between handlers and URLs
    • HTTP exception by return not raise
    • Well written code
    

    View Slide

96. 3. Test
    asyncio based tests are
    well supported by libraries.
    

    View Slide

97. 3. Test (official)
    $ pip install pytest-aiohttp
    

    View Slide

98. 3. Test (official)
    from aiohttp import web
    async def hello(request):
    return web.Response(text='Hello, world')
    async def test_hello(test_client, loop):
    app = web.Application()
    app.router.add_get('/', hello)
    client = await test_client(app)
    resp = await client.get('/')
    assert resp.status == 200
    text = await resp.text()
    assert 'Hello, world' in text
    

    View Slide

99. 3. Test (preference)
    $ pip install asynctest
    

    View Slide

100. 3. Test (preference)
     class TestFetch(asynctest.TestCase):
     use_default_loop = True
     forbid_get_event_loop = True
     async def test_fetch(self):
     outcome = await fetch()
     self.assertEqual(outcome, 'Hello, Pycon!')
     

     View Slide

101. 4. Optimization
     async def update_entity(request):
     await data_repository.update_category(entity_id)
     await cache_repository.clear(entity_id)
     

     View Slide

102. 4. Optimization
     async def update_entity(request):
     await asyncio.gather(
     data_repository.update_category(entity_id),
     cache_repository.clear(entity_id),
     loop=request.app.loop
     )
     

     View Slide

103. 4. Optimization
     $ pip install uvloop
     

     View Slide

104. 4. Optimization
     import asyncio
     import uvloop
     asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
     

     View Slide

105. 4. Optimization
     

     View Slide

106. 4. Optimization
     $ wrk -t12 -c400 -d10s aiohttp
     Running 10s test @ aiohttp
     12 threads and 400 connections
     Thread Stats Avg Stdev Max +/- Stdev
     Latency 50.67ms 18.08ms 116.25ms 69.37%
     Req/Sec 601.92 98.27 0.92k 77.78%
     72351 requests in 10.07s, 11.32MB read
     Socket errors: connect 0, read 250, write 0, timeout 0
     Requests/sec: 7184.30
     Transfer/sec: 1.12MB
     

     View Slide

107. 4. Optimization
     $ wrk -t12 -c400 -d10s aiohttp
     Running 10s test @ aiohttp
     12 threads and 400 connections
     Thread Stats Avg Stdev Max +/- Stdev
     Latency 50.67ms 18.08ms 116.25ms 69.37%
     Req/Sec 601.92 98.27 0.92k 77.78%
     72351 requests in 10.07s, 11.32MB read
     Socket errors: connect 0, read 250, write 0, timeout 0
     Requests/sec: 7184.30
     Transfer/sec: 1.12MB
     1.1x faster
     

     View Slide

108. 4. Optimization
     CPU Bound(ex. HTTP Request Parser) >> I/O Bound
     

     View Slide

109. 4. Optimization
     $ pip install cchardet
     

     View Slide

110. 4. Optimization
     $ pip install ujson
     

     View Slide

111. 4. Optimization
     async def hello(request):
     return json_response(
     data={'greeting': 'Hello, Pycon!'},
     dumps=ujson.dumps
     )
     

     View Slide

112. 4. Optimization
     $ pip install aiohttp-ultrajson
     

     View Slide

113. 4. Optimization
     async def hello(request):
     return jsonify({'greeting': 'Hello, Pycon!'})
     

     View Slide

114. 5. Deployment
     $ pip install gunicorn
     

     View Slide

115. 5. Deployment
     gunicorn app:app --bind localhost:8080
     --worker-class aiohttp.GunicornWebWorker
     

     View Slide

116. 5. Deployment (w/ UVLoop)
     gunicorn app:app --bind localhost:8080
     --worker-class aiohttp.GunicornUVLoopWebWorker
     

     View Slide

117. 5. Deployment (production)
     import multiprocessing
     # Server socket
     bind = 'unix:app.sock'
     backlog = 2048
     # Worker processes
     workers = multiprocessing.cpu_count() * 2 + 1
     worker_class = 'aiohttp.worker.GunicornUVLoopWebWorker'
     timeout = 30
     

     View Slide

118. 5. Deployment (production)
     import multiprocessing
     # Server socket
     bind = 'unix:app.sock'
     backlog = 2048
     # Worker processes
     workers = multiprocessing.cpu_count() * 2 + 1
     worker_class = 'aiohttp.worker.GunicornUVLoopWebWorker'
     timeout = 30
     import multiprocessing
     # Server socket
     bind = 'unix:app.sock'
     backlog = 2048
     # Worker processes
     workers = multiprocessing.cpu_count() * 2 + 1
     worker_class = 'aiohttp.worker.GunicornUVLoopWebWorker'
     timeout = 30
     

     View Slide

119. 5. Deployment (production)
     import multiprocessing
     # Server socket
     bind = 'unix:app.sock'
     backlog = 2048
     # Worker processes
     workers = multiprocessing.cpu_count() * 2 + 1
     worker_class = 'aiohttp.worker.GunicornUVLoopWebWorker'
     timeout = 30
     import multiprocessing
     # Server socket
     bind = 'unix:app.sock'
     backlog = 2048
     # Worker processes
     workers = multiprocessing.cpu_count() * 2 + 1
     worker_class = 'aiohttp.worker.GunicornUVLoopWebWorker'
     timeout = 30
     

     View Slide

120. 5. Deployment (production)
     import multiprocessing
     # Server socket
     bind = 'unix:app.sock'
     backlog = 2048
     # Worker processes
     workers = multiprocessing.cpu_count() * 2 + 1
     worker_class = 'aiohttp.worker.GunicornUVLoopWebWorker'
     timeout = 30
     import multiprocessing
     # Server socket
     bind = 'unix:app.sock'
     backlog = 2048
     # Worker processes
     workers = multiprocessing.cpu_count() * 2 + 1
     worker_class = 'aiohttp.worker.GunicornUVLoopWebWorker'
     timeout = 30
     

     View Slide

121. 5. Deployment (production)
     gunicorn --config file:/app/gunicorn.py app:app
     

     View Slide

122. 6. Fun and Difficulties
     • (AFAIK) No one uses aiohttp
     • (AFAIK) No one knows aiohttp
     • DIY is required
     • aiohttp and aiobotocore contributions
     • small community but well maintained
     • Asynchronous programming is usually more
     difficult
     

     View Slide

123. • (AFAIK) No one uses aiohttp
     • (AFAIK) No one knows aiohttp
     • DIY is required
     • aiohttp and aiobotocore contributions
     • small community but well maintained
     • Asynchronous programming is usually more
     difficult
     6. Fun and Difficulties
     

     View Slide

124. • (AFAIK) No one uses aiohttp
     • (AFAIK) No one knows aiohttp
     • DIY is required
     • aiohttp and aiobotocore contributions
     • small community but well maintained
     • Asynchronous programming is usually more
     difficult
     6. Fun and Difficulties
     

     View Slide

125. • (AFAIK) No one uses aiohttp
     • (AFAIK) No one knows aiohttp
     • DIY is required
     • aiohttp and aiobotocore contributions
     • small community but well maintained
     • Asynchronous programming is usually more
     difficult
     6. Fun and Difficulties
     

     View Slide

126. • (AFAIK) No one uses aiohttp
     • (AFAIK) No one knows aiohttp
     • DIY is required
     • aiohttp and aiobotocore contributions
     • small community but well maintained
     • Asynchronous programming is usually more
     difficult
     6. Fun and Difficulties
     

     View Slide

127. • (AFAIK) No one uses aiohttp
     • (AFAIK) No one knows aiohttp
     • DIY is required
     • aiohttp and aiobotocore contributions
     • small community but well maintained
     • Asynchronous programming is usually more
     difficult
     6. Fun and Difficulties
     

     View Slide

128. to use aiohttp,
     or not to use aiohttp?
     

     View Slide

129. aiohttp requires asyncio
     supported libraries
     

     View Slide

130. • motor (MongoDB)
     • aiomysql (MySQL)
     • asyncpg (PostgreSQL)
     • aioredis (Redis)
     • aiomcache (Memcached)
     • aiokafka (Apache Kafka)
     • aiozmq (ZeroMQ)
     

     View Slide

131. Is your team ready for
     asynchronous programming?
     

     View Slide

132. In my case, Rainist was
     • Using RxJava, RxSwift, and RxJs highly
     on client side
     • Implementing CPU bound services with
     Scala, Akka HTTP, and Monix
     

     View Slide

133. from __future__ import asyncio
     

     View Slide

134. Rainist: We are hiring!
     https://rainist.com
     https://medium.com/rainist-engineering
     

     View Slide

135. References
     1. Luciano Ramalho, , 2015
     2. PEP 380 Syntax for Delegating to a Subgenerator
     3. PEP 3156 Asynchronous IO Support Rebooted: the "asyncio"
     Module
     4. https://magic.io/blog/uvloop-blazing-fast-python-networking/
     5. http://aiohttp.readthedocs.io/en/stable/
     6. https://nikhilm.github.io/uvbook/basics.html
     

     View Slide