Slide 1

Slide 1 text

TAKING Andrew Godwin @andrewgodwin CHANNELS ASYNC

Slide 2

Slide 2 text

Hi, I’m Andrew Godwin • Django core team member • Senior Software Engineer at • Does network programming for "fun"

Slide 3

Slide 3 text

2015: "django-onair"

Slide 4

Slide 4 text

2015: Channels 0.1

Slide 5

Slide 5 text

2017: Channels 1.0

Slide 6

Slide 6 text

Python 2.7 compatible Twisted for webserver Synchronous Django code

Slide 7

Slide 7 text

Webserver (Twisted) Django (Sync worker process) Clients Channel Layer (Redis or other) Webserver (Twisted) Django (Sync worker process)

Slide 8

Slide 8 text

Too many moving pieces No asyncio support Easy to shoot yourself in the foot

Slide 9

Slide 9 text

I was wrong.

Slide 10

Slide 10 text

2018: Channels 2.0

Slide 11

Slide 11 text

Asyncio-native Python 3.5 and up Supports async coroutines and sync threads

Slide 12

Slide 12 text

Webserver (Twisted) Django (Sync worker process) Clients Webserver (Twisted) Django (Sync worker process) Channel Layer (Redis or other)

Slide 13

Slide 13 text

75% rewrite

Slide 14

Slide 14 text

Django had to be partially async

Slide 15

Slide 15 text

WSGI/ASGI Handler (request translator) URL Routing (path to view mapping) Django Middleware (auth, sessions, etc.) Views (logic and presentation) ORM (database kerfuffling)

Slide 16

Slide 16 text

WSGI/ASGI Handler (request translator) URL Routing (path to view mapping) Django Middleware (auth, sessions, etc.) Views (logic and presentation) ORM (database kerfuffling) ASGI Routing (path/protocol/etc. mapping) ASGI Middleware (auth, sessions, etc.) Consumers (logic and presentation) synchronous

Slide 17

Slide 17 text

Async-native most of the way

Slide 18

Slide 18 text

But how?

Slide 19

Slide 19 text

sync_to_async async_to_sync

Slide 20

Slide 20 text

sync_to_async async_to_sync

Slide 21

Slide 21 text

Synchronous code has to run in threads ThreadPoolExecutor does most of the hard work

Slide 22

Slide 22 text

loop = asyncio.get_event_loop() future = loop.run_in_executor( self.threadpool, func, ) return await future

Slide 23

Slide 23 text

Calling the ORM Rendering templates Handing off to Django views

Slide 24

Slide 24 text

sync_to_async async_to_sync

Slide 25

Slide 25 text

Async code runs on the event loop We need to go find it! Or make our own.

Slide 26

Slide 26 text

# Make a future for the return information call_result = Future() # Use call_soon_threadsafe to schedule a synchronous callback on the # main event loop's thread if not (self.main_event_loop and self.main_event_loop.is_running()): # Make our own event loop and run inside that. loop = asyncio.new_event_loop() asyncio.set_event_loop(loop) try: loop.run_until_complete(self.main_wrap(args, kwargs, call_result)) finally: try: if hasattr(loop, "shutdown_asyncgens"): loop.run_until_complete(loop.shutdown_asyncgens()) finally: loop.close() asyncio.set_event_loop(self.main_event_loop) else: self.main_event_loop.call_soon_threadsafe( self.main_event_loop.create_task, self.main_wrap( args, kwargs, call_result, ), ) # Wait for results from the future. return call_result.result()

Slide 27

Slide 27 text

Make a Future Jump to the main thread and add the coroutine Tie the coroutine's end to triggering the Future Block the thread on the Future

Slide 28

Slide 28 text

Lets us provide async-native APIs And still let synchronous code use them

Slide 29

Slide 29 text

Both async and sync code are useful Channels lets you write your code as both

Slide 30

Slide 30 text

Async interfaces are separate to sync You can't provide both through one API

Slide 31

Slide 31 text

bit.ly/py-async-simp https://www.aeracode.org/2018/02/19/python-async-simplified/

Slide 32

Slide 32 text

ASGI Handler (ASGI-to-request translator) URL Routing (path to view mapping) Django Middleware (auth, sessions, etc.) Views (logic and presentation) ORM (database kerfuffling) ASGI Routing (path/protocol/etc. mapping) ASGI Middleware (auth, sessions, etc.) Consumers (logic and presentation) synchronous

Slide 33

Slide 33 text

How to make WSGI async?

Slide 34

Slide 34 text

ASGI: It's like WSGI but with an A in it Also asyncio and stuff like that

Slide 35

Slide 35 text

class Application: def __init__(self, scope): ... async def __call__(self, receive, send): event = await receive() ... await send(data)

Slide 36

Slide 36 text

"Turtles all the way down" Routing is an ASGI app. Middleware is too.

Slide 37

Slide 37 text

What does this mean for Django?

Slide 38

Slide 38 text

How much can we make async? Can we keep sync compatability?

Slide 39

Slide 39 text

What does an async ORM look like? Is it even a sensible endeavour?

Slide 40

Slide 40 text

Do we need to replace WSGI? Or does nobody want long-polling or websockets?

Slide 41

Slide 41 text

Multiple servers (daphne, uvicorn) Multiple frameworks (Django, …?)

Slide 42

Slide 42 text

Do we want to have everyone writing async? What's the balance? How do we be flexible enough?

Slide 43

Slide 43 text

What is Django?

Slide 44

Slide 44 text

Thanks. Andrew Godwin @andrewgodwin aeracode.org