Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Hacking Django Channels for Fun and Profit

Hacking Django Channels for Fun and Profit

Django is growing some great async features and Channels has been great for handling websockets connecting from your visitor's browser. But what happens when you need to do more? What if you want to keep long-running connections from Django to other websockets, such as Discord servers? How do you do this and still leverage all the batteries included with Django? We will show an approach that makes this all possible and easy as a developer.

4d1fa184d439599ed301821daec94063?s=128

Calvin Hendryx-Parker

June 01, 2021
Tweet

Transcript

  1. HACKING DJANGO CHANNELS FOR FUN AND PROFIT DJANGOCON EUROPE, JUNE

    2021 CALVIN HENDRYX-PARKER
  2. BACKSTORY » The web was mostly synchronous up until a

    few years ago » Async is becoming more a part of Django with each 3.x release
  3. REQUESTS AND RESPONSES CommonMiddleware SessionMiddleware CsrfViewMiddleware AuthenticationMiddleware MessageMiddleware HttpRequest HttpResponse

    process_request process_view process_template_response process_response process_exception view function
  4. Django Request Response Loop See https://www.youtube.com/watch?v=RLo9RJhbOrQ

  5. WSGI VS ASGI LET’S MAKE IT WORK BI-DIRECTIONAL! Ok… so

    just swap wsgi.py with asgi.py? See See more on ASGI here: https://arunrocks.com/a-guide-to-asgi-in-django-30-and-its- performance/ https://youtu.be/uRcnaI8Hnzg
  6. YOU NEED TO KNOW THAT A WEBSOCKET IS… » Let’s

    see it in action!
  7. ENTER CHANNELS » Consumers 👈 What we think when we

    talk about Channels and WebSockets » Channel Layers 👈 How we talk to and between our code » Background Workers 👈 Allows for background tasks like Celery
  8. CONSUMER EXAMPLE from channels.generic.websocket import JsonWebsocketConsumer class MyConsumer(JsonWebsocketConsumer): def connect(self):

    self.accept() self.send_json({"connected": "true"}) def disconnect(self, close_code): pass def receive_json(self, content, **kwargs)): type_code = content["type"] data = content["message"] if type_code == "greeting": if (name := data.get("name")): self.send_json({ "type": "reply_to_greeting", "message": f"hello there, {name}!" })
  9. None
  10. ASGI ROUTING application = SentryAsgiMiddleware( ProtocolTypeRouter( { # (http->django views

    is added by default) "websocket": AuthMiddlewareStack( URLRouter(loudswarm.chat.routing.websocket_urlpatterns) ), "channel": ChannelNameRouter( { "discord_send": DiscordSender, } ), }, ) )
  11. LET’S TALK ABOUT BACKGROUND WORKERS THAT LAST BIT IS WHERE

    WE WILL FOCUS
  12. WHAT IF WE WANT THE REVERSE?

  13. A WORKER IN CHANNELS? What can you do with it?

  14. LISTEN FOR MESSAGES ON THE CHANNEL LAYER AND THEN DO

    SOME WORK. » Fast » Easy » Lightweight Beware: at-most-once operation
  15. REAL WORLD USE CASE DISCORD CHAT BOT BUILT INSIDE OF

    DJANGO
  16. LET’S HACK OUR OWN WORKER We want something that Channels

    can do, but doesn’t out of the box
  17. We will do like the Channels runworker and make our

    own from the asgiref.server.StatelessServer # Only the one method is different from the default worker async def handle(self): """ Listens on all the provided channels and handles the messages. """ # For each channel, launch its own listening coroutine lackeys = [] for channel in self.channels: lackeys.append(asyncio.ensure_future(self.listener(channel))) # Add coroutine for outgoing websocket connection to Discord API lackeys.append(asyncio.ensure_future(self.outgoing_connection())) # Most of our time is spent here, waiting until all the lackeys exit await asyncio.wait(lackeys) # See if any of the listeners had an error (e.g. channel layer error) [lackey.result() for lackey in lackeys]
  18. WHY?

  19. TWO EXAMPLES We receive messages from Discord and we send

    them to our clients class ChatClient(discord.Client): async def on_message(self, message): data = { "channel": str(message.channel.id), "event_ts": str(message.created_at.timestamp()), "text": message.content, "ts": str(message.created_at.timestamp()), "user": message.author.nick, "event_id": str(message.id), "event_time": int(message.created_at.timestamp()), "team_id": str(message.guild.id), } await make_webhook_transaction(data)
  20. We generate a notification on a scheduled celery task and

    we want to send it to Discord so we write a consumer that listens for our message from discordworker import discord_client class DiscordSender(AsyncConsumer): async def send_to_discord(self, message): text_msg = message["message"] chat_chan = int(message["discord_channel"]) channel = discord_client.client.get_channel(chat_chan) await channel.send(text_msg) # this is Discord's "channel", not ours
  21. and we send it like this await channel_layer.send( "discord_send", #

    name from ChannelNameRouter config dict( type="send_to_discord", # maps to method name on consumer discord_channel=self.chat_chan, message=message, ), )
  22. Use this for any old long running task…

  23. WISH LIST » Get this added to Channels codebase »

    Show examples of long running and single shot coroutines » Add options for two classifications of coroutines » Ones that start right away » Optionally ones the run after those stop
  24. TIPS AND TRICKS (AKA NOT IN THE DOCS) » Channel

    Layer Capacity defaults to 100 If you push more to a channel group, they drop silently
  25. SO LONG AND THANKS FOR ALL THE FISH…