Phoenix Channels - a Distributed PubSub and Presence Platform

Channels are a really exciting and powerful part of the Phoenix Framework. They allow us to easily add soft-realtime features to our applications. Channels are based on a simple idea - sending and receiving messages.

In this talk, we'll take a look under the hood and learn how to build incredibly powerful, event driven systems with Phoenix Channels. We'll also look at how Phoenix makes it easier to track presence in a distributed system.

Transcript

1. Phoenix Channels A Distributed PubSub and Presence Platform @scrogson / https://github.com/scrogson
2. Sonny Scroggin @scrogson Phoenix Core Team Member since 2014
3. What is Phoenix?
4. Distributed Web Services Framework
5. Written in Elixir
6. Runs on the BEAM (Erlang VM)
7. Preemptive scheduler Functional programming language Immutable data Actor-based concurrency model
8. Back to Phoenix
9. What can I build with it? • Stateless (typical CRUD, REST APIs, etc). • Stateful (persistent connections, efficient, event-based) • Distributed Systems
10. Stateless Request / Response
11. Request Response
12. Controller request Model Domain (business logic) Presentation response View Model
13. Stateful Event / Message Driven Architecture
14. Channels
15. Client
16. Topics “room:lobby”
17. Topics “room:lobby” topic
18. Topics “room:lobby” topic subtopic
19. Channel Routing channel "room:*", RoomChannel
20. Server API
21. UserSocket defmodule Chat.Endpoint do use Phoenix.Endpoint, otp_app: :chat socket "/socket", Chat.UserSocket # Serve at "/" the given assets from "priv/static" directory plug Plug.Static, at: "/", from: :chat, only: ~w(css images js favicon.ico robots.txt) # Code reloading will only work if the :code_reloader key of # the :phoenix application is set to true in your config file. if code_reloading? do socket "/phoenix/live_reload/socket", Phoenix.LiveReloader.Socket
22. UserSocket defmodule Chat.UserSocket do use Phoenix.Socket channel "room:*", Chat.RoomChannel transport :websocket, Phoenix.Transports.WebSocket transport :longpoll, Phoenix.Transports.LongPoll def connect(_params, socket) do {:ok, socket} end def id(_socket), do: nil end
23. UserSocket defmodule Chat.UserSocket do use Phoenix.Socket channel "room:*", Chat.RoomChannel transport :websocket, Phoenix.Transports.WebSocket transport :longpoll, Phoenix.Transports.LongPoll def connect(_params, socket) do {:ok, socket} end def id(_socket), do: nil end
24. UserSocket defmodule Chat.UserSocket do use Phoenix.Socket channel "room:*", Chat.RoomChannel transport :websocket, Phoenix.Transports.WebSocket transport :longpoll, Phoenix.Transports.LongPoll def connect(_params, socket) do {:ok, socket} end def id(_socket), do: nil end
25. UserSocket defmodule Chat.UserSocket do use Phoenix.Socket channel "room:*", Chat.RoomChannel transport :websocket, Phoenix.Transports.WebSocket transport :longpoll, Phoenix.Transports.LongPoll def connect(_params, socket) do {:ok, socket} end def id(_socket), do: nil end
26. RoomChannel defmodule Chat.RoomChannel do use Phoenix.Channel def join("room:lobby", _msg, socket) do {:ok, socket} end def join("room:" <> subtopic, _msg, socket) do case RoomServer.member?(socket.assigns[:user]) do true -> {:ok, socket} false -> {:error, %{reason: "unauthorized"}} end end def handle_in("new:msg", msg, socket) do broadcast! socket, "new:msg", %{user: msg["user"], body: msg["body"]} {:noreply, socket} end end
27. RoomChannel defmodule Chat.RoomChannel do use Phoenix.Channel def join("room:lobby", _msg, socket) do {:ok, socket} end def join("room:" <> subtopic, _msg, socket) do case RoomServer.member?(socket.assigns[:user]) do true -> {:ok, socket} false -> {:error, %{reason: "unauthorized"}} end end def handle_in("new:msg", msg, socket) do broadcast! socket, "new:msg", %{user: msg["user"], body: msg["body"]} {:noreply, socket} end end
28. RoomChannel defmodule Chat.RoomChannel do use Phoenix.Channel def join("room:lobby", _msg, socket) do {:ok, socket} end def join("room:" <> subtopic, _msg, socket) do case RoomServer.member?(socket.assigns[:user]) do true -> {:ok, socket} false -> {:error, %{reason: "unauthorized"}} end end def handle_in("new:msg", msg, socket) do broadcast! socket, "new:msg", %{user: msg["user"], body: msg["body"]} {:noreply, socket} end end
29. RoomChannel defmodule Chat.RoomChannel do use Phoenix.Channel def join("room:lobby", _msg, socket) do {:ok, socket} end def join("room:" <> subtopic, _msg, socket) do case RoomServer.member?(socket.assigns[:user]) do true -> {:ok, socket} false -> {:error, %{reason: "unauthorized"}} end end def handle_in("new:msg", msg, socket) do broadcast! socket, "new:msg", %{user: msg["user"], body: msg["body"]} {:noreply, socket} end end
30. RoomChannel defmodule Chat.RoomChannel do use Phoenix.Channel def join("room:lobby", _msg, socket) do {:ok, socket} end def join("room:" <> subtopic, _msg, socket) do case RoomServer.member?(socket.assigns[:user]) do true -> {:ok, socket} false -> {:error, %{reason: "unauthorized"}} end end def handle_in("new:msg", msg, socket) do broadcast! socket, "new:msg", %{user: msg["user"], body: msg["body"]} {:noreply, socket} end end
31. RoomChannel defmodule Chat.RoomChannel do use Phoenix.Channel def join("room:lobby", _msg, socket) do {:ok, socket} end def join("room:" <> subtopic, _msg, socket) do case RoomServer.member?(socket.assigns[:user]) do true -> {:ok, socket} false -> {:error, %{reason: "unauthorized"}} end end def handle_in("new:msg", msg, socket) do broadcast! socket, "new:msg", %{user: msg["user"], body: msg["body"]} {:noreply, socket} end end
32. RoomChannel defmodule Chat.RoomChannel do use Phoenix.Channel def join("room:lobby", _msg, socket) do {:ok, socket} end def join("room:" <> subtopic, _msg, socket) do case RoomServer.member?(socket.assigns[:user]) do true -> {:ok, socket} false -> {:error, %{reason: "unauthorized"}} end end def handle_in("new:msg", msg, socket) do broadcast! socket, "new:msg", %{user: msg["user"], body: msg["body"]} {:noreply, socket} end end
33. Client API
34. Client API Warning: JavaScript may be hazardous to your health…
35. import {Socket} from "phoenix" const socket = new Socket("/socket", { params: { userToken: window.userToken } }) socket.connect() const chan = socket.channel("room:lobby", {}) chan.join().receive("ignore", () => console.log("auth error")) .receive("ok", () => console.log("join ok")) .after(10000, () => console.log("Connection interruption")) chan.onClose(e => console.log("channel closed", e)) chan.push("new:msg", {user: "scrogson", body: "Hallo!"}) chan.on("new:msg", msg => { $messages.append(this.messageTemplate(msg)) })
36. import {Socket} from "phoenix" const socket = new Socket("/socket", { params: { userToken: window.userToken } }) socket.connect() const chan = socket.channel("room:lobby", {}) chan.join().receive("ignore", () => console.log("auth error")) .receive("ok", () => console.log("join ok")) .after(10000, () => console.log("Connection interruption")) chan.onClose(e => console.log("channel closed", e)) chan.push("new:msg", {user: "scrogson", body: "Hallo!"}) chan.on("new:msg", msg => { $messages.append(this.messageTemplate(msg)) })
37. import {Socket} from "phoenix" const socket = new Socket("/socket", { params: { userToken: window.userToken } }) socket.connect() const chan = socket.channel("room:lobby", {}) chan.join().receive("ignore", () => console.log("auth error")) .receive("ok", () => console.log("join ok")) .after(10000, () => console.log("Connection interruption")) chan.onClose(e => console.log("channel closed", e)) chan.push("new:msg", {user: "scrogson", body: "Hallo!"}) chan.on("new:msg", msg => { $messages.append(this.messageTemplate(msg)) })
38. import {Socket} from "phoenix" const socket = new Socket("/socket", { params: { userToken: window.userToken } }) socket.connect() const chan = socket.channel("room:lobby", {}) chan.join().receive("ignore", () => console.log("auth error")) .receive("ok", () => console.log("join ok")) .after(10000, () => console.log("Connection interruption")) chan.onClose(e => console.log("channel closed", e)) chan.push("new:msg", {user: "scrogson", body: "Hallo!"}) chan.on("new:msg", msg => { $messages.append(this.messageTemplate(msg)) })
39. import {Socket} from "phoenix" const socket = new Socket("/socket", { params: { userToken: window.userToken } }) socket.connect() const chan = socket.channel("room:lobby", {}) chan.join().receive("ignore", () => console.log("auth error")) .receive("ok", () => console.log("join ok")) .after(10000, () => console.log("Connection interruption")) chan.onClose(e => console.log("channel closed", e)) chan.push("new:msg", {user: "scrogson", body: "Hallo!"}) chan.on("new:msg", msg => { $messages.append(this.messageTemplate(msg)) })
40. import {Socket} from "phoenix" const socket = new Socket("/socket", { params: { userToken: window.userToken } }) socket.connect() const chan = socket.channel("room:lobby", {}) chan.join().receive("ignore", () => console.log("auth error")) .receive("ok", () => console.log("join ok")) .after(10000, () => console.log("Connection interruption")) chan.onClose(e => console.log("channel closed", e)) chan.push("new:msg", {user: "scrogson", body: "Hallo!"}) chan.on("new:msg", msg => { $messages.append(this.messageTemplate(msg)) })
41. import {Socket} from "phoenix" const socket = new Socket("/socket", { params: { userToken: window.userToken } }) socket.connect() const chan = socket.channel("room:lobby", {}) chan.join().receive("ignore", () => console.log("auth error")) .receive("ok", () => console.log("join ok")) .after(10000, () => console.log("Connection interruption")) chan.onClose(e => console.log("channel closed", e)) chan.push("new:msg", {user: "scrogson", body: "Hallo!"}) chan.on("new:msg", msg => { $messages.append(this.messageTemplate(msg)) })
42. import {Socket} from "phoenix" const socket = new Socket("/socket", { params: { userToken: window.userToken } }) socket.connect() const chan = socket.channel("room:lobby", {}) chan.join().receive("ignore", () => console.log("auth error")) .receive("ok", () => console.log("join ok")) .after(10000, () => console.log("Connection interruption")) chan.onClose(e => console.log("channel closed", e)) chan.push("new:msg", {user: "scrogson", body: "Hallo!"}) chan.on("new:msg", msg => { $messages.append(this.messageTemplate(msg)) })
43. import {Socket} from "phoenix" const socket = new Socket("/socket", { params: { userToken: window.userToken } }) socket.connect() const chan = socket.channel("room:lobby", {}) chan.join().receive("ignore", () => console.log("auth error")) .receive("ok", () => console.log("join ok")) .after(10000, () => console.log("Connection interruption")) chan.onClose(e => console.log("channel closed", e)) chan.push("new:msg", {user: "scrogson", body: "Hallo!"}) chan.on("new:msg", msg => { $messages.append(this.messageTemplate(msg)) })
44. The Outside View F# C# iOS
45. F# C# iOS NODE01 NODE02 browser
46. The Inside View Client Server PubSub
47. The Inside View Client socket.connect() Server PubSub
48. The Inside View Client socket.connect() Server PubSub registers with pubsub server
49. The Inside View Client channel.join(“room:lobby”) Server PubSub
50. The Inside View Client Server PubSub spawns a channel process
51. The Inside View Client Server PubSub registers with pubsub under the “room:lobby” topic
52. The Inside View Client channel.join(“room:general”) Server PubSub
53. The Inside View Client Server channel.join(“room:elixir”) PubSub
54. The Inside View Client Server X PubSub
55. The Inside View Client Server PubSub
56. Client Server room:lobby channel user socket broadcast(“room:lobby”, “new_msg”, %{ user: “System”, body: “Hey” }) PubSub
57. Client Server PubSub
58. Client Server PubSub
59. Client Server PubSub
60. Presence
62. The back-story Chris McCord Creator of Phoenix José Valim Creator of Elixir
63. 2 3 1 Agent
64. NODE01 NODE02 JOE ROBERT MIKE CHRIS JOSÉ SONNY Agent Agent
65. NODE01 NODE02 JOE ROBERT MIKE CHRIS JOSÉ SONNY Agent Agent Presence.list(“room:lobby”) => [JOE, ROBERT, MIKE] Presence.list(“room:lobby”) => [CHRIS, JOSÉ, SONNY]
66. NODE01 NODE02 JOE ROBERT MIKE CHRIS JOSÉ SONNY Redis shared state access solution
67. NODE01 NODE02 JOE ROBERT MIKE CHRIS JOSÉ SONNY Redis Presence.list(“room:lobby”) => [JOE, ROBERT, MIKE, CHRIS, JOSÉ, SONNY] Presence.list(“room:lobby”) => [JOE, ROBERT, MIKE, CHRIS, JOSÉ, SONNY]
68. NODE01 NODE02 JOE ROBERT MIKE CHRIS JOSÉ SONNY Redis
69. NODE01 NODE02 JOE ROBERT MIKE CHRIS JOSÉ SONNY Redis Presence.list(“room:lobby”) => [JOE, ROBERT, MIKE, CHRIS, JOSÉ, SONNY] orphaned data
70. JOE ROBERT MIKE NODE01 NODE02 NODE03 PRODUCTION
71. JOE ROBERT MIKE NODE01 NODE02 NODE03 PRODUCTION
72. SONNY CHRIS JOSÉ JOE ROBERT MIKE NODE01 NODE02 NODE03 PRODUCTION
73. CAP
74. CAP Consistency Availability Partition-Tolerance
75. CAP Consistency Availability Partition-Tolerance
76. CAP Consistency Availability Partition-Tolerance
77. CAP Consistency Availability Partition-Tolerance Choose Two
78. CAP Consistency Availability Partition-Tolerance But you have to choose partition-tolerance
79. CAP Consistency Availability Partition-Tolerance
80. CAP Consistency Availability Partition-Tolerance
81. Conflict-Free Replicated Data Types
82. CRDTs are used to replicate data across multiple computers in a network, executing updates without the need for remote synchronization.
83. CRDTs • Strong eventual consistency • Conflicts are mathematically impossible • Replication without remote synchronization
84. Delta State Conflict-Free Replicated Datatypes
85. NODE01 NODE02 NODE03 JOE ROBERT MIKE [JOE] [JOE] [ROBERT] [ROBERT]
86. NODE01 NODE02 NODE03 JOE ROBERT ROBERT MIKE JOE ROBERT [MIKE] [MIKE] [JOE]
87. NODE01 NODE02 NODE03 JOE ROBERT MIKE ROBERT MIKE ??? MIKE JOE ROBERT [JOE] [JOE] [JOE] should node02 add or discard Joe? slow message
88. Time Doesn't Exist in Distributed Systems
89. Vector Clocks
90. {:n1, 0}
91. {:n1, 0} starts off at zero
92. {:n1, 0} {:n1, 1} an internal event occurred
93. {:n1, 0} {:n1, 1} an internal event occurred increments the clock
94. {:n1, 0} {:n1, 1} {:n1, 2} another internal event occurred
95. {:n1, 0} {:n1, 1} {:n1, 2} another internal event occurred increments the clock
96. NODE02 NODE01 NODE03 {1, 0, 0} {1, 1, 0} {1, 2, 3} {2, 2, 3} {0, 0, 1} {0, 0, 2} {0, 0, 3} A B D E C F G
97. Server API
98. defmodule RoomChannel do use Phoenix.Channel def join("room:" <> id, _, socket) do send self(), :after_join {:ok, socket} end def handle_info(:after_join, socket) do id = socket.assigns.user_id push socket, "presence_state", Presence.list(socket) Presence.track(socket, id, %{status: "available"}) {:noreply, socket} end end
99. defmodule RoomChannel do use Phoenix.Channel def join(“room:” <> id, _, socket) do send self(), :after_join {:ok, socket} end def handle_info(:after_join, socket) do id = socket.assigns.user_id push socket, "presence_state", Presence.list(socket) Presence.track(socket, id, %{status: “available"}) {:noreply, socket} end end
100. Client API
101. import {Socket, Presence} from "phoenix" const socket = new Socket("/socket") socket.connect() const room = socket.channel(`room:${roomId}`) let presenceState = {} room.on("presence_state", state => { Presence.syncState(presenceState, state) }) room.on("presence_diff", diff => { Presence.syncDiff(presenceState, diff) }) console.log("users", Presence.list(presenceState))
102. import {Socket, Presence} from "phoenix" const socket = new Socket("/socket") socket.connect() const room = socket.channel(`room:${roomId}`) let presenceState = {} room.on("presence_state", state => { Presence.syncState(presenceState, state) }) room.on("presence_diff", diff => { Presence.syncDiff(presenceState, diff) }) console.log("users", Presence.list(presenceState))
103. import {Socket, Presence} from "phoenix" const socket = new Socket("/socket") socket.connect() const room = socket.channel(`room:${roomId}`) let presenceState = {} room.on("presence_state", state => { Presence.syncState(presenceState, state) }) room.on("presence_diff", diff => { Presence.syncDiff(presenceState, diff) }) console.log("users", Presence.list(presenceState))
104. import {Socket, Presence} from "phoenix" const socket = new Socket("/socket") socket.connect() const room = socket.channel(`room:${roomId}`) let presenceState = {} room.on("presence_state", state => { Presence.syncState(presenceState, state) }) room.on("presence_diff", diff => { Presence.syncDiff(presenceState, diff) }) console.log("users", Presence.list(presenceState))
105. import {Socket, Presence} from "phoenix" const socket = new Socket("/socket") socket.connect() const room = socket.channel(`room:${roomId}`) let presenceState = {} room.on("presence_state", state => { Presence.syncState(presenceState, state) }) room.on("presence_diff", diff => { Presence.syncDiff(presenceState, diff) }) console.log("users", Presence.list(presenceState))
106. import {Socket, Presence} from "phoenix" const socket = new Socket("/socket") socket.connect() const room = socket.channel(`room:${roomId}`) let presenceState = {} room.on("presence_state", state => { Presence.syncState(presenceState, state) }) room.on("presence_diff", diff => { Presence.syncDiff(presenceState, diff) }) console.log("users", Presence.list(presenceState))
107. import {Socket, Presence} from "phoenix" const socket = new Socket("/socket") socket.connect() const room = socket.channel(`room:${roomId}`) let presenceState = {} room.on("presence_state", state => { Presence.syncState(presenceState, state) }) room.on("presence_diff", diff => { Presence.syncDiff(presenceState, diff) }) console.log("users", Presence.list(presenceState))
108. Demo
109. But wait, there’s more!
110. Service Discovery
111. NODE01 NODE02 NODE03 EMAIL(2) EMAIL(2) EMAIL(2) [EMAIL] [EMAIL] [EMAIL] [EMAIL]
112. Presence.track("services", “email", %{ pid: self(), max_jobs: 100, work_load: 0 }) Presence.list("services") => %{”email” => [...]}
113. # pseudo code Service.track(pid, “email", %{ pid: pid, max_jobs: 100, work_load: 0 }) Service.all => %{“email” => [...]} Service.list("email") => [%{pid: pid, max_jobs: 100, work_load: 50}, ...)]
114. Check out the guides! http://phoenixframework.org
115. Thank You!