Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a Realtime WebSocket API (BEAM Edition)

Building a Realtime WebSocket API (BEAM Edition)

Sometimes consumers of your APIs require near-realtime communication because regular RESTful HTTP apis can be a few milliseconds too slow. These performant and scalable APIs can be made over websocket TCP connections where events are pushed from client and server in near-realtime fashion.

This talk is a story of how I built such an API. We'll look at why this decision to create a websocket API was made and we will take a look at the data that supported this decision. We will take a deep dive into Phoenix websockets, channels, and transports to expose the underlying architecture. Finally, we look at how we tested the API, how we authenticated users over the channels, and how Phoenix helped this all happen with relative ease.

94d3675b2e2431b326b0de7aff01b742?s=128

Jamie Wright
PRO

March 05, 2020
Tweet

More Decks by Jamie Wright

Other Decks in Programming

Transcript

  1. Websocket API CodeBEAM SF 2020 Building a real time ⋆

    ⋆ in Phoenix
  2. @jwright #CodeBEAMSF

  3. jamie @brilliantfantastic.com #CodeBEAMSF

  4. @jwright http://speakerdeck.com/jwright

  5. Background Some #CodeBEAMSF

  6. Background Some #CodeBEAMSF

  7. Background Some #CodeBEAMSF

  8. Background Some http://chronic.io

  9. Background Some

  10. Clients Multiple

  11. Source One #CodeBEAMSF

  12. Server is gold copy Source One #CodeBEAMSF

  13. Server is gold copy Server pushes events to clients Source

    One #CodeBEAMSF
  14. Server is gold copy Server pushes events to clients Clients

    displays event Source One #CodeBEAMSF
  15. Tick Events at any second #CodeBEAMSF

  16. Tick { remaining_time: 1497 } { remaining_time: 1497 } #CodeBEAMSF

  17. Plugins Events at any second #CodeBEAMSF

  18. Plugins Events at any second  #CodeBEAMSF

  19. Timers Team { timers: { id: 724 } }

  20. Timers Team Suzie’s iMac Leon’s MacBook Sandra’s iPhone Sandra’s Slack

    App { timers: { id: 724 } }
  21. Background Some Demo #CodeBEAMSF

  22. None
  23. None
  24. Websockets #CodeBEAMSF

  25. Websocket What’s a #CodeBEAMSF

  26. Websocket What’s a #CodeBEAMSF

  27. Websocket What’s a Protocol #CodeBEAMSF

  28. Websocket #CodeBEAMSF

  29. Websocket Full-duplex connection #CodeBEAMSF

  30. Websocket Full-duplex connection Both sides can chat at same time

    #CodeBEAMSF
  31. Http #CodeBEAMSF

  32. Http Simplex connection #CodeBEAMSF

  33. Http Simplex connection One side can chat at a time

    #CodeBEAMSF
  34. Websocket What’s a https://tools.ietf.org/html/rfc6455

  35. Websocket What’s a https://tools.ietf.org/html/rfc6455 RFC

  36. Websocket What’s a https://www.w3.org/TR/websockets/

  37. Websocket What’s a https://www.w3.org/TR/websockets/ W3C Spec

  38. Websocket What’s a

  39. Websocket What’s a

  40. Websocket What’s a

  41. Websocket What’s a

  42. Websocket What’s a #CodeBEAMSF

  43. Websocket What’s a Starts with a Handshakes via HTTP #CodeBEAMSF

  44. #CodeBEAMSF

  45. #CodeBEAMSF

  46. #CodeBEAMSF

  47. #CodeBEAMSF

  48. None
  49. TCP (C’est une pipe)

  50. TCP (C’est une pipe) HTTP

  51. TCP (C’est une pipe) HTTP W ebSocket

  52. Phoenix Why

  53. Phoenix Why #CodeBEAMSF

  54. Phoenix Why WebSockets #CodeBEAMSF

  55. Phoenix Why #CodeBEAMSF

  56. Phoenix Why Elixir #CodeBEAMSF

  57. Phoenix Why #CodeBEAMSF

  58. Fast Phoenix Why #CodeBEAMSF

  59. Phoenix Why #CodeBEAMSF

  60. Phoenix Why Stateful #CodeBEAMSF

  61. Phoenix Why #CodeBEAMSF

  62. Phoenix Why Hot Code Reloading #CodeBEAMSF

  63. Example APIs #CodeBEAMSF

  64. Example APIs https://api.slack.com/rtm

  65. Example APIs https://api.slack.com/rtm Send messages in channel

  66. Example APIs https://api.slack.com/rtm Send messages in channel Receive team events

  67. Example APIs https://api.slack.com/rtm Send messages in channel Receive team events

    Receive user presence
  68. Example APIs https://getstream.io/docs/#realtime

  69. Example APIs https://getstream.io/docs/#realtime Receive feed changes

  70. Works How it #CodeBEAMSF

  71. Authenticating

  72. Authenticating

  73. Authenticating

  74. Authenticating

  75. Authenticating

  76. Connecting #CodeBEAMSF

  77. Connecting

  78. Connecting

  79. Connecting

  80. Connecting

  81. Connecting #CodeBEAMSF

  82. Connecting Give clients a starting point #CodeBEAMSF

  83. Connecting Give clients a starting point Identify events from a

    single endpoint (client) #CodeBEAMSF
  84. Connecting Give clients a starting point Identify events from a

    single endpoint (client) Opportunity to authenticate #CodeBEAMSF
  85. None
  86. None
  87. None
  88. Connecting

  89. Connecting

  90. Connecting

  91. Connecting

  92. Connecting

  93. Connecting

  94. Connecting

  95. Connecting

  96. Connecting

  97. Connecting

  98. Connecting

  99. Channels Joining #CodeBEAMSF

  100. Channels Joining #CodeBEAMSF

  101. Channels Joining Custom Protocol #CodeBEAMSF

  102. Channels Joining #CodeBEAMSF

  103. Channels Joining Multiplexed #CodeBEAMSF

  104. Channels Joining #CodeBEAMSF

  105. Channels Joining PubSub #CodeBEAMSF

  106. Channels Joining #CodeBEAMSF

  107. Channels Joining Built In #CodeBEAMSF

  108. Channels The Problem with #CodeBEAMSF

  109. Problems WS Joining Channels #CodeBEAMSF

  110. Problems { topic: “events:1234”, event: “phx_join” } WS WS Joining

    Channels #CodeBEAMSF
  111. Problems { topic: “events:1234”, event: “phx_join” } { topic: “events:1234”,

    event: “phx_reply”, payload: { ok: true, token: “1234” }}} WS WS WS WS Joining Channels #CodeBEAMSF
  112. Problems Joining Channels #CodeBEAMSF

  113. Problems Joining Channels Remove complicated setup from clients #CodeBEAMSF

  114. Problems Joining Channels Remove complicated setup from clients Build reconnection/

    heartbeat logic #CodeBEAMSF
  115. Problems Joining Channels Remove complicated setup from clients Build reconnection/

    heartbeat logic :cowboy_websocket https://bit.ly/2WjtKxZ #CodeBEAMSF
  116. Messages #CodeBEAMSF

  117. Messages #CodeBEAMSF

  118. Messages Meat #CodeBEAMSF

  119. Messages #CodeBEAMSF

  120. Messages Event name #CodeBEAMSF

  121. Messages Event name Event data #CodeBEAMSF

  122. Messages Incoming #CodeBEAMSF

  123. Messages Incoming #CodeBEAMSF

  124. Messages Incoming #CodeBEAMSF

  125. Messages Incoming

  126. Messages Incoming

  127. Messages Incoming

  128. Messages Incoming

  129. Messages Incoming

  130. Messages Incoming

  131. Messages Incoming

  132. Authenticating

  133. Authenticating

  134. Authenticating

  135. Authenticating

  136. Authenticating

  137. Authenticating

  138. Authenticating

  139. Messages Incoming #CodeBEAMSF

  140. Messages Incoming #CodeBEAMSF

  141. Messages Outgoing #CodeBEAMSF

  142. Messages Outgoing

  143. Messages Outgoing

  144. Messages Outgoing

  145. Messages Outgoing

  146. Messages Outgoing

  147. Outgoing

  148. Outgoing

  149. Outgoing

  150. None
  151. None
  152. None
  153. Responses Creating #CodeBEAMSF

  154. Responses Creating

  155. Responses Creating

  156. Responses Creating

  157. Responses Creating

  158. Responses Creating

  159. Responses Creating

  160. without Channels Dispatching #CodeBEAMSF

  161. without Channels Dispatching

  162. without Channels Dispatching

  163. without Channels Dispatching #CodeBEAMSF

  164. Review #CodeBEAMSF

  165. Review HTTP Connecting #CodeBEAMSF

  166. Review POST /graphql HTTP HTTP Connecting { mutation: { connect:

    { … } } #CodeBEAMSF
  167. Review POST /graphql { url: “ws://…”, token: “1234” } HTTP

    HTTP HTTP HTTP Connecting { mutation: { connect: { … } } #CodeBEAMSF
  168. Review WS Sending Messages #CodeBEAMSF

  169. Review [“timers:start”, { duration: 1500, timestamp: 1273485 } ] WS

    WS Sending Messages #CodeBEAMSF
  170. Review [“timers:start”, { duration: 1500, timestamp: 1273485 } ] [“timers:started”,

    { id: 1234, duration: 1500, remaining_time: 1500, status: “running” } ] WS WS WS WS Sending Messages #CodeBEAMSF
  171. Thank You! #CodeBEAMSF

  172. Questions? #CodeBEAMSF

  173. @jwright #CodeBEAMSF