$30 off During Our Annual Pro Sale. View Details »

Building a Realtime WebSocket API (BEAM Edition)

Building a Realtime WebSocket API (BEAM Edition)

Sometimes consumers of your APIs require near-realtime communication because regular RESTful HTTP apis can be a few milliseconds too slow. These performant and scalable APIs can be made over websocket TCP connections where events are pushed from client and server in near-realtime fashion.

This talk is a story of how I built such an API. We'll look at why this decision to create a websocket API was made and we will take a look at the data that supported this decision. We will take a deep dive into Phoenix websockets, channels, and transports to expose the underlying architecture. Finally, we look at how we tested the API, how we authenticated users over the channels, and how Phoenix helped this all happen with relative ease.

Jamie Wright
PRO

March 05, 2020
Tweet

More Decks by Jamie Wright

Other Decks in Programming

Transcript

  1. Websocket
    API
    CodeBEAM SF
    2020
    Building
    a real time
    ⋆ ⋆
    in Phoenix

    View Slide

  2. @jwright
    #CodeBEAMSF

    View Slide

  3. jamie
    @brilliantfantastic.com
    #CodeBEAMSF

    View Slide

  4. @jwright
    http://speakerdeck.com/jwright

    View Slide

  5. Background
    Some
    #CodeBEAMSF

    View Slide

  6. Background
    Some
    #CodeBEAMSF

    View Slide

  7. Background
    Some
    #CodeBEAMSF

    View Slide

  8. Background
    Some
    http://chronic.io

    View Slide

  9. Background
    Some

    View Slide

  10. Clients
    Multiple

    View Slide

  11. Source
    One
    #CodeBEAMSF

    View Slide

  12. Server is gold copy
    Source
    One
    #CodeBEAMSF

    View Slide

  13. Server is gold copy
    Server pushes events to
    clients
    Source
    One
    #CodeBEAMSF

    View Slide

  14. Server is gold copy
    Server pushes events to
    clients
    Clients displays event
    Source
    One
    #CodeBEAMSF

    View Slide

  15. Tick
    Events at any
    second
    #CodeBEAMSF

    View Slide

  16. Tick
    { remaining_time: 1497 }
    { remaining_time: 1497 }
    #CodeBEAMSF

    View Slide

  17. Plugins
    Events at any
    second
    #CodeBEAMSF

    View Slide

  18. Plugins
    Events at any
    second

    #CodeBEAMSF

    View Slide

  19. Timers
    Team
    { timers: { id: 724 } }

    View Slide

  20. Timers
    Team
    Suzie’s iMac Leon’s MacBook
    Sandra’s iPhone Sandra’s Slack App
    { timers: { id: 724 } }

    View Slide

  21. Background
    Some
    Demo
    #CodeBEAMSF

    View Slide

  22. View Slide

  23. View Slide

  24. Websockets
    #CodeBEAMSF

    View Slide

  25. Websocket
    What’s a
    #CodeBEAMSF

    View Slide

  26. Websocket
    What’s a
    #CodeBEAMSF

    View Slide

  27. Websocket
    What’s a
    Protocol
    #CodeBEAMSF

    View Slide

  28. Websocket
    #CodeBEAMSF

    View Slide

  29. Websocket
    Full-duplex
    connection
    #CodeBEAMSF

    View Slide

  30. Websocket
    Full-duplex
    connection
    Both sides can
    chat at same
    time
    #CodeBEAMSF

    View Slide

  31. Http
    #CodeBEAMSF

    View Slide

  32. Http
    Simplex
    connection
    #CodeBEAMSF

    View Slide

  33. Http
    Simplex
    connection
    One side can
    chat at a time
    #CodeBEAMSF

    View Slide

  34. Websocket
    What’s a
    https://tools.ietf.org/html/rfc6455

    View Slide

  35. Websocket
    What’s a
    https://tools.ietf.org/html/rfc6455
    RFC

    View Slide

  36. Websocket
    What’s a
    https://www.w3.org/TR/websockets/

    View Slide

  37. Websocket
    What’s a
    https://www.w3.org/TR/websockets/
    W3C Spec

    View Slide

  38. Websocket
    What’s a

    View Slide

  39. Websocket
    What’s a

    View Slide

  40. Websocket
    What’s a

    View Slide

  41. Websocket
    What’s a

    View Slide

  42. Websocket
    What’s a
    #CodeBEAMSF

    View Slide

  43. Websocket
    What’s a
    Starts with
    a
    Handshakes
    via HTTP
    #CodeBEAMSF

    View Slide

  44. #CodeBEAMSF

    View Slide

  45. #CodeBEAMSF

    View Slide

  46. #CodeBEAMSF

    View Slide

  47. #CodeBEAMSF

    View Slide

  48. View Slide

  49. TCP (C’est une pipe)

    View Slide

  50. TCP (C’est une pipe)
    HTTP

    View Slide

  51. TCP (C’est une pipe)
    HTTP
    W
    ebSocket

    View Slide

  52. Phoenix
    Why

    View Slide

  53. Phoenix
    Why
    #CodeBEAMSF

    View Slide

  54. Phoenix
    Why
    WebSockets
    #CodeBEAMSF

    View Slide

  55. Phoenix
    Why
    #CodeBEAMSF

    View Slide

  56. Phoenix
    Why
    Elixir
    #CodeBEAMSF

    View Slide

  57. Phoenix
    Why
    #CodeBEAMSF

    View Slide

  58. Fast
    Phoenix
    Why
    #CodeBEAMSF

    View Slide

  59. Phoenix
    Why
    #CodeBEAMSF

    View Slide

  60. Phoenix
    Why
    Stateful
    #CodeBEAMSF

    View Slide

  61. Phoenix
    Why
    #CodeBEAMSF

    View Slide

  62. Phoenix
    Why
    Hot Code
    Reloading
    #CodeBEAMSF

    View Slide

  63. Example APIs
    #CodeBEAMSF

    View Slide

  64. Example APIs
    https://api.slack.com/rtm

    View Slide

  65. Example APIs
    https://api.slack.com/rtm
    Send messages in channel

    View Slide

  66. Example APIs
    https://api.slack.com/rtm
    Send messages in channel
    Receive team events

    View Slide

  67. Example APIs
    https://api.slack.com/rtm
    Send messages in channel
    Receive team events
    Receive user presence

    View Slide

  68. Example APIs
    https://getstream.io/docs/#realtime

    View Slide

  69. Example APIs
    https://getstream.io/docs/#realtime
    Receive feed changes

    View Slide

  70. Works
    How it
    #CodeBEAMSF

    View Slide

  71. Authenticating

    View Slide

  72. Authenticating

    View Slide

  73. Authenticating

    View Slide

  74. Authenticating

    View Slide

  75. Authenticating

    View Slide

  76. Connecting
    #CodeBEAMSF

    View Slide

  77. Connecting

    View Slide

  78. Connecting

    View Slide

  79. Connecting

    View Slide

  80. Connecting

    View Slide

  81. Connecting
    #CodeBEAMSF

    View Slide

  82. Connecting
    Give clients a starting point
    #CodeBEAMSF

    View Slide

  83. Connecting
    Give clients a starting point
    Identify events from a
    single endpoint (client)
    #CodeBEAMSF

    View Slide

  84. Connecting
    Give clients a starting point
    Identify events from a
    single endpoint (client)
    Opportunity to
    authenticate
    #CodeBEAMSF

    View Slide

  85. View Slide

  86. View Slide

  87. View Slide

  88. Connecting

    View Slide

  89. Connecting

    View Slide

  90. Connecting

    View Slide

  91. Connecting

    View Slide

  92. Connecting

    View Slide

  93. Connecting

    View Slide

  94. Connecting

    View Slide

  95. Connecting

    View Slide

  96. Connecting

    View Slide

  97. Connecting

    View Slide

  98. Connecting

    View Slide

  99. Channels
    Joining
    #CodeBEAMSF

    View Slide

  100. Channels
    Joining
    #CodeBEAMSF

    View Slide

  101. Channels
    Joining
    Custom
    Protocol
    #CodeBEAMSF

    View Slide

  102. Channels
    Joining
    #CodeBEAMSF

    View Slide

  103. Channels
    Joining
    Multiplexed
    #CodeBEAMSF

    View Slide

  104. Channels
    Joining
    #CodeBEAMSF

    View Slide

  105. Channels
    Joining
    PubSub
    #CodeBEAMSF

    View Slide

  106. Channels
    Joining
    #CodeBEAMSF

    View Slide

  107. Channels
    Joining
    Built In
    #CodeBEAMSF

    View Slide

  108. Channels
    The Problem with
    #CodeBEAMSF

    View Slide

  109. Problems
    WS
    Joining Channels
    #CodeBEAMSF

    View Slide

  110. Problems
    { topic: “events:1234”, event: “phx_join” }
    WS
    WS
    Joining Channels
    #CodeBEAMSF

    View Slide

  111. Problems
    { topic: “events:1234”, event: “phx_join” }
    { topic: “events:1234”, event: “phx_reply”,
    payload: { ok: true, token: “1234” }}}
    WS
    WS
    WS
    WS
    Joining Channels
    #CodeBEAMSF

    View Slide

  112. Problems
    Joining Channels
    #CodeBEAMSF

    View Slide

  113. Problems
    Joining Channels
    Remove complicated
    setup from clients
    #CodeBEAMSF

    View Slide

  114. Problems
    Joining Channels
    Remove complicated
    setup from clients
    Build reconnection/
    heartbeat logic
    #CodeBEAMSF

    View Slide

  115. Problems
    Joining Channels
    Remove complicated
    setup from clients
    Build reconnection/
    heartbeat logic
    :cowboy_websocket
    https://bit.ly/2WjtKxZ
    #CodeBEAMSF

    View Slide

  116. Messages
    #CodeBEAMSF

    View Slide

  117. Messages
    #CodeBEAMSF

    View Slide

  118. Messages
    Meat
    #CodeBEAMSF

    View Slide

  119. Messages
    #CodeBEAMSF

    View Slide

  120. Messages
    Event name
    #CodeBEAMSF

    View Slide

  121. Messages
    Event name
    Event data
    #CodeBEAMSF

    View Slide

  122. Messages
    Incoming
    #CodeBEAMSF

    View Slide

  123. Messages
    Incoming
    #CodeBEAMSF

    View Slide

  124. Messages
    Incoming
    #CodeBEAMSF

    View Slide

  125. Messages
    Incoming

    View Slide

  126. Messages
    Incoming

    View Slide

  127. Messages
    Incoming

    View Slide

  128. Messages
    Incoming

    View Slide

  129. Messages
    Incoming

    View Slide

  130. Messages
    Incoming

    View Slide

  131. Messages
    Incoming

    View Slide

  132. Authenticating

    View Slide

  133. Authenticating

    View Slide

  134. Authenticating

    View Slide

  135. Authenticating

    View Slide

  136. Authenticating

    View Slide

  137. Authenticating

    View Slide

  138. Authenticating

    View Slide

  139. Messages
    Incoming
    #CodeBEAMSF

    View Slide

  140. Messages
    Incoming
    #CodeBEAMSF

    View Slide

  141. Messages
    Outgoing
    #CodeBEAMSF

    View Slide

  142. Messages
    Outgoing

    View Slide

  143. Messages
    Outgoing

    View Slide

  144. Messages
    Outgoing

    View Slide

  145. Messages
    Outgoing

    View Slide

  146. Messages
    Outgoing

    View Slide

  147. Outgoing

    View Slide

  148. Outgoing

    View Slide

  149. Outgoing

    View Slide

  150. View Slide

  151. View Slide

  152. View Slide

  153. Responses
    Creating
    #CodeBEAMSF

    View Slide

  154. Responses
    Creating

    View Slide

  155. Responses
    Creating

    View Slide

  156. Responses
    Creating

    View Slide

  157. Responses
    Creating

    View Slide

  158. Responses
    Creating

    View Slide

  159. Responses
    Creating

    View Slide

  160. without Channels
    Dispatching
    #CodeBEAMSF

    View Slide

  161. without Channels
    Dispatching

    View Slide

  162. without Channels
    Dispatching

    View Slide

  163. without Channels
    Dispatching
    #CodeBEAMSF

    View Slide

  164. Review
    #CodeBEAMSF

    View Slide

  165. Review
    HTTP
    Connecting
    #CodeBEAMSF

    View Slide

  166. Review
    POST /graphql
    HTTP
    HTTP
    Connecting
    { mutation: { connect: { … } }
    #CodeBEAMSF

    View Slide

  167. Review
    POST /graphql
    { url: “ws://…”, token: “1234” }
    HTTP
    HTTP
    HTTP
    HTTP
    Connecting
    { mutation: { connect: { … } }
    #CodeBEAMSF

    View Slide

  168. Review
    WS
    Sending Messages
    #CodeBEAMSF

    View Slide

  169. Review
    [“timers:start”, { duration: 1500,
    timestamp: 1273485 } ]
    WS
    WS
    Sending Messages
    #CodeBEAMSF

    View Slide

  170. Review
    [“timers:start”, { duration: 1500,
    timestamp: 1273485 } ]
    [“timers:started”, { id: 1234, duration: 1500,
    remaining_time: 1500, status: “running” } ]
    WS
    WS
    WS
    WS
    Sending Messages
    #CodeBEAMSF

    View Slide

  171. Thank You!
    #CodeBEAMSF

    View Slide

  172. Questions?
    #CodeBEAMSF

    View Slide

  173. @jwright
    #CodeBEAMSF

    View Slide