Building Bulletproof Real-Time Apps with Phoenix LiveView + Stream Data

Building Bulletproof Real-Time Apps with Phoenix LiveView + Stream Data

Elixir School's Day 2 workshop at ElixirConf 2019

D8ca232edbd039a0cf5409400814dd49?s=128

Sophie DeBenedetto

August 28, 2019
Tweet

Transcript

  1. 3.

    Hello! I’m Sophie I’m an engineer and sometimes teacher at

    The Flatiron School where I build education and business tools in Elixir and Ruby. @sm_debenedetto 3 *and this is my dog
  2. 4.

    And... I’m Michael I’m a lead engineer at RentPath, where

    we build websites to help people find a place to live. @michaelstalker 4 *and this is my family
  3. 5.

    Elixir School A free, open-source Elixir curriculum Through the hard

    work of many volunteers around the world, we’ve developed and translated content covering everything from intro topics to deep dives. 5
  4. 6.

    Get Involved! Help us grow the Elixir School community Write

    a lesson, contribute a TIL (or longer!) blog post or provide a translation. Open an issue here describing your idea or simply open a PR with your work to get started. 6
  5. 7.
  6. 8.

    What We’ll Learn ◦ Phoenix LiveView ◦ Phoenix PubSub ◦

    Phoenix Presence ◦ Property-based Testing with StreamData 8
  7. 9.

    What We’ll Build A real-time ticket estimation tool ◦ Users

    can see other users in the “estimation room” in real-time ◦ Users can vote on tickets ◦ Users can see the winning vote tallied in real-time and move on to the next ticket 9
  8. 11.

    11

  9. 12.

    User Starts Estimation Round When I click “start pointing!” Then

    the first card for estimation appears for all users in the room Features Real-Time User Presence When a new user joins/leaves Then I see the list of present users update in the UI 12
  10. 13.

    Ticket is Estimated When another user submits a vote Then

    I see the UI update to indicate they have voted Features Winner is Calculated When the last user submits a vote Then all users see the winning vote And a user can click “next card” 13
  11. 14.

    Application Starting State Log In with a username (We built

    some dummy authorization flows) See the static ticket estimation page Click the start button and... Oh no you can’t estimate a ticket! 14
  12. 15.

    We Need Real-Time! In the current state of our app,

    users cannot collaborate on ticket estimation. 15
  13. 16.

    What is the Real-Time Web? Users receive new information from

    the server as soon as it is available—no request required This represents a significant departure from traditional HTTP communication Made possible by WebSockets 16
  14. 18.

    What are WebSockets? A Protocol built on top of TCP

    that allows for bi-directional, "full-duplex" communication between the client and the server by creating a persistent connection between the two. 18
  15. 20.

    20 Server Client Step 1. Request to initiate WS connection

    Client Client GET / Host: elixirschool.com Upgrade: websocket Connection: upgrade
  16. 22.

    22 We’ll use WebSockets to maintain a persistent, bi-directional and

    stateful connection between our ticket estimation clients and the server with the help of Phoenix LiveView
  17. 24.

    What is LiveView? Phoenix LiveView leverages server-rendered HTML and Phoenix’s

    native WebSocket tooling so you can build fancy real-time features without all that complicated JavaScript. 24
  18. 29.

    Connecting • Client connects to Phoenix server over WebSockets, at

    the LiveView socket endpoint • Phoenix server starts a LiveView process for that user’s socket 29
  19. 30.

    Rendering the Template • The LiveView leex (Live EEX) template

    is rendered, using the data from the LiveView process’ state 30
  20. 31.

    Handling Changes • Certain user interactions on the page will

    send a message over the socket to the LiveView process • The LiveView process will update its state and push a message back down the socket • The leex template will re-render with the new state 31
  21. 37.
  22. 42.

    42 # lib/pointing_party_web.ex def view do quote do ... import

    Phoenix.LiveView, only: [live_render: 2, live_render: 3] end end
  23. 50.
  24. 51.

    Connection Process • User visits ”/cards” in the browser •

    CardController mounts the LiveView • The LiveView renders the LEEX template as static HTML 51
  25. 52.

    Connection Process • User visits ”/cards” in the browser •

    CardController mounts the LiveView • The LiveView renders the LEEX template as static HTML • A JS snippet on that static HTML page sends the “WebSocket connect” request 52
  26. 53.

    Connection Process • User visits ”/cards” in the browser •

    CardController mounts the LiveView • The LiveView renders the LEEX template as static HTML • A JS snippet on that static HTML page sends the “WebSocket connect” request • The LiveView process re-renders the template over WebSockets, is now listening for events from the client 53
  27. 54.

    54 LiveView Client Request GET /cards Render LEEX Template <h1>Start

    pointing!</h1> Step 1. Mount the LiveView + render static HTML CardController Mount LiveView
  28. 56.

    56 Client Connect to LiveView socket Step 3. Re-render the

    template LiveView Re-render template over WS connection LV Socket
  29. 57.

    We Will • Mount the LiveView socket endpoint • Define

    a LiveView and LEEX template • Teach our controller to render the LiveView 57
  30. 61.

    61 # app/pointing_party_web/live/card_live.ex defmodule PointingPartyWeb.CardLive do use Phoenix.LiveView alias PointingPartyWeb.CardView

    def render(assigns) do Phoenix.View.render(CardView, "index.html", assigns) end def mount(_session, socket) do {:ok, assign(socket, is_pointing: false)} end end
  31. 63.

    63 <div> <%= if !@is_pointing do %> <div> <button> Start

    the Party! </button> </div> <% else %> <!-- card display here --> <% end %> </div>
  32. 67.

    67 defmodule PointingPartyWeb.CardController do use PointingPartyWeb, :controller import Phoenix.LiveView.Controller def

    index(conn, _params) do %{assigns: %{username: username}} = conn live_render(conn, PointingPartyWeb.CardLive, session: %{username: username}) end end
  33. 76.

    76 <div> <%= if !@is_pointing do %> <div> <button> Start

    the Party! </button> </div> <% else %> <!-- card display here --> <% end %> </div>
  34. 77.

    We Will • Add a DOM element binding that will

    send an event from the client to LiveView 77
  35. 78.

    We Will • Add a DOM element binding that will

    send an event from the client to LiveView • Teach our LiveView how to handle this event 78
  36. 84.

    84 def handle_event("start_party", _value, socket) do [current_card | remaining_cards] =

    Card.cards() socket = assign(socket, is_pointing: true, current_card: current_card, remaining_cards: remaining_cards) {:noreply, socket} end
  37. 86.

    86 <div> <%= if !@is_pointing do %> <div> <button phx-click="start_party">

    Start the Party! </button> </div> <% else %> <!-- card display here --> <% end %> </div>
  38. 87.

    87 <div> <h2><%= @current_card.title %></h2> <p><%= @current_card.description %></p> <form phx-submit="vote">

    <label for="story-points">Story Points</label> <select name="points"> <option value="0">0</option> <option value="1">1</option> <option value="3">3</option> <option value="5">5</option> </select> </form> </div>
  39. 90.

    Feature Roadmap 2. Users see list of present users 3.

    All votes are tallied, displayed 1. User initiates pointing round 90
  40. 91.

    91

  41. 92.

    User initiates pointing round 92 Define the LiveView • Define

    a LiveView and LEEX template - We did this for you :) • Render the LiveView from the controller - We did this for you :) Handle Events • Add a phx-click event to the “start pointing” button • Define a handle_event/3 function to match this event and update socket.assigns accordingly Configure LiveView • We did this for you :)
  42. 95.

    95 The pointing round is only started for the single

    user who clicked “start pointing”
  43. 99.

    What is PubSub? PubSub (“publish/subscribe”) is a pattern in which

    we publish messages to a “topic”, such that those messages can be consumed by any number of subscribers. 99
  44. 100.

    What is Phoenix PubSub? The Phoenix PubSub library allows us

    subscribe Elixir processes to a shared topic and publish messages to those processes. Phoenix Channels abstract away the interactions with Phoenix PubSub, but you can also use the PubSub library directly, which is exactly what we’ll do from within our LiveView. 100
  45. 101.

    Phoenix PubSub and Distributed Elixir Phoenix PubSub is configured by

    default to be distribution-friendly with Phoenix.PubSub.PG2. With this adapter, messages are broadcast to processes sharing a topic across nodes. 101
  46. 107.

    107 When a user sends an event to LiveView, we

    will broadcast it over PubSub to all the other users’ LiveViews
  47. 108.

    108 Our LiveView code will implement a handle_info/3 function that

    receives the broadcast, updates state, and causes the template to re-render
  48. 109.

    109 Client Submit “Start_pointing” event LiveView Client Client LiveView LiveView

    PubSub Broadcast to subscribers handle_info and update socket Re-render template
  49. 114.

    114 How can we sync and share stateful info, like

    who is present, across LiveView processes?
  50. 116.

    What is Phoenix Presence? The Phoenix.Presence module allows us to:

    • Register a process under a given topic • Store that info in a decentralized and resilient data store. • Broadcast presence-related events and sync presence data with ease. 116
  51. 118.

    Phoenix Presence Uses a CRDT A CRDT (Conflict-free Replicated Data

    Type) backs Phoenix Presence. What’s so special about a CFRDT? Unlike centralized data stores like Redis, Phoenix Presence… 118
  52. 119.

    “ …gives you high availability and performance because we don't

    have to send all data through a central server. It also gives you resilience to failure because the system automatically recovers from failures. - Chris McCord 119
  53. 121.

    Configure Presence • Define a module that uses Phoenix.Presence and

    shares a PubSub server with the rest of our application. 121
  54. 122.

    Register a Process • Use the Presence.Track behaviour to register

    a given user’s LiveView process under a shared topic 122
  55. 123.

    Broadcast Presence to Existing Users • Teach LiveView to handle

    the presence broadcast by fetching the list of present users from the presence data store 123
  56. 127.

    127 1 1 1 1 2 1 1 3 3

    1 1 4 6 4 1 …
  57. 128.
  58. 129.
  59. 132.

    132 1 1 1 1 2 1 1 3 3

    1 1 4 6 4 1
  60. 133.

    133 1 1 1 1 2 1 1 3 3

    1 1 4 6 4 1 …
  61. 134.

    “ 134 Given an integer n >= 0, write a

    function that returns the nth row of Pascal’s Triangle.
  62. 137.

    Row 0 describe "row/1" do test "returns [1] for first

    row" do assert Pascal.row(0) == [1] end end 137
  63. 138.

    Row 1 describe "row/1" do test "returns [1, 1] for

    second row" do assert Pascal.row(1) == [1, 1] end end 138
  64. 139.

    Row 4 describe "row/1" do test "returns value for 5th

    row" do assert Pascal.row(4) == [1, 4, 6, 4, 1] end end 139
  65. 140.
  66. 143.

    Disadvantages ◦ Requires you to get edge cases right ◦

    Doesn’t cover large indexes ◦ Doesn’t require a deep understanding of your domain
  67. 151.
  68. 154.

    154 [ [%{gc: []}], [%{O: [-1]}, %{f: [1, -2]}], [%{}],

    [ %{_: [], kTy: [4, -2, 0], u2h: [4, -2], w: [0, -3, 2, -4]}, %{_CK: [-2, -3, 3]} ], [ %{r: [-3, 2, -1, 2]}, %{_0: [1], _I: [-5, 0, 4, -3], _O: [-3, 3, 0], kz: [1, 4, 5, -1, -5]}, %{fr: [0, -2, 1, 3, 4]}, ] ]
  69. 156.

    Integers, Revisited StreamData.integer() |> Enum.take(25) # [0, -2, 1, -3,

    -1, 1, 4, 7, -1, 9, 0, 8, 9, -12, 12, -5, 0, -7, -16, 8, 21, 1, -16, -7, -19] 156
  70. 158.

    158 1 1 1 1 2 1 1 3 3

    1 1 4 6 4 1 …
  71. 159.

    Possible Test Implementation property "number of row elements is n+1"

    do check all number <- StreamData.integer(), row_index = abs(number) do element_count = length(Pascal.row(row_index)) assert element_count == row_index + 1 end end 159
  72. 164.

    assert length(Pascal.row(0)) == 1 assert length(Pascal.row(1)) == 2 assert length(Pascal.row(0))

    == 1 assert length(Pascal.row(2)) == 3 # … assert length(Pascal.row(47)) == 48 164
  73. 165.

    assert length(Pascal.row(0)) == 1 assert length(Pascal.row(1)) == 2 assert length(Pascal.row(0))

    == 1 assert length(Pascal.row(2)) == 3 # … assert length(Pascal.row(47)) == 48 assert length(Pascal.row(58)) == 117 165
  74. 166.

    assert length(Pascal.row(0)) == 1 assert length(Pascal.row(1)) == 2 assert length(Pascal.row(0))

    == 1 assert length(Pascal.row(2)) == 3 # … assert length(Pascal.row(47)) == 48 assert length(Pascal.row(58)) == 117 # ?!?! 166
  75. 167.

    Stuff that’s true of any solution 167 1 1 1

    1 2 1 1 3 3 1 1 4 6 4 1 …
  76. 168.

    Stuff that’s true of any solution ◦ Each row is

    symmetrical 168 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 …
  77. 169.

    Stuff that’s true of any solution ◦ Each row is

    symmetrical ◦ The first and last elements of each row are 1 169 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 …
  78. 170.

    Stuff that’s true of any solution ◦ Each row is

    symmetrical ◦ The first and last elements of each row are 1 ◦ After row 0, the second element in each list is the row index 170 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 …
  79. 171.

    Stuff that’s true of any solution ◦ Each row is

    symmetrical ◦ The first and last elements of each row are 1 ◦ After row 0, the second element in each list is the row index ◦ So is the second-to-last element 171 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 …
  80. 172.

    Stuff that’s true of any solution ◦ We can represent

    any row as a list 172 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 …
  81. 173.

    Stuff that’s true of any solution ◦ We can represent

    any row as a list ◦ Row 0 just has one element: 1 173 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 …
  82. 174.

    Stuff that’s true of any solution ◦ We can represent

    any row as a list ◦ Row 0 just has one element: 1 ◦ For all rows after 0, 1 appears twice 174 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 …
  83. 175.

    Stuff that’s true of any solution ◦ We can represent

    any row as a list ◦ Row 0 just has one element: 1 ◦ For all rows after 0, 1 appears twice ◦ Each row n has n+1 elements in it 175 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 …
  84. 176.

    Stuff that’s true of any solution ◦ We can represent

    any row as a list ◦ Row 0 just has one element: 1 ◦ For all rows after 0, 1 appears twice ◦ Each row n has n+1 elements in it ◦ In each row, numbers go up, then down 176 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 …
  85. 177.

    Potential Properties ◦ Data type ◦ Data shape ◦ Count

    of specific values ◦ Presence/emptiness ◦ Size ◦ Membership ◦ Range ◦ Symmetry ◦ Relationships
  86. 179.

    Voting Results def calculate_votes(users) do case winning_vote(users) do top_two when

    is_list(top_two) -> {"tie", top_two} winner -> {"winner", winner} end end
  87. 180.

    A Property Test property "calculated vote is a list or

    an integer" do check all users <- user_generator, {_event, winner} = calculate_votes(users) # We'll assert something here later end end
  88. 182.
  89. 184.

    What’s true of any solution? def calculate_votes(users) do case winning_vote(users)

    do top_two when is_list(top_two) -> {"tie", top_two} winner -> {"winner", winner} end end
  90. 187.

    Resources ◦ “An introduction to property-based testing” by Scott Wlaschin

    ◦ “Choosing properties for property-based testing” by Scott Wlaschin ◦ Property-Based Testing with PropEr, Erlang, and Elixir and PropEr Testing by Fred Hebert ◦ SteamData documentation ◦ “Testing the Hard Stuff and Staying Sane” by John Hughes ◦ “Picking Properties to Test in Property Based Testing” by Michael Stalker 187
  91. 188.