Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Keeping it Real(time)

Keeping it Real(time)

Modern web app interfaces can be dynamic with lots of data changing and flowing, but behind this there's often still a mass of AJAX requests & timers, making our work pseudo-live at best.

Web sockets allow us to create two way live data flow between server & client; WebRTC lets us allow our clients to connect to one another directly. We'll see how PHP fits into this front end ecosystem and how to augment our applications with real time components.

This talk has been given at: PHP Leeds (Jun '18), PHP Sheffield (Jan '19), PHP Benelux 2019, DPC 2019. There is also a full day workshop available for teams, companies, user groups or conferences, and student discounts available on workshop bookings.

Mike Lehan

June 08, 2019
Tweet

More Decks by Mike Lehan

Other Decks in Technology

Transcript

  1. KEEPING IT REAL(TIME)

    View Slide

  2. Hey there
    I’m Mike Lehan
    Software engineer, CTO of StuRents.com, skydiver, northerner
    Follow me on Twitter @M1ke
    I love feedback: joind.in/talk/04f9a

    View Slide


  3. Isn’t it all real time?

    View Slide

  4. WebRTC
    Browser to browser connections for video, audio & data
    Websockets
    Bi-directional communication with a standalone server

    View Slide

  5. Live chat
    Chat tool for users of our website to communicate in real
    time built with websockets. In production for >3 years
    ■ Hosted on smallest DO droplet; no outages with up to
    1000 connections
    ■ Hundreds of messages per day; live chats convert
    sales for clients in minutes rather than days
    Video calls
    Go face to face instantly from text chat using WebRTC. In
    limited beta release, used internally for 2 years
    My experience with real time tech

    View Slide

  6. A Specification
    for Live Chat
    What are we going to build?

    View Slide

  7. We need to...
    ■ Send & receive messages
    ■ Maintain privacy
    ■ Store chat history
    ■ Indicate who is online

    View Slide

  8. We can accomplish this with basic JavaScript + web
    ■ Message sent via asynchronous HTTP (AJAX)
    ■ Authenticate all requests at app level
    ■ Check for messages on a schedule
    ■ Save messages to shared data store
    ■ Record presence by logging AJAX requests

    View Slide


  9. Do you slow down your server, or your users?

    View Slide

  10. Websockets + WAMP
    A protocol defining actions one can take
    within a websocket system

    View Slide

  11. Crossbar, Autobahn and WAMP

    View Slide

  12. What we’re aiming to build
    ...and how they all talk to each other
    User
    Crossbar
    (WAMP Router)
    App
    Database
    Javascript
    User
    User
    Web server
    Worker (PHP)
    Lookup storage
    (Redis)
    Our network
    Chat server
    12
    Message Q

    View Slide

  13. Let’s break it down… step 1
    User
    App
    Database
    User
    Browser (JS)
    User
    Web server
    Our network
    ■ User loads a web page
    ■ App authenticates the user
    ■ Returns page HTML & JS for chat
    13

    View Slide

  14. Let’s break it down… step 2
    ■ JS establishes websocket connection
    ■ User can carry out actions defined by WAMP
    User
    Crossbar
    (WAMP Router)
    App
    Database
    User
    Javscript
    User
    Web server
    Our network
    Chat server
    14

    View Slide

  15. Subscribe
    To a named topic
    Publish
    To the same topic
    Message received
    Gets sent to all
    current subscribers
    Pub-sub
    connection.onopen = function (session, details) {
    // This is where you’d subscribe to things and
    // cache the session to your JS app to use it for
    // other actions your users may carry out
    };
    // Subscribing
    session.subscribe("myapp.thread-1", function(args) {
    console.log("Received message", args[0]);
    });
    // Publishing
    session.publish("myapp.thread-1", [‘hello’]);

    View Slide

  16. Register
    A procedure + name
    Call
    A named procedure
    with a data block
    Receive
    A block of data in
    response to the call
    Remote procedure call (RPC)
    // Register a procedure
    $session->register('myapp.add-numbers', function (array $args) {
    return $args[0] + $args[1];
    });
    // Call a procedure
    $session->call('myapp.add-numbers', [2, 3])->then(
    function ($res) {
    echo "Result: {$res}\n";
    },
    function ($error) {
    echo "Call Error: {$error}\n";
    });

    View Slide

  17. How are we doing on our goals?
    ■ Send & receive messages
    ■ Maintain privacy
    ■ Store chat history
    ■ Indicate who is online

    View Slide

  18. Authenticator
    Your existing
    application
    Real time
    application
    Make sure to add security...

    View Slide

  19. Permissions
    Define what the worker is
    allowed to do
    Startup
    Tell crossbar how to
    start the worker
    WAMPCRA
    Define the worker as
    your authenticator
    Using crossbar workers
    TCP
    Enable a TCP listener on the
    websocket router
    Connection
    The worker must
    connect like other
    clients
    Register
    The worker registers
    an RPC to auth other
    users
    The worker needs to establish a websocket connection

    View Slide

  20. EVENT LOOPS
    while (true) {
    // check handlers
    foreach ($handlers as $handler){
    if ($handler->shouldRun()){
    $handler->doSomething();
    }
    }
    sleep(1);
    }

    View Slide

  21. $loop = React\EventLoop\Factory::create();
    $wamp = new Thruway\Connection([
    "url" => $url,
    ], $loop);
    $connection->on("open", function($session){
    // do WAMP actions on the session
    });
    $connection->on("error", function($reason){
    // handle the error, e.g. log, restart
    });
    $connection->open();
    REACT PHP

    View Slide

  22. React
    PHP event loop + sockets
    Thruway
    WAMP implementation in PHP
    Ratchet
    Uses React to connect to websockets

    View Slide

  23. Let’s break it down… step 3
    User
    Crossbar
    (WAMP Router)
    App
    Database
    User
    Javascript
    User
    Web server
    Worker (PHP)
    Our network
    Chat server
    ■ JS establishes websocket connection
    ■ Crossbar authenticates user with app
    ■ User can carry out actions defined by WAMP
    23

    View Slide

  24. User Crossbar Worker App Database
    HTTP
    HTTP
    ?
    HTTP (API)
    HTTP (API)
    24
    WS
    WS (RPC)
    WS (RPC)
    WS
    ?
    ?
    Requests webpage
    Saves auth token
    Sends page + data
    Asks to connect
    Calls authenticator
    Sends user data
    Fetches auth token
    Returns auth token
    Verifies auth token
    Generates WAMP permission JSON
    Saves permissions
    Accepts connection
    “Onopen” called

    View Slide

  25. {
    "uri": "myapp.keep-alive",
    "match": "prefix",
    "allow": {
    "call": false,
    "register": false,
    "publish": true,
    "subscribe": false
    },
    "disclose": {
    "publisher": true
    }
    }
    {
    "uri": "myapp.thread-1",
    "match": "exact",
    "allow": {
    "call": false,
    "register": false,
    "publish": true,
    "subscribe": true
    },
    "disclose": {
    "publisher": true
    }
    }
    {
    "authorizer": "your.rpc",
    }
    Authenticator JSON
    This lets the router know
    what this client can do in
    future; the client is informed
    of this as well so they don’t
    get surprised by actions being
    blocked.
    URI applies to pub-sub topics
    and RPC procedure names
    Permissions are granular

    View Slide

  26. How are we doing on our goals?
    ■ Send & receive messages
    ■ Maintain privacy
    ■ Store chat history
    ■ Indicate who is online

    View Slide


  27. But it lost all my chats...

    View Slide

  28. ■ Pub-sub topics and RPC procedures to enable a client to
    monitor or control aspects of crossbar
    ■ Clients must have appropriate permissions to
    subscribe/call
    ■ Session publishes session join, leave; procedures to count,
    list or get session info and force disconnect
    ■ Subscription publishes subscription create, subscribe,
    leave; procedures to list, lookup, search
    ■ Registration publishes procedure create, delete;
    procedures to list, search
    Crossbar meta events

    View Slide

  29. User Crossbar Worker App Database
    29
    WS
    ...
    First subscriber to
    thread
    Subscribes to thread
    WS (RPC)
    Subscribe to meta create topic
    ...
    Publish message
    Relay to all
    subscribers
    Receives message
    Send to app
    Validates & stores
    WS
    WS
    TCP
    ?
    Meta create
    publish
    WS
    WS

    View Slide

  30. The full picture… step 4
    User
    Crossbar
    (WAMP Router)
    App
    Database
    Javascript
    User
    User
    Web server
    Worker (PHP)
    Lookup storage
    (Redis)
    Our network
    Chat server
    30
    Message Q

    View Slide

  31. How are we doing on our goals?
    ■ Send & receive messages
    ■ Maintain privacy
    ■ Store chat history
    ■ Indicate who is online

    View Slide


  32. Wait, who are you again?

    View Slide

  33. ■ Meta events allow access to WAMP internals but are too
    wide-ranging - permissions only at the URI level
    ■ By combining meta events with some form of memory
    the worker(s) can provide more restrictive lookups to
    clients
    ■ PHP worker could store in memory but this scales badly
    - if worker crashes all data is lost
    ■ Use the right tool - Redis is a super fast key-value store
    which can act like a cache
    Provide users with subscription information

    View Slide

  34. User Crossbar Worker Redis
    34
    ...
    WS (RPC) Register & subscribe
    Calls user procedure
    Looks up user
    Send result
    WS (RPC)
    Redis
    Caches publisher => user
    WS (RPC)
    Connects to crossbar
    Calls authenticator
    Stores user Redis
    WS
    WS (RPC)
    Stores topic Redis
    Subscribes to topic WS
    Publish meta event WS

    View Slide

  35. Worker (PHP) registers a procedure called “users”
    function (array $args, ClientSession $session){
    $thread_id = array_shift($args);
    $publisher = array_shift($args);
    // Fetch the name from our lookup storage
    return $this->getNameForThread($thread_id, $publisher);
    }
    Client (JS) can call this procedure on receipt
    const threadId = 1;
    session.subscribe("app.thread-"+threadId, function(args){
    const publisher = args[1];
    session.call("app.users", [threadId, publisher],
    function(name){
    console.log(“Message was from “+name);
    });
    });

    View Slide

  36. Who’s online
    ■ User can query for session related to individual message
    ■ OR user can query for all sessions subscribing to a thread
    Keep-alive
    ■ Not necessary to handle keep-alive manually
    ■ Crossbar can detect user going offline
    ■ Publishes to meta topic when this happens
    Who’s online & keep-alives

    View Slide

  37. How are we doing on our goals?
    ■ Send & receive messages
    ■ Maintain privacy
    ■ Store chat history
    ■ Indicate who is online

    View Slide

  38. In summary
    We produced a live chat system using
    ■ Crossbar WAMP router
    ■ Autobahn JS client-side
    ■ Thruway/Ratchet/React server-side
    ■ Message queue
    ■ Existing web application
    More robust, more scalable, instant when
    compared to a “basic” web implementation

    View Slide

  39. And the rest
    These tools can build it all

    View Slide

  40. WebRTC
    Going beyond text chat - can our users
    talk face to face?

    View Slide


  41. What about PHP?

    View Slide

  42. Signalling
    Hooking together two
    browsers requires
    those browsers know
    one another exist.
    Browsers generate
    data packet to identify
    location and
    communicate these
    Ideal to send over
    websockets
    STUN (+ICE)
    To find an optimal
    network path (e.g.
    bypassing NAT on
    routers) an external
    server can coordinate
    a network route. The
    server is known as a
    STUN server, the data
    sent is known as an ICE
    candidate
    TURN
    Some networks don’t
    play well with WebRTC
    - particularly larger
    corporate networks
    using symmetric NAT
    TURN servers can be
    used here - these relay
    WebRTC data directly
    (i.e. no longer simply
    peer to peer)
    Serverless… with servers?

    View Slide

  43. ■ WebRTC appears in a lot of places but still isn’t the best
    supported tool
    ■ Implemented by Google Hangouts, Slack, Skype, Discord
    etc, all of which have invested heavily
    ■ Routing and TURN can be an issue for corporate networks
    ■ Safari tends to lag behind, especially on iOS
    ■ Lots of confusion about permissions to access user camera
    and microphone via a web browser
    ■ Check callstats.io for monitoring and Twilio for some ready
    made WebRTC packages
    All the “gotchas”

    View Slide

  44. Cheers for listening!
    Any questions? Ask away...
    Or find me on:
    Twitter @M1ke
    Slack #phpnw & #og-aws
    Presentation template by SlidesCarnival
    joind.in/talk/04f9a
    speakerdeck.com/m1ke/keeping-it-real-time

    View Slide