Keeping it Real(time)

Modern web app interfaces can be dynamic with lots of data changing and flowing, but behind this there's often still a mass of AJAX requests & timers, making our work pseudo-live at best.

Web sockets allow us to create two way live data flow between server & client; WebRTC lets us allow our clients to connect to one another directly. We'll see how PHP fits into this front end ecosystem and how to augment our applications with real time components.

This talk has been given at: PHP Leeds (Jun '18), PHP Sheffield (Jan '19), PHP Benelux 2019, DPC 2019. There is also a full day workshop available for teams, companies, user groups or conferences, and student discounts available on workshop bookings.

Transcript

1. KEEPING IT REAL(TIME)
2. Hey there I’m Mike Lehan Software engineer, CTO of StuRents.com, skydiver, northerner Follow me on Twitter @M1ke I love feedback: joind.in/talk/04f9a
3. “ Isn’t it all real time?
4. WebRTC Browser to browser connections for video, audio & data Websockets Bi-directional communication with a standalone server
5. Live chat Chat tool for users of our website to communicate in real time built with websockets. In production for >3 years ▪ Hosted on smallest DO droplet; no outages with up to 1000 connections ▪ Hundreds of messages per day; live chats convert sales for clients in minutes rather than days Video calls Go face to face instantly from text chat using WebRTC. In limited beta release, used internally for 2 years My experience with real time tech
6. A Specification for Live Chat What are we going to build?
7. We need to... ▪ Send & receive messages ▪ Maintain privacy ▪ Store chat history ▪ Indicate who is online
8. We can accomplish this with basic JavaScript + web ▪ Message sent via asynchronous HTTP (AJAX) ▪ Authenticate all requests at app level ▪ Check for messages on a schedule ▪ Save messages to shared data store ▪ Record presence by logging AJAX requests
9. “ Do you slow down your server, or your users?
10. Websockets + WAMP A protocol defining actions one can take within a websocket system
11. Crossbar, Autobahn and WAMP
12. What we’re aiming to build ...and how they all talk to each other User Crossbar (WAMP Router) App Database Javascript User User Web server Worker (PHP) Lookup storage (Redis) Our network Chat server 12 Message Q
13. Let’s break it down… step 1 User App Database User Browser (JS) User Web server Our network ▪ User loads a web page ▪ App authenticates the user ▪ Returns page HTML & JS for chat 13
14. Let’s break it down… step 2 ▪ JS establishes websocket connection ▪ User can carry out actions defined by WAMP User Crossbar (WAMP Router) App Database User Javscript User Web server Our network Chat server 14
15. Subscribe To a named topic Publish To the same topic Message received Gets sent to all current subscribers Pub-sub connection.onopen = function (session, details) { // This is where you’d subscribe to things and // cache the session to your JS app to use it for // other actions your users may carry out }; // Subscribing session.subscribe("myapp.thread-1", function(args) { console.log("Received message", args[0]); }); // Publishing session.publish("myapp.thread-1", [‘hello’]);
16. Register A procedure + name Call A named procedure with a data block Receive A block of data in response to the call Remote procedure call (RPC) // Register a procedure $session->register('myapp.add-numbers', function (array $args) { return $args[0] + $args[1]; }); // Call a procedure $session->call('myapp.add-numbers', [2, 3])->then( function ($res) { echo "Result: {$res}\n"; }, function ($error) { echo "Call Error: {$error}\n"; });
17. How are we doing on our goals? ▪ Send & receive messages ▪ Maintain privacy ▪ Store chat history ▪ Indicate who is online
18. Authenticator Your existing application Real time application Make sure to add security...
19. Permissions Define what the worker is allowed to do Startup Tell crossbar how to start the worker WAMPCRA Define the worker as your authenticator Using crossbar workers TCP Enable a TCP listener on the websocket router Connection The worker must connect like other clients Register The worker registers an RPC to auth other users The worker needs to establish a websocket connection
20. EVENT LOOPS while (true) { // check handlers foreach ($handlers as $handler){ if ($handler->shouldRun()){ $handler->doSomething(); } } sleep(1); }
21. $loop = React\EventLoop\Factory::create(); $wamp = new Thruway\Connection([ "url" => $url, ], $loop); $connection->on("open", function($session){ // do WAMP actions on the session }); $connection->on("error", function($reason){ // handle the error, e.g. log, restart }); $connection->open(); REACT PHP
22. React PHP event loop + sockets Thruway WAMP implementation in PHP Ratchet Uses React to connect to websockets
23. Let’s break it down… step 3 User Crossbar (WAMP Router) App Database User Javascript User Web server Worker (PHP) Our network Chat server ▪ JS establishes websocket connection ▪ Crossbar authenticates user with app ▪ User can carry out actions defined by WAMP 23
24. User Crossbar Worker App Database HTTP HTTP ? HTTP (API) HTTP (API) 24 WS WS (RPC) WS (RPC) WS ? ? Requests webpage Saves auth token Sends page + data Asks to connect Calls authenticator Sends user data Fetches auth token Returns auth token Verifies auth token Generates WAMP permission JSON Saves permissions Accepts connection “Onopen” called
25. { "uri": "myapp.keep-alive", "match": "prefix", "allow": { "call": false, "register": false, "publish": true, "subscribe": false }, "disclose": { "publisher": true } } { "uri": "myapp.thread-1", "match": "exact", "allow": { "call": false, "register": false, "publish": true, "subscribe": true }, "disclose": { "publisher": true } } { "authorizer": "your.rpc", } Authenticator JSON This lets the router know what this client can do in future; the client is informed of this as well so they don’t get surprised by actions being blocked. URI applies to pub-sub topics and RPC procedure names Permissions are granular
26. How are we doing on our goals? ▪ Send & receive messages ▪ Maintain privacy ▪ Store chat history ▪ Indicate who is online
27. “ But it lost all my chats...
28. ▪ Pub-sub topics and RPC procedures to enable a client to monitor or control aspects of crossbar ▪ Clients must have appropriate permissions to subscribe/call ▪ Session publishes session join, leave; procedures to count, list or get session info and force disconnect ▪ Subscription publishes subscription create, subscribe, leave; procedures to list, lookup, search ▪ Registration publishes procedure create, delete; procedures to list, search Crossbar meta events
29. User Crossbar Worker App Database 29 WS ... First subscriber to thread Subscribes to thread WS (RPC) Subscribe to meta create topic ... Publish message Relay to all subscribers Receives message Send to app Validates & stores WS WS TCP ? Meta create publish WS WS
30. The full picture… step 4 User Crossbar (WAMP Router) App Database Javascript User User Web server Worker (PHP) Lookup storage (Redis) Our network Chat server 30 Message Q
31. How are we doing on our goals? ▪ Send & receive messages ▪ Maintain privacy ▪ Store chat history ▪ Indicate who is online
32. “ Wait, who are you again?
33. ▪ Meta events allow access to WAMP internals but are too wide-ranging - permissions only at the URI level ▪ By combining meta events with some form of memory the worker(s) can provide more restrictive lookups to clients ▪ PHP worker could store in memory but this scales badly - if worker crashes all data is lost ▪ Use the right tool - Redis is a super fast key-value store which can act like a cache Provide users with subscription information
34. User Crossbar Worker Redis 34 ... WS (RPC) Register & subscribe Calls user procedure Looks up user Send result WS (RPC) Redis Caches publisher => user WS (RPC) Connects to crossbar Calls authenticator Stores user Redis WS WS (RPC) Stores topic Redis Subscribes to topic WS Publish meta event WS
35. Worker (PHP) registers a procedure called “users” function (array $args, ClientSession $session){ $thread_id = array_shift($args); $publisher = array_shift($args); // Fetch the name from our lookup storage return $this->getNameForThread($thread_id, $publisher); } Client (JS) can call this procedure on receipt const threadId = 1; session.subscribe("app.thread-"+threadId, function(args){ const publisher = args[1]; session.call("app.users", [threadId, publisher], function(name){ console.log(“Message was from “+name); }); });
36. Who’s online ▪ User can query for session related to individual message ▪ OR user can query for all sessions subscribing to a thread Keep-alive ▪ Not necessary to handle keep-alive manually ▪ Crossbar can detect user going offline ▪ Publishes to meta topic when this happens Who’s online & keep-alives
37. How are we doing on our goals? ▪ Send & receive messages ▪ Maintain privacy ▪ Store chat history ▪ Indicate who is online
38. In summary We produced a live chat system using ▪ Crossbar WAMP router ▪ Autobahn JS client-side ▪ Thruway/Ratchet/React server-side ▪ Message queue ▪ Existing web application More robust, more scalable, instant when compared to a “basic” web implementation
39. And the rest These tools can build it all
40. WebRTC Going beyond text chat - can our users talk face to face?
41. “ What about PHP?
42. Signalling Hooking together two browsers requires those browsers know one another exist. Browsers generate data packet to identify location and communicate these Ideal to send over websockets STUN (+ICE) To find an optimal network path (e.g. bypassing NAT on routers) an external server can coordinate a network route. The server is known as a STUN server, the data sent is known as an ICE candidate TURN Some networks don’t play well with WebRTC - particularly larger corporate networks using symmetric NAT TURN servers can be used here - these relay WebRTC data directly (i.e. no longer simply peer to peer) Serverless… with servers?
43. ▪ WebRTC appears in a lot of places but still isn’t the best supported tool ▪ Implemented by Google Hangouts, Slack, Skype, Discord etc, all of which have invested heavily ▪ Routing and TURN can be an issue for corporate networks ▪ Safari tends to lag behind, especially on iOS ▪ Lots of confusion about permissions to access user camera and microphone via a web browser ▪ Check callstats.io for monitoring and Twilio for some ready made WebRTC packages All the “gotchas”
44. Cheers for listening! Any questions? Ask away... Or find me on: Twitter @M1ke Slack #phpnw & #og-aws Presentation template by SlidesCarnival joind.in/talk/04f9a speakerdeck.com/m1ke/keeping-it-real-time