@J7mbo

Two words about myself @J7mbo James Mallison

@J7mbo

Websockets and Torrents A match made with PHP! (and not nodeJS) @J7mbo • The story of my interest in websockets • Implementing the technology for arbitrary (torrent) data • Problems I encountered and solved • A Demo!! • What I learned

Legit speeds @J7mbo

@J7mbo

Transmission - Worst Torrent GUI Ever @J7mbo

(I was a graduate…) @J7mbo

• Browser makes AJAX request • Server performs CLI call • Server returns Json Response AJAX Request Browser Server Command line Json Response Browser Server AJAX to Server @J7mbo

@J7mbo

@J7mbo

• Browser makes AJAX request • Server SSHs to VPS over TOR • SSH over TOR using Torify • VPS performs CLI call • VPS returns Json Response • Server returns Json Response Browser My Server SSH to VPS Json Response Browser My Server Command line AJAX Request Ajax with SSH to VPS @J7mbo TOR (torify)

@J7mbo

@J7mbo

• Browser sends ws:// or wss:// • Server responds with HTTP/ 1.1 Switching Protocols • Open and persistent connection • Connection can be closed by either server or client at any time. Websockets @J7mbo

Request to Server Response to Client END Http VS Websockets • Send a request • HALF DUPLEX • Wait for a response • Rinse, repeat Http Event Loop WS Connection Started Send response Send response Send another response Receive message Websockets • Initialise connection • FULL DUPLEX • Send / receive at any time • Close connection @J7mbo

Duplex wtf? @J7mbo Half Duplex In a conversation, only one person can talk at once The other person has to listen and wait to give a response Full Duplex In a conversation, everyone’s able to talk and listen at once (Like Internals) Everyone is able to understand what each other is saying (Not Like Internals)

@J7mbo Event Loop

• Not just while(){} loops • Use interrupt driven I/O instead of polling • IDI/O flag is set on CPU, process suspended and buffer filled • Flag changes, callback executed (Still requires flag checks, not magic) • Useful for IPC • PHP’s PCNTL similar to this (signal based) @J7mbo Event Loops

PCNTL Extension @J7mbo

@J7mbo Ratchet

• Code flow controlled by user • Allows software to react • User calls .subscribe() in JS, onSubscribe() is called in the PHP event handler @J7mbo Event Driven? @J7mbo

The Web Application Messaging Protocol @J7mbo WAMP Events

onOpen() @J7mbo

onOpen() @J7mbo

onSubscribe() @J7mbo

onSubscribe() @J7mbo

Websocket Event Handler + Loop HTTP or Websocket? ProxyPass Http Ws Apache + PHP Client makes a request Enable mod_proxy • ProxyPass /wss/ wss://127.0.0.1:9000/ @J7mbo

Event Loop .subscribe(func, topic) 1 onSubscribe($conn, $topic) • store connection in array • start timer for connection • store timer against connection 2 High level functionality + events Timer hit • Get torrent data via CLI • This takes ages… … … 3 ‘Broadcast’ data back to topic • $topic->broadcast($data) 4 Client calls .unsubscribe(func, topic) • Event handler onUnsubscribe() called • Stop timer for connection • Remove from internal connections array 5 @J7mbo

Event Loop Problem 1 - Topic ‘Broadcast’ data back to topic • $topic->broadcast($data) 4 Option 1: $topic = ‘torrents’ • - Not on a per-user or per-connection basis • - Everyone subscribed to ‘torrents’ gets the data pushed to them • - Open same page, data gets sent to both pages twice Option 2: $topic = { ‘topic’ => ‘torrents’, ‘userId’ => 245 } • Every subscription is now unique per user id • - Same user can open same page and start getting duplicate data like before @J7mbo

Problem 2: Authentication and Duplicate data The Problem • Pass user id from PHP to JS variable • Subscribe to torrents with topic containing user id • Client could have just changed user id JS variable before subscribing One Solution • Create unique token on each standard HTTP request in your application • Use PHP 7.1s CSPRNG rather than open_ssl_random_pseudo_bytes() • Pass token with user id in topic and check against database • Recreate token in database so that a second check will fail $topic = { ‘topic’ => ‘torrents’, ‘userId’ => 245, ‘token’ => ‘385e33f741’ // bin2hex(random_bytes(5)) in PHP 7.1 }; @J7mbo

Problem 3: Blocking!! The Problem • Whilst command line call is being made (5 seconds), nothing else can happen • No .subscribe ——> onSubscribe() : Nobody else can do anything • This is blocking! Same goes for authentication (checking database) A few solutions • Use another process to do the work - on the same machine • Use a job queue - requires a way to get data back into event loop once done • Requires good architecture decisions for low coupling + high cohesion for event naming, job naming, whether worker knows who’s job it is etc @J7mbo

Blocking - Option 1 - Child Process The Result • Event loop can continue, fine for small projects or just as a proof of concept • Process is only on one machine - not scalable @J7mbo

Blocking - Option 2 - Job Queue (and ZMQ) In onSubscribe() - add to queue ZMQ listening - When setting up event handler @J7mbo

Blocking - Option 2 - Job Queue (and ZMQ) Worker sends data over :5555 when done to onZmqResponse() @J7mbo

Event Loop .subscribe(func, topic) 1 High level architecture onSubscribe($conn, $topic) • store connection in array • add job to queue (fire and forget) 2 Queue Workers pick up job • Perform job • Send result to :5555 3 Workers Anything sent to port 5555 goes to onZmqRequest() Broadcasted to client 4 • Event loop is just an intermediary for data retrieval and sending • Nobody is blocked from registering their interest in receiving data • Workers can be in Java, C*, Erlang, Python, Ruby, JS etc @J7mbo

Demo ? @J7mbo

What about scaling? (One way without a load balancer) @J7mbo Client API DB / Redis Workers Queue Call API Event Loop Event Loop Event Loop Call chosen ws:// server Round Robin? ZMQ Scalable Scalable Broadcast data to client

• Using an event loop != non-blocking or asynchronous • Easiest way to not block, use threads, processes or an external library • All connections between client / ws server over SSH with TOR • Could accomplish with NodeJS (but didn’t want to sell my soul) • Could have just made RPC calls to the server anyway instead of cmd- line (I found this out right at the end of it all) • Didn’t have to broadcast on a $topic, as $connection had methods of sending data to specific connections What I learned @J7mbo

• Event loops and reactors in PHP: React, amphp • Websocket server-side libraries: ratchet, php-websockets • WAMP-specced client-side libraries: Autobahn.js, AngularWAMP, Minion • Event-loop underlying libs: LibEv, LibEvent, LibUV (windows) Your Choices @J7mbo

• @j7mbo (PHP6 jokes, php, oop, software engineering) • PHPNW15: Justin Carmony - Scaling & Managing Asynchronous Workers (and staying sane!) - Watch this for queues etc - it’s an awesome talk! Thanks @J7mbo https://joind.in/talk/63c1c

DEMO (if it works)