A browser based distributed grid
computing platform
Tom Gallacher
Slide 2
Slide 2 text
Dis.io
Slide 3
Slide 3 text
What is Dis.io?
● Grid computing platform
● Based in the web browser
● Implementable by any web site
● Horizontally scalable
● Transparent to participants
● Built entirely in JavaScript
Slide 4
Slide 4 text
Why Dis.io?
● Platform agnostic
● Potential contributors is proportionate to daily visitors
● Scalable across multiple sites
Slide 5
Slide 5 text
Architecture
Managers, Distributors and Clients
Slide 6
Slide 6 text
Scalable
● Each component can be spawned as many times needed
● Each can be placed on different machines
● Components find each other using Bonjour/ZeroConf/MDNS
Slide 7
Slide 7 text
Terms
A payload is:
● An object of parameters
● Sent to Clients
A work unit is:
● Executable code which takes a payload
A task is:
● Amalgamation of payloads and work units
Slide 8
Slide 8 text
Client
● Browser-side implementation
● Uses the Web Worker API
● Implements Web Sockets via Socket.io
Slide 9
Slide 9 text
Managers
The Provider.
● Sends work units to Distributors
● Breaks down tasks into payloads
● Uses ZeroMQ for communication to and from distributors
● Saves results sent from distributors
Slide 10
Slide 10 text
Distributors
The Hub.
● Communications with Clients and Managers
● Uses a combination of ZeroMQ and Web sockets
● Caches Messages if a manager is offline
● Provides payloads and work units to clients
Slide 11
Slide 11 text
Dashboard
The Control Panel.
● Facilitates creation of Tasks and work units.
● Status of various components
Slide 12
Slide 12 text
Demo
Slide 13
Slide 13 text
Start the MongoDB database
Slide 14
Slide 14 text
Start the Dashboard's Webserver
Slide 15
Slide 15 text
Create a task
Slide 16
Slide 16 text
Spool up the Manager
Slide 17
Slide 17 text
Start Distributors
Slide 18
Slide 18 text
Verify connection on the Manager
Slide 19
Slide 19 text
Open a site that is serving the client
Slide 20
Slide 20 text
Number of completed tasks before
Slide 21
Slide 21 text
Number of completed after client has connected
Slide 22
Slide 22 text
Still continues if managers goes down
Slide 23
Slide 23 text
Performance Benchmarks
Slide 24
Slide 24 text
Dis.io cluster vs a single machine
Slide 25
Slide 25 text
Time per datapoint
Slide 26
Slide 26 text
What was learned
● Distributed systems are very complex
● Effective communication between nodes is needed
● Performance is based on the JavaScript Engine.
Slide 27
Slide 27 text
Limitations
● JavaScript is slower than compiled languages
● It's possible for clients to spoof results
● Useable only in modern browser
● Task creation is not flexible
Slide 28
Slide 28 text
Recap
● Dis.io browser based computing platform
● Designed with scalability in mind
● Out performs a single process
● Easy management