Dis.io

A browser based distributing grid computing platform. My final year project defence.

Transcript

1. A browser based distributed grid computing platform Tom Gallacher
2. Dis.io
3. What is Dis.io? • Grid computing platform • Based in the web browser • Implementable by any web site • Horizontally scalable • Transparent to participants • Built entirely in JavaScript
4. Why Dis.io? • Platform agnostic • Potential contributors is proportionate to daily visitors • Scalable across multiple sites
5. Architecture Managers, Distributors and Clients
6. Scalable • Each component can be spawned as many times needed • Each can be placed on different machines • Components find each other using Bonjour/ZeroConf/MDNS
7. Terms A payload is: • An object of parameters • Sent to Clients A work unit is: • Executable code which takes a payload A task is: • Amalgamation of payloads and work units
8. Client • Browser-side implementation • Uses the Web Worker API • Implements Web Sockets via Socket.io
9. Managers The Provider. • Sends work units to Distributors • Breaks down tasks into payloads • Uses ZeroMQ for communication to and from distributors • Saves results sent from distributors
10. Distributors The Hub. • Communications with Clients and Managers • Uses a combination of ZeroMQ and Web sockets • Caches Messages if a manager is offline • Provides payloads and work units to clients
11. Dashboard The Control Panel. • Facilitates creation of Tasks and work units. • Status of various components
12. Demo
13. Start the MongoDB database
14. Start the Dashboard's Webserver
15. Create a task
16. Spool up the Manager
17. Start Distributors
18. Verify connection on the Manager
19. Open a site that is serving the client
20. Number of completed tasks before
21. Number of completed after client has connected
22. Still continues if managers goes down
23. Performance Benchmarks
24. Dis.io cluster vs a single machine
25. Time per datapoint
26. What was learned • Distributed systems are very complex • Effective communication between nodes is needed • Performance is based on the JavaScript Engine.
27. Limitations • JavaScript is slower than compiled languages • It's possible for clients to spoof results • Useable only in modern browser • Task creation is not flexible
28. Recap • Dis.io browser based computing platform • Designed with scalability in mind • Out performs a single process • Easy management
29. Any Questions?