Adventures in P2P JavaScript

Come on a peer to peer JavaScript adventure, and learn about the challenges that await you if you build P2P applications in the browser!

Transcript

1. P2P JavaScript Adventures
2. 24 years old
3. Hacker
4. say "hi"
5. filldisk.com
6. you can always make it work in the browser somehow
7. config.php~
9. dominic, substack, max, mafintosh
10. "Mad science"
11. WebRTC
12. PeerCDN Closed source, sold to Yahoo
13. WebTorrent
14. WebTorrent » Open source » Works in node & browser » No install (pure JavaScript) » Insanely fast » Exposes files as streams » Stream into <video>, VLC, Chromecast, Airplay
15. var WebTorrent = require('webtorrent') var client = new WebTorrent() var magnetUri = '...' client.add(magnetUri, function (torrent) { console.log('Got torrent metadata!', torrent.infoHash) // Let's say the first file is a video var file = torrent.files[0] var video = document.createElement('video') document.body.appendChild(video) file.createReadStream().pipe(video) })
16. Is the browser fast enough for BitTorrent?
17. Browserify vs. Node » 888,614 ops/sec (browserify buffer.concat) » 1,832,307 ops/sec (node buffer.concat) » 4,094 ops/sec (browserify buffer.readDoubleBE) » 1,587,308 ops/sec (node buffer.readDoubleBE)
18. Buffer inherits from Object » Not meant for binary » slice() doesn't inherit from parent buffer » buffer[0] = 1000.22
19. Buffer should inherit from Uint8Array
20. function Buffer (length) { return new Uint8Array(length) }
21. function Buffer (length) { return augment(new Uint8Array(length)) } function augment (arr) { arr.write = Buffer.prototype.write arr.toString = Buffer.prototype.toString arr.toJSON = Buffer.prototype.toJSON arr.equals = Buffer.prototype.equals // … return arr }
22. Browserify vs. Node » 1,438,825 ops/sec ops/sec (browserify buffer.concat) » 1,832,307 ops/sec (node buffer.concat) » 1,057,233 ops/sec (browserify buffer.readDoubleBE) » 1,587,308 ops/sec (node buffer.readDoubleBE)
23. you can always make it work in the browser somehow
24. Extra credit » Passes iojs test suite » Use Object implementation for old browsers » Support IE6, 7, 8
25. Small modules
26. studynotes / package.json "dependencies": { "MD5": "^1.2.0", "add-commas": "0.0.4", "bcrypt": "^0.8.0", "body-parser": "^1.0.1", "compression": "^1.0.1", "connect-flash": "^0.1.1", "connect-mongo": "^0.7.0", "connect-slashes": "^1.2.0", "cookie-parser": "^1.0.1", "csurf": "^1.1.0", "debug": "^2.1.1", "escape-string-regexp": "^1.0.2", "express": "^4.0.0", "express-session": "^1.0.2", "font-awesome": "^4.2.0", "html-parser": "^0.8.0", "html-truncate": "^1.0.3", "http-status-codes": "^1.0.2", "humane-js": "^3.1.0", "jade": "^1.1.5", "jquery": "^2.1.1", "jsdom": "^4.0.2", "keymaster": "^1.6.2", "lodash.countby": "^3.0.0", "lodash.throttle": "^3.0.1", "maxcdn": "^0.1.5", "moment": "^2.5.1", "mongoose": "~3.8.0", "mongoose-validator": "^1.0.3", "nodemailer": "^1.1.1", "object-values": "^1.0.0", "once": "^1.3.0", "optimist": "^0.6.0", "passport": "^0.2.0", "passport-local": "^1.0.0", "posix": "^2.0.0", "run-auto": "^1.0.0", "run-parallel": "^1.0.0", "run-series": "^1.0.2", "run-waterfall": "^1.0.2", "serve-favicon": "^2.0.1", "simple-websocket": "^1.0.4", "stripe": "^3.1.0", "transparency": "^0.10.0", "underscore.string": "^3.0.1", "ws": "^0.7.0", "xtend": "^4.0.0" }
27. “When applications are done well, they are just the really application-specific, brackish residue that can't be so easily abstracted away. All the nice, reusable components sublimate away onto github and npm where everybody can collaborate to advance the commons.” — substack
28. webtorrent modules » bittorrent-dht (distributed hash table client) » bittorrent-tracker (tracker server/client) » bittorrent-protocol (bittorrent protocol stream) » create-torrent (create .torrent files) » parse-torrent (parse torrent identifiers) » simple-peer (simple WebRTC connections) » many more...
29. sharing P2P primitives
30. otrtalk » OTR (off-the-record messaging) » encryption » authentication » forward secrecy » uses DHT to discover buddies » telehash » bittorrent (bittorrent-dht)
31. mafintosh » peerflix (used by popcorn time) » torrent-mount » peerwiki
32. OPEN open source
33. easy to test
34. share your stackoverflow snippets
35. independently versioned
36. more likely to eventually be "done"
37. xhr var xhr = require('xhr') xhr('http://example.com', function (err, resp, body) { if (err) throw err console.log(body) })
38. drag-drop var dragDrop = require('drag-drop') dragDrop('#dropTarget', function (files) { console.log(files) // got the files })
39. WebRTC
42. simple-peer var SimplePeer = require('simple-peer') var peer1 = new SimplePeer({ initiator: true }) var peer2 = new SimplePeer() peer1.on('signal', function (data) { peer2.signal(data) }) peer2.on('signal', function (data) { peer1.signal(data) })
43. simple-peer peer1.send('hey peer2, how is it going?') peer2.on('data', function (data) { console.log('got a message from peer1: ' + data) })
44. browserify
45. the "browser" field in package.json
46. webtorrent / package.json { "browser": { "./lib/helper.js": "./lib/helper-browser.js" } }
47. webtorrent / package.json { "browser": { "bittorrent-tracker": "webtorrent-tracker" } }
48. jsdom / package.json { "browser": { "canvas": "false" } }
49. Streaming <video>
50. http <video> <video src="rickastley.mov"></video> » uses HTTP range requests » supports video seeking
51. what about video from WebRTC? var data = new Buffer(1000000) // from remote peer var video = document.querySelector('video') var url = Object.createObjectURL(new Blob([ data ])) video.src = url video.play()
52. what about streaming? » Blob is immutable » we'll need a different approach
53. MediaSource API
54. MediaSource API » Allows JavaScript to generate media streams for playback » DASH ("Dynamic Adaptive Streaming over HTTP") » Change video quality as network conditions change » Supported in Chrome, IE11, Safari 8, Fx Nightly » Requires H.264 or VP8/9 codecs » Supports seeking
55. MediaSource API var mediaSource = new MediaSource() var video = document.querySelector('video') video.src = window.URL.createObjectURL(mediaSource) var sourceBuffer = mediaSource.addSourceBuffer('video/mp4') file.createReadStream().on('data', function (data) { sourceBuffer.appendBuffer(data) })
56. How does seeking work? » Requires video files conform to ISO MPEG-DASH » Basically NONE currently do » Requires your code to map timecode -> byte offset
57. <video> seeking video.addEventListener('waiting', function () { var time = video.currentTime // stop fetching current byte range var byteOffset = timeToByte(time) // tell all peers to start giving me data from `byteOffset` file.createReadStream().on('data', function (data) { sourceBuffer.appendBuffer(data) }) })
58. time code -> byte offset » Usually MP4 files are conferted to ISO MPEG-DASH ("re-packaged") offline » For WebTorrent, what can we do?
59. can seeders repackage the video?
60. for new torrents, yes
61. for existing torrents?
62. no, hash verification would fail
63. can the downloader repackage the video?
64. yes, with mp4box.js! » process MP4 files in the browser » support for progressive parsing
65. var WebTorrent = require('webtorrent') var client = new WebTorrent() var magnetUri = '...' client.add(magnetUri, function (torrent) { var file = torrent.files[0] var video = document.createElement('video') document.body.appendChild(video) file.createReadStream().pipe(video) }) coming soon to webtorrent!
66. you can always make it work in the browser somehow
67. JavaScript Standard Style
68. What to do with good PRs that have poor code style?
69. give feedback, then wait
70. merge and fix it after
71. use a linter
72. super annoying .rc files » .jshintrc » .eslintrc » .jscsrc » how to keep them in sync?
73. standard $ npm install standard -g $ standard Error: Use JavaScript Standard Style lib/torrent.js:950:11: Expected '===' and instead saw '=='.
74. package.json { "name": "my-cool-package", "devDependencies": { "standard": "^3.0.0" }, "scripts": { "test": "standard && node tests.js" } }
75. saves time in two ways: » No configuration. The easiest way to enforce consistent style in your module/project. Just drop it in. » Catch style errors before they're submitted in PRs. Saves precious code review time by eliminating back-and-forth between maintainer and contributor.
76. “Adopting standard style means ranking the importance of community conventions higher than personal style, which does not make sense for 100% of projects and development cultures. At the same time, open source can be a hostile place for newbies. Setting up clear, automated contributor expectations makes a project healthier.” – Max Ogden
77. lots of people loved it » me » max ogden » mafintosh » henrik jorteg (&yet) » forrest norvell (npm) » lots more...
78. please add a config option for X!
79. lots of people had opinions "Like I said though, this is just my opinion..." "could we add X? maybe." "possibly" ";-)"
80. ;
81. first rule of javascript: never talk about semicolons
82. objections to the name » not a TC39 or ECMA standard » "How dare you call this standard!"
83. semistandard » "All the goodness of feross/standard with semicolons sprinkled on top"
84. ❤
85. learn more about WebRTC » www.html5rocks.com/en/tutorials/webrtc/basics » webrtchacks.com » "simple-peer" on npm
86. learn more about webtorrent webtorrent.io webtorrent.io/roadmap