Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Adventures in P2P JavaScript

Adventures in P2P JavaScript

Come on a peer to peer JavaScript adventure, and learn about the challenges that await you if you build P2P applications in the browser!

B498d33041627b07726dc29c28f02df7?s=128

Feross Aboukhadijeh

March 28, 2015
Tweet

Transcript

  1. P2P JavaScript Adventures

  2. 24 years old

  3. Hacker

  4. say "hi"

  5. filldisk.com

  6. you can always make it work in the browser somehow

  7. config.php~

  8. None
  9. dominic, substack, max, mafintosh

  10. "Mad science"

  11. WebRTC

  12. PeerCDN Closed source, sold to Yahoo

  13. WebTorrent

  14. WebTorrent » Open source » Works in node & browser

    » No install (pure JavaScript) » Insanely fast » Exposes files as streams » Stream into <video>, VLC, Chromecast, Airplay
  15. var WebTorrent = require('webtorrent') var client = new WebTorrent() var

    magnetUri = '...' client.add(magnetUri, function (torrent) { console.log('Got torrent metadata!', torrent.infoHash) // Let's say the first file is a video var file = torrent.files[0] var video = document.createElement('video') document.body.appendChild(video) file.createReadStream().pipe(video) })
  16. Is the browser fast enough for BitTorrent?

  17. Browserify vs. Node » 888,614 ops/sec (browserify buffer.concat) » 1,832,307

    ops/sec (node buffer.concat) » 4,094 ops/sec (browserify buffer.readDoubleBE) » 1,587,308 ops/sec (node buffer.readDoubleBE)
  18. Buffer inherits from Object » Not meant for binary »

    slice() doesn't inherit from parent buffer » buffer[0] = 1000.22
  19. Buffer should inherit from Uint8Array

  20. function Buffer (length) { return new Uint8Array(length) }

  21. function Buffer (length) { return augment(new Uint8Array(length)) } function augment

    (arr) { arr.write = Buffer.prototype.write arr.toString = Buffer.prototype.toString arr.toJSON = Buffer.prototype.toJSON arr.equals = Buffer.prototype.equals // … return arr }
  22. Browserify vs. Node » 1,438,825 ops/sec ops/sec (browserify buffer.concat) »

    1,832,307 ops/sec (node buffer.concat) » 1,057,233 ops/sec (browserify buffer.readDoubleBE) » 1,587,308 ops/sec (node buffer.readDoubleBE)
  23. you can always make it work in the browser somehow

  24. Extra credit » Passes iojs test suite » Use Object

    implementation for old browsers » Support IE6, 7, 8
  25. Small modules

  26. studynotes / package.json "dependencies": { "MD5": "^1.2.0", "add-commas": "0.0.4", "bcrypt":

    "^0.8.0", "body-parser": "^1.0.1", "compression": "^1.0.1", "connect-flash": "^0.1.1", "connect-mongo": "^0.7.0", "connect-slashes": "^1.2.0", "cookie-parser": "^1.0.1", "csurf": "^1.1.0", "debug": "^2.1.1", "escape-string-regexp": "^1.0.2", "express": "^4.0.0", "express-session": "^1.0.2", "font-awesome": "^4.2.0", "html-parser": "^0.8.0", "html-truncate": "^1.0.3", "http-status-codes": "^1.0.2", "humane-js": "^3.1.0", "jade": "^1.1.5", "jquery": "^2.1.1", "jsdom": "^4.0.2", "keymaster": "^1.6.2", "lodash.countby": "^3.0.0", "lodash.throttle": "^3.0.1", "maxcdn": "^0.1.5", "moment": "^2.5.1", "mongoose": "~3.8.0", "mongoose-validator": "^1.0.3", "nodemailer": "^1.1.1", "object-values": "^1.0.0", "once": "^1.3.0", "optimist": "^0.6.0", "passport": "^0.2.0", "passport-local": "^1.0.0", "posix": "^2.0.0", "run-auto": "^1.0.0", "run-parallel": "^1.0.0", "run-series": "^1.0.2", "run-waterfall": "^1.0.2", "serve-favicon": "^2.0.1", "simple-websocket": "^1.0.4", "stripe": "^3.1.0", "transparency": "^0.10.0", "underscore.string": "^3.0.1", "ws": "^0.7.0", "xtend": "^4.0.0" }
  27. “When applications are done well, they are just the really

    application-specific, brackish residue that can't be so easily abstracted away. All the nice, reusable components sublimate away onto github and npm where everybody can collaborate to advance the commons.” — substack
  28. webtorrent modules » bittorrent-dht (distributed hash table client) » bittorrent-tracker

    (tracker server/client) » bittorrent-protocol (bittorrent protocol stream) » create-torrent (create .torrent files) » parse-torrent (parse torrent identifiers) » simple-peer (simple WebRTC connections) » many more...
  29. sharing P2P primitives

  30. otrtalk » OTR (off-the-record messaging) » encryption » authentication »

    forward secrecy » uses DHT to discover buddies » telehash » bittorrent (bittorrent-dht)
  31. mafintosh » peerflix (used by popcorn time) » torrent-mount »

    peerwiki
  32. OPEN open source

  33. easy to test

  34. share your stackoverflow snippets

  35. independently versioned

  36. more likely to eventually be "done"

  37. xhr var xhr = require('xhr') xhr('http://example.com', function (err, resp, body)

    { if (err) throw err console.log(body) })
  38. drag-drop var dragDrop = require('drag-drop') dragDrop('#dropTarget', function (files) { console.log(files)

    // got the files })
  39. WebRTC

  40. None
  41. None
  42. simple-peer var SimplePeer = require('simple-peer') var peer1 = new SimplePeer({

    initiator: true }) var peer2 = new SimplePeer() peer1.on('signal', function (data) { peer2.signal(data) }) peer2.on('signal', function (data) { peer1.signal(data) })
  43. simple-peer peer1.send('hey peer2, how is it going?') peer2.on('data', function (data)

    { console.log('got a message from peer1: ' + data) })
  44. browserify

  45. the "browser" field in package.json

  46. webtorrent / package.json { "browser": { "./lib/helper.js": "./lib/helper-browser.js" } }

  47. webtorrent / package.json { "browser": { "bittorrent-tracker": "webtorrent-tracker" } }

  48. jsdom / package.json { "browser": { "canvas": "false" } }

  49. Streaming <video>

  50. http <video> <video src="rickastley.mov"></video> » uses HTTP range requests »

    supports video seeking
  51. what about video from WebRTC? var data = new Buffer(1000000)

    // from remote peer var video = document.querySelector('video') var url = Object.createObjectURL(new Blob([ data ])) video.src = url video.play()
  52. what about streaming? » Blob is immutable » we'll need

    a different approach
  53. MediaSource API

  54. MediaSource API » Allows JavaScript to generate media streams for

    playback » DASH ("Dynamic Adaptive Streaming over HTTP") » Change video quality as network conditions change » Supported in Chrome, IE11, Safari 8, Fx Nightly » Requires H.264 or VP8/9 codecs » Supports seeking
  55. MediaSource API var mediaSource = new MediaSource() var video =

    document.querySelector('video') video.src = window.URL.createObjectURL(mediaSource) var sourceBuffer = mediaSource.addSourceBuffer('video/mp4') file.createReadStream().on('data', function (data) { sourceBuffer.appendBuffer(data) })
  56. How does seeking work? » Requires video files conform to

    ISO MPEG-DASH » Basically NONE currently do » Requires your code to map timecode -> byte offset
  57. <video> seeking video.addEventListener('waiting', function () { var time = video.currentTime

    // stop fetching current byte range var byteOffset = timeToByte(time) // tell all peers to start giving me data from `byteOffset` file.createReadStream().on('data', function (data) { sourceBuffer.appendBuffer(data) }) })
  58. time code -> byte offset » Usually MP4 files are

    conferted to ISO MPEG-DASH ("re-packaged") offline » For WebTorrent, what can we do?
  59. can seeders repackage the video?

  60. for new torrents, yes

  61. for existing torrents?

  62. no, hash verification would fail

  63. can the downloader repackage the video?

  64. yes, with mp4box.js! » process MP4 files in the browser

    » support for progressive parsing
  65. var WebTorrent = require('webtorrent') var client = new WebTorrent() var

    magnetUri = '...' client.add(magnetUri, function (torrent) { var file = torrent.files[0] var video = document.createElement('video') document.body.appendChild(video) file.createReadStream().pipe(video) }) coming soon to webtorrent!
  66. you can always make it work in the browser somehow

  67. JavaScript Standard Style

  68. What to do with good PRs that have poor code

    style?
  69. give feedback, then wait

  70. merge and fix it after

  71. use a linter

  72. super annoying .rc files » .jshintrc » .eslintrc » .jscsrc

    » how to keep them in sync?
  73. standard $ npm install standard -g $ standard Error: Use

    JavaScript Standard Style lib/torrent.js:950:11: Expected '===' and instead saw '=='.
  74. package.json { "name": "my-cool-package", "devDependencies": { "standard": "^3.0.0" }, "scripts":

    { "test": "standard && node tests.js" } }
  75. saves time in two ways: » No configuration. The easiest

    way to enforce consistent style in your module/project. Just drop it in. » Catch style errors before they're submitted in PRs. Saves precious code review time by eliminating back-and-forth between maintainer and contributor.
  76. “Adopting standard style means ranking the importance of community conventions

    higher than personal style, which does not make sense for 100% of projects and development cultures. At the same time, open source can be a hostile place for newbies. Setting up clear, automated contributor expectations makes a project healthier.” – Max Ogden
  77. lots of people loved it » me » max ogden

    » mafintosh » henrik jorteg (&yet) » forrest norvell (npm) » lots more...
  78. please add a config option for X!

  79. lots of people had opinions "Like I said though, this

    is just my opinion..." "could we add X? maybe." "possibly" ";-)"
  80. ;

  81. first rule of javascript: never talk about semicolons

  82. objections to the name » not a TC39 or ECMA

    standard » "How dare you call this standard!"
  83. semistandard » "All the goodness of feross/standard with semicolons sprinkled

    on top"
  84. learn more about WebRTC » www.html5rocks.com/en/tutorials/webrtc/basics » webrtchacks.com » "simple-peer"

    on npm
  85. learn more about webtorrent webtorrent.io webtorrent.io/roadmap