Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Lessons from the experimental edge of technology

Lessons from the experimental edge of technology

When asked if I wanted to build a Chrome Experiment: a thing that shows off some of the new technology coming to browsers - of course I said yes! Then promptly I bit off the biggest bite of tech I could get my hands on: WebRTC, two way video, peer connection API, orientation events, CSS3 animations, Three.js: generally a lot of tech I didn't really understand!

This talk is about the experience of working with WebRTC for video & data streaming and creating a game with embedded video. How some of the problems I encountered blew my schedule to pieces and perhaps how I could have done things differently.

Remy Sharp

May 29, 2013
Tweet

More Decks by Remy Sharp

Other Decks in Technology

Transcript

  1. Lessons on the experimental edge of web technology Remy Sharp

    / @rem http://www.flickr.com/photos/spaztacular/3537911581/
  2. Three simple tasks 1. Get a room (aka pin for

    my game) 2. Listen and action remote connections 3. Add my video stream
  3. // `pin` is the unique game id // basically a

    room name rtc.connect('wss:localhost:8000', pin); 1. Get a room
  4. rtc.on('add remote stream', function(stream, id) { // `stream` is the

    video stream // 'remote' refers to a video element on the page // with the id of 'remote' rtc.attachStream(stream, 'remote'); // in the actual game we'd trigger an event to // notify that the remote stream has been added, // and we'd start playing }); 2. Listen for join
  5. // this will give the video & audio stream //

    to any peers rtc.createStream({ 'video': true, 'audio': true }, function(stream) { // since this game doesn't show own our // face, we do nothing with the stream }); 3. Add my stream
  6. navigator.getUserMedia({ 'video': { 'mandatory': { maxWidth: 160, maxHeight: 90 }

    } }, function(stream) { video.src = URL.createObjectURL(stream); // will be true when stream is running // in video elem console.log(video.videoWidth === 160); });
  7. var express = require('express'), app = express(), server = require('http').createServer(app),

    webRTC = require('webrtc.io').listen(server); server.listen(8000); Bind webrtc.io. Bosh.
  8. function dataChannelOpen() { for (var id in rtc.dataChannels) { //

    note: string based ready states // (rather than usual ints) if (rtc.dataChannels[id].readyState == 'open') { return true; } } return false; } Test for open channel
  9. Streaming shit loads of event data • If you can:

    avoid it • Head Shots only sends orientation event change • Peer only gets a single message
  10. Mechanics 1.Open Websocket for game discovery 2.Connect video & audio

    stream 3.Test if data channel is "open" 4.Allow game to start...
  11. How it originally worked 1. Welcome screen with join &

    start button 2. Start - automatically created a new game (now in waiting mode) 3. Join - prompted the user for a pin code 4. Waiting - this used a hanging XHR request 5. Play - wherein both players prompted for access to their camera
  12. SPA

  13. Crazy latest tech also means latest standards. •Micro DOM &

    event library: min.js •Extensive use of HTML5's classList & dataset •Micro data binding: bind.js •Super minimal xhr
  14. // `pin` is the unique game id var $ =

    document.querySelectorAll.bind(document); Node.prototype.on = Node.prototype.addEventListener; $('#somelink')[0].on('touchstart', handleTouch); Less is $$$$ https://github.com/remy/min.js
  15. var callback; if (typeof mapping[path.join('.')] === 'function') { callback =

    mapping[path.join('.')]; } else { callback = function (value) { if (callback.elements) { forEach.call(callback.elements, function (element) { element.innerHTML = value; }); } }; callback.elements = $(mapping[path.join('.')] || '☺'); } Object.defineProperty(target, key, { set: function (v) { value = v; if (callback) { callback(value); } }, get: function () { return value; } }); target[key] = value; Data binding ~40 lines
  16. var callback; if (typeof mapping[path.join('.')] === 'function') { callback =

    mapping[path.join('.')]; } else { callback = function (value) { if (callback.elements) { forEach.call(callback.elements, function (element) { element.innerHTML = value; }); } }; callback.elements = $(mapping[path.join('.')] || '☺'); } Object.defineProperty(target, key, { set: function (v) { value = v; if (callback) { callback(value); } }, get: function () { return value; } }); target[key] = value; Setter/getter action
  17. var callback; if (typeof mapping[path.join('.')] === 'function') { callback =

    mapping[path.join('.')]; } else { callback = function (value) { if (callback.elements) { forEach.call(callback.elements, function (element) { element.innerHTML = value; }); } }; callback.elements = $(mapping[path.join('.')] || '☺'); } Object.defineProperty(target, key, { set: function (v) { value = v; if (callback) { callback(value); } }, get: function () { return value; } }); target[key] = value; innerHTML setting
  18. var callback; if (typeof mapping[path.join('.')] === 'function') { callback =

    mapping[path.join('.')]; } else { callback = function (value) { if (callback.elements) { forEach.call(callback.elements, function (element) { element.innerHTML = value; }); } }; callback.elements = $(mapping[path.join('.')] || '☺'); } Object.defineProperty(target, key, { set: function (v) { value = v; if (callback) { callback(value); } }, get: function () { return value; } }); target[key] = value; Initialise
  19. game = new Bind({ me: { // original object score:

    10 } }, { 'me.score': '#myscore' }); // DOM updates game.me.score = 20; https://github.com/remy/bind Binding
  20. game = new Bind(result.data, { 'me.score': '#myscore', 'them.score': function (value)

    { $.trigger('theirScore', value); }, 'turn': function (myturn) { turnEl.dataset.turn = myturn; $.trigger('myturn', myturn); } }); https://github.com/remy/bind
  21. The biggies •Newness of WebRTC ate way more time than

    I anticipated •Google engineers are there to help •Maybe the horse came before the cart?
  22. Is WebRTC ready for prime time? •I'm not so sure

    •Support and interop is being aggressively worked on...but Microsoft...Apple? •Mobile looking hopeful •Give it 12 months* * Check expiry date