$30 off During Our Annual Pro Sale. View Details »

Mixed Realities in your browsers

Mixed Realities in your browsers

The Slides for my Session at Codemotion Amsterdam 2018.

Carsten Sandtner

May 09, 2018
Tweet

More Decks by Carsten Sandtner

Other Decks in Technology

Transcript

  1. Mixed Realities in your Browsers
    Carsten Sandtner (@casarock)
    AMSTERDAM | MAY 8-9, 2018

    View Slide

  2. about://me
    Carsten Sandtner
    Technical Director 

    mediaman //
    Web development since 1998
    @casarock

    https://www.casa-rock.de


    View Slide

  3. View Slide

  4. View Slide

  5. LET’S DO SOME TIME TRAVEL!

    View Slide

  6. VIEWMASTER

    View Slide

  7. VIRTUALBOY

    View Slide

  8. Oculus Rift
    • Kickstarter 2013
    • Disruption in VR
    • Many other followed
    • HTC Vive
    • PS VR
    • etc.
    Image by Oculus VR, LLC

    View Slide

  9. PROJECT HEADSIGHT

    View Slide

  10. MICROSOFT HOLOLENS

    View Slide

  11. 2017 - The AR-Revolution

    View Slide

  12. BASIC CONCEPTS - PART 1:
    VIRTUAL REALITY

    View Slide

  13. https://developer.mozilla.org/en-US/docs/Web/API/WebVR_API/Concepts

    View Slide

  14. https://developer.mozilla.org/en-US/docs/Web/API/WebVR_API/Concepts

    View Slide

  15. View Slide

  16. SENSORS
    https://developer.mozilla.org/en-US/docs/Web/API/WebVR_API/Concepts

    View Slide

  17. FIELD OF VIEW
    https://developer.mozilla.org/en-US/docs/Web/API/WebVR_API/Concepts

    View Slide

  18. STEREOSCOPIC IMAGES
    https://developer.mozilla.org/en-US/docs/Web/API/WebVR_API/Concepts

    View Slide

  19. BASIC CONCEPTS PART 1.1:
    WHAT IS WEBVR?

    View Slide

  20. Browser APIs that enable
    WebGL rendering to headsets
    and access to VR sensors

    View Slide

  21. Available APIs
    VRDisplay
    VRDisplayCapabilities
    VRDisplayEvent
    VRFrameData
    VRPose
    VREyeParameters
    VRFieldOfView
    VRLayerInit
    VRStageParameters

    View Slide

  22. Get VR Displays
    var frameData = new VRFrameData();
    var vrDisplay;
    navigator.getVRDisplays().then(function (displays) {
    vrDisplay = displays[0];
    console.log('Display found');
    // Starting the presentation when the button
    // is clicked: It can only be called in response to a user gesture
    btn.addEventListener('click', function () {
    vrDisplay.requestPresent([{ source: canvas }]).then(function () {
    drawVRScene();
    });
    });
    });

    View Slide

  23. Draw your VR Scene
    function drawVRScene() {
    // WebVR: Request the next frame of the animation
    vrSceneFrame = vrDisplay.requestAnimationFrame(drawVRScene);
    // Populate frameData with the data of the next frame to display
    vrDisplay.getFrameData(frameData);
    // You can get the position, orientation, etc.
    // of the display from the current frame's pose
    // curFramePose is a VRPose object
    var curFramePose = frameData.pose;
    var curPos = curFramePose.position;
    var curOrient = curFramePose.orientation;
    // Here happens some WebGL Stuff!
    // WebVR: Indicate that we are ready to present the rendered frame to the VR display
    vrDisplay.submitFrame();
    }

    View Slide

  24. Import WebVR polyfill
    Set up camera
    Initialize scene
    Set up lights
    Declare and pass canvas
    Install VREffect
    Listen to window resize
    Create render loop
    Instantiate renderer
    Figure out responsiveness
    Preload assets
    Deal with metatags and mobile

    View Slide

  25. A-FRAME

    View Slide

  26. Use markup to create VR
    experiences that work across
    desktop, iOS, Android, and the
    Oculus Rift.

    View Slide

  27. Basic VR Scene with aframe




    Hello, World!




    depth="1" position="1 1 1" rotation="0 0 0" scale="1 1 1">





    View Slide

  28. Basic VR Scene with animation


    . . .


    depth="1" position="1 1 1" rotation="0 0 0"
    scale="1 1 1">
    repeat="indefinite"
    to="0 180 0">




    View Slide

  29. View Slide

  30. ENTITY COMPONENT SYSTEM

    View Slide

  31. Composing an Entity
    geometry="primitive: sphere; radius: 1.5"
    material="color: #343434; roughness: 0.4; sphericalEnvMap: #texture">

    View Slide

  32. Composing an Entity
    geometry="primitive: sphere; radius: 1.5"
    material="color: #343434; roughness: 0.4; sphericalEnvMap: #texture"
    position="-1 2 4" rotation="45 0 90" scale="2 2 2">

    View Slide

  33. Composing an Entity
    geometry="primitive: sphere; radius: 1.5"
    material="color: #343434; roughness: 0.4; sphericalEnvMap: #texture"
    position="-1 2 4" rotation="45 0 90" scale="2 2 2"
    animation="property: rotation; loop: true; to: 0 360 0"
    movement-pattern="type: spline; speed: 4">

    View Slide

  34. AWESOME USE CASES

    View Slide

  35. http://www.360syria.com/

    View Slide

  36. 36
    https://aframe.io/a-painter/

    View Slide

  37. 37
    https://pepsigoback.com/

    View Slide

  38. View Slide

  39. BASIC CONCEPTS - PART 2
    MIXED REALITY

    View Slide

  40. Reality
    Augmented
    Reality
    Mixed
    Reality
    Virtual
    Reality
    Mixed Reality?

    View Slide

  41. https://www.gizmodo.com.au/2017/04/lemmings-in-hololens-adds-new-dimension-to-animal-suicide/

    View Slide

  42. https://thenextweb.com/insider/2016/06/16/the-force-is-strong-with-this-leap/

    View Slide

  43. Reality
    Augmented
    Reality
    Mixed
    Reality
    Virtual
    Reality
    XR -
    whateveryouwant
    Reality
    XR!

    View Slide

  44. TYPES OF AR

    View Slide

  45. MARKER BASED AR
    https://www.flickr.com/photos/piair/8679229646/

    View Slide

  46. MARKERLESS AR
    https://www.flickr.com/photos/wikitude/30944213892/

    View Slide

  47. TRACKING

    View Slide

  48. BASIC CONCEPTS PART 2.1:
    WHAT IS WEBXR?

    View Slide

  49. WebVR 2 API

    WebXR Device API

    View Slide

  50. WebXR
    • is not backwards compatible with WebVR!
    • AR is not in focus on start - „Coming soon!“
    • Focused on tracking based AR (Phone-based)

    View Slide

  51. WebXR
    navigator.xr.requestDevice().then(device => {
    // Resolves to an XRDevice if one is available, or to null otherwise.
    if (device) {
    onXRAvailable(device);
    }
    }).catch(error => {
    // An error occurred while requesting an XRDevice.
    console.error('Unable to request an XR device: ', error);
    });

    View Slide

  52. https://www.w3.org/blog/2018/01/towards-the-immersive-web/

    View Slide

  53. A-FRAME GOES XR

    View Slide

  54. HTTPS://GITHUB.COM/MOZILLA/AFRAME-XR/
    https://blog.mozvr.com/responsive-webxr-a-painter-xr/

    View Slide

  55. WebXR with A-Frame

    View Slide

  56. https://github.com/mozilla/aframe-xr/

    View Slide

  57. This is just the
    beginning
    • WebXR is not ready
    • APIs are in flux and are
    changing constantly
    • The web is joining the
    hype train!
    https://twitter.com/LeapMotion/status/988749463215857664

    View Slide

  58. ADVANTAGES OF WEBXR

    View Slide

  59. WebXR is progressive

    View Slide

  60. View Slide

  61. NO GATEKEEPERS

    View Slide

  62. INSTANT

    View Slide

  63. CONNECTED

    View Slide

  64. View Slide

  65. https://mixedreality.mozilla.org/
    https://mixedreality.mozilla.org/
    https://aframe.io
    https://github.com/immersive-web/webxr
    https://immersive-web.github.io/webxr-samples/
    https://immersive-web.github.io/webxr/
    https://github.com/mozilla/aframe-xr
    Some Links

    View Slide

  66. THAT’S ALL. THANKS!
    @casarock
    https://github.com/casarock
    https://casa-rock.de/

    View Slide