Mobile Day Mexico 2017

Transcript

1. VR/AR/Mixed Reality Rabimba Karanjai @rabimba
2. VR/AR/MR in a Nutshell Rabimba Karanjai, Mozilla Research Mixed Reality Mozilla Tech Speaker @rabimba
3. What the Heck Doodle? ARKit ARCore MR VR XR AR Oculus Sony Vive Meta HoloLens
4. What You’ll Learn Today • What is VR / AR / MR? • Why is this all happening now? • What do I really need to know?
5. What is VR/AR/MR? VR: Virtual Reality AR: Augmented Reality MR: Mixed Reality XR: We don’t know Reality
6. Virtual Reality • Completely immersive • Wide field of view • Fast refresh rate (90+fps) • Fast sensor refresh
7. Augmented Reality • Overlaid on real world • Anchored to real world • Non-immersive
9. Partial AR
13. Why Now?
14. The Long History of VR/AR/MR Ivan Sutherland, MIT Sword of Damocles (1968) 3D wireframe rooms
16. 1980s: VPL • Data Glove • Eye Phone • Data Suit
17. Mattel’s Nintendo Power Glove
18. 1990s: Virtuality
20. Virtual Reality Markup Language • Scene based • In the browser • Slow to download • Slow to render
22. Smartphone Era
23. Smartphone+ Era • Dual cameras • Depth sensing cameras • Machine Learning in hardware • Even more powerful GPU & CPU
24. What do I need to know?
25. Tech Stack for VR & AR Input: Sensors & Camera Processing: Image & Sensor Fusion Drawing: 3D Graphics API Application: Framework
26. Input: Sensors & Camera • Accelerometer: relative to gravity • Magnetometer: relative to earth’s magnetic field • Gyroscope: relative angle movement ◦ 9DOF sensor breakout board for 15$ @ Adafruit! • Optical Camera: standard phone camera • Stereo Camera: dual camera setup • Depth Sensing Camera: Kinect, Face ID
27. Processing the 3D Environment • Markers: Symbols or Beacons • SLAM: Simultaneous Localization and Mapping • MonoSLAM: SLAM in real-time with just one camera
28. HTC Vive Sensor Beacon two beacons + headset sensors = highly accurate
29. Processing: VR/AR APIs • Vuforia • ARToolkit • WebVR • ARKit • ARCore • WebXR
30. Vuforia
31. ARToolkit
32. WebVR • Mobile: used by web apps for Google Cardboard on any phone. • Desktop: works with major HMDs • Shipping: Firefox for Windows • Beta: Chrome, FF for other platforms
33. ARKit
34. Tango ARCore
35. WebXR: Work In Progress! • Proposed Future Web Standard • Extend WebVR with new features • Work with ARKit & ARCore • Polyfill for old browsers using WebRTC + sensors https://github.com/mozilla/webxr-api https://github.com/mozilla/webxr-polyfill
36. Drawing: Graphics APIs • Medium Level ◦ OpenGL (ES) ◦ DirectX ◦ WebGL • Low Level ◦ Metal ◦ Vulkan ◦ Direct X 12 • High Level ◦ Unity ◦ Unreal ◦ Three.js ◦ Babylon.js
37. Application: Frameworks • Scene Kit • Cardboard & Daydream • Hololens Academy • React VR • ArgonJS • A-Frame
38. What do you *really* need to know? • Start with web stack • Follow Mozilla and Google and Microsoft • Play with A-Frame • Get a Cardboard viewer.
39. What can I do? • Experiment! We are still in the Model T days! • No one knows what the killer app will be. • First new medium in decades • Go play!
40. State of WebVR and Aframe The web framework for building VR experiences @rabimba | Mozilla VR | aframe.io
41. Friction of VR Ecosystems Gatekeepers Installs Closed
42. WebVR An open virtual reality platform with the advantages of the Web Open Connected Instant
43. Browser APIs that enable WebGL rendering to headsets and access to VR sensors
44. https://w3c.github.io/webvr/
45. Firefox Nightly Chromium (Experimental) Samsung Internet Mobile Polyfill
46. A-Frame A declarative framework for building virtual reality experiences on the Web
47. Hello World <a-scene> <a-box color="#4CC3D9" position="-1 0.5 -3" rotation="0 45 0"></a-box> <a-cylinder color="#FFC65D" position="1 0.75 -3" radius="0.5" height="1.5 <a-sphere color="#EF2D5E" position="0 1.25 -5" radius="1.25"></a-sphere> <a-plane color="#7BC8A4" position="0 0 -4" rotation="-90 0 0" width="4" h <a-sky color="#ECECEC"></a-sky> </a-scene>
48. Hello World
49. <a-animation> <a-box> <a-camera> <a-circle> <a-collada-model> <a-cone> <a-cursor> <a-curvedimage> <a-cylinder> <a-dodecahedron> <a-isocahedron> <a-image> <a-light> <a-obj-model> <a-plane> <a-ring> <a-tetrahedron> <a-torus> <a-torus-knot> <a-sky> <a-sound> <a-sphere> <a-videosphere>
50. Registry Curated collection of A-Frame components
52. Registry Curated collection of A-Frame components
53. Motion Capture Record then develop room scale VR on the go Remix Glitch
54. Community https://aframe.io/blog/
55. Powerful and Extensible
57. Art - A-Painter @mozillavr
58. Journalism - Fear of the Sky Amnesty International UK
59. Journalism - Journey to Mars The Washington Post
60. Guri VR Build your own world in VR
61. Sandbox - Aincraft @andgokevin
62. Aincraft Lets play
63. Data Visualization - Adit @datatitian
64. Gaming - A-Blast @mozillavr
65. AR - AR.js + A-Frame @jerome_etienne
66. Come build WebVR with us! aframe.io/community
67. Thank you @rabimba | rabimba@mozilla.com| aframe.io