wotxr-20190320rzr

https://social.samsunginter.net/@rzr/101788928870087007# #WoTxR : From #WebOfThings to #XR using #FLOSS : #IoTJs @MozillaIoT @AFrameVr ... @ #LavalVirtual2019

Transcript

1. Samsung Open Source Group 1 https://social.samsunginter.net/@rzr #LavalVirtual2019
   WoTxR:
   From Web of Things to XR
   using Free Libre Open Source Software
   Philippe Coval
   Samsung Open Source Group / SRUK
   mailto:p.coval@samsung.com
   The 21st international Exhibition and Conference
   https://is.gd/webxr #LavalVirtual2019, Laval, France <2019-03-20>
   

   View Slide

2. Samsung Open Source Group 2 https://social.samsunginter.net/@rzr #LavalVirtual2019
   $ who is Philippe Coval
   ●
   Software engineer for Samsung OSG
   – Belongs to SRUK team, based in Rennes, France
   – Interest: Web of Things with “Privacy by Design”
   ●
   Not an XR expert but past 3D experiences*:
   – Studied 3D at University (Toulouse, 2000), more VR/AR at INRIA, Orange
   – Diet3D, OpenGL, VRML/EAI, ARToolkitPlus…
   ●
   In 2019 at FOSDEM, met famous XR expert Fabien Benetou:
   – And we hacked WoTxR PoC: bridge between WoT+XR (check @MozHacks)
   ●
   Ping me online: https://social.samsunginter.net/@rzr
   

   View Slide

3. Samsung Open Source Group 3 https://social.samsunginter.net/@rzr #LavalVirtual2019
   WoTxR: Realtime webthings + XR twins
   https://youtu.be/s3r8pQtzhAU#wotxr-20190320rzr
   

   View Slide

4. Samsung Open Source Group 4 https://social.samsunginter.net/@rzr #LavalVirtual2019
   Javascript the language of Web (of Things)
   ●
   IoT.js an alternative runtime inspired by Node.js:
   – Powered by JerryScript engine designed for micro-controllers
   – Base features: IO (I2C, GPIO...), Network (HTTP/S, MQTT, WS) …
   ●
   Modules: mastodon-lite, generic-sensors-lite
   – Supporting: RT, GNU/Linux ...
   ●
   WebThings can be build using webthing-iotjs module:
   – Standalone HTTP servers exposing Mozilla Things API:
   ●
   RESTful architecture: read, update operations
   – Can be connected to MozIoT “PrivacyByDesign” gateway
   – Or queried from the browser in SVG, HTML or 3D !
   

   View Slide

5. Samsung Open Source Group 5 https://social.samsunginter.net/@rzr #LavalVirtual2019
   Run a “color sensor WebThing” with IoT.js
   ●
   Install IoT.js for WebThing-IotJs (GNU/Linux, TizenRT, WLS...)
   – https://github.com/rzr/webthing-iotjs/wiki/IotJs
   ●
   git clone https://github.com/samsunginternet/color-sensor-js
   – cd color-sensor-js
   – iotjs lib/tcs34725.js # => log: value=[7779,36778,11173,42766]
   – make -C example/color-sensor-webthing start
   – curl http://localhost:8888/properties/ # => {"color": "#af0695"}
   ●
   Or run webthing in the cloud:
   – https://color-sensor-webthing.glitch.me
   

   View Slide

6. Samsung Open Source Group 6 https://social.samsunginter.net/@rzr #LavalVirtual2019
   ●
   function ColorProperty(thing) {
   webthing.Property.call(this, thing, 'color',
   new webthing.Value(‘#000000’),
   { '@type': 'colorProperty', type: 'string',
   readOnly: true });
   }
   //…
   ●
   var thing = new webthing.Thing
   (‘ColorSensor’[‘ColorControl’]);
   ●
   thing.addProperty(new ColorProperty(thing));
   //…
   ●
   server.start();
   Implementing WebThings
   $ curl http://localhost:8888
   { ...
   "properties":{"color": ...
   "links":[
   {"rel":"property",
   "href":"/properties/color"
   …
   

   View Slide

7. Samsung Open Source Group 7 https://social.samsunginter.net/@rzr #LavalVirtual2019
   Sensor updating webthing
   var ColorSensor = require("color-sensor-js/example");
   function ColorProperty(thing) {
   var that = this;
   Webthing.Property.call(..., "color", new webthing.Value...
   this.sensor = new ColorSensor();
   this.sensor.onreading = function() {
   that.value.notifyOfExternalUpdate(self.sensor.color);
   };
   this.sensor.start();
   }
   $ curl http://localhost:8888/properties # => {"color": "#af0695"}
   

   View Slide

8. Samsung Open Source Group 8 https://social.samsunginter.net/@rzr #LavalVirtual2019
   Immersive Web
   ●
   Not new idea:
   – VRML97 plugins controlled by Java applets (EAI, RMI...)
   ●
   But a successful evolution:
   – HTML5, WebGL, Three.js, WebVR, OpenVR, WebXR, OpenXR...
   ●
   A-Frame: Free and Libre opensource framework
   – Originally from Mozilla, HTML for VR, Uses Web components
   – Support: VR headsets, GearVR, GearVR2017 (with controller)
   ●
   Using most workbrowsers: SamungInternet (svrbrowser), Firefox or chromium based...
   ●
   Exokit: Native VR/AR/XR engine for JavaScript (WebGL)
   

   View Slide

9. Samsung Open Source Group 9 https://social.samsunginter.net/@rzr #LavalVirtual2019
   A-Frame’s scene in DOM
   ●
   <br/>
   ●
   
   ●
   
   
   ●
   radius=".6" position="0 -1 -5">
   
   ●
   
   

   View Slide

10. Samsung Open Source Group 10 https://social.samsunginter.net/@rzr #LavalVirtual2019
    DOM manipulation from Javascript
    <br/>setTimeout(function(){<br/>var el = document.getElementById("color");<br/>el.setAttribute("color", "#00FF00");<br/>}, 1000);<br/>
    ●
    <br/>
    ●
    
    ●
    
    
    ●
    ●
    
    

    View Slide

11. Samsung Open Source Group 11 https://social.samsunginter.net/@rzr #LavalVirtual2019
    ●
    // TODO: Update with your webthing IP in LAN : eg: http://localhost:8888
    ●
    const url = "https://samsunginternet-color-sensor-js.glitch.me";
    const name = "color";
    ●
    const el = document.getElementById(name); // The light bulb’s sphere
    fetch(url)
    .then(function(response) { return response.json(); })
    .then(function(json) { el.setAttribute(name, json[name]); });
    ●
    // Challenge: use PUT to update properties of actuators’
    Updating device property to his 3D twin
    

    View Slide

12. Samsung Open Source Group 12 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Let’s boot digital worlds on the web of thing
    ●
    Build more webthings using your favorite language
    ●
    Install Mozilla gateway in your home network
    ●
    Connect your webthings to gateway
    – Control them using the progressive web app
    ●
    Try the alternate Progressive Web App to browse them in XR:
    – http://github.com/samsunginternet/webthings-webapp
    ●
    More to come:
    – https://github.com/rzr/webthing-iotjs/wiki/XR
    

    View Slide

13. Samsung Open Source Group 13 https://social.samsunginter.net/@rzr #LavalVirtual2019
    References
    ●
    https://github.com/rzr/webthing-iotjs/wiki/
    – https://github.com/rzr/webthing-iotjs/wiki/Sensor
    ●
    https://github.com/SamsungInternet/color-sensor-js
    ●
    https://github.com/rzr/webthing-iotjs/wiki/XR
    – https://github.com/tizenteam/aframe-ui-widgets/pull/17#
    ●
    https://hacks.mozilla.org/2019/03/connecting-real-things-to-virtual-worlds-using-web/
    ●
    https://hub.samsunginter.net/
    – https://github.com/immersive-web/webvr-polyfill
    ●
    https://aframe.io/
    ●
    https://iot.mozilla.org/
    ●
    https://monado.dev/
    ●
    https://www.khronos.org/openxr
    ●
    https://github.com/webmixedreality/exokit
    ●
    https://www.laval-virtual.com/schedule-2019/
    

    View Slide

14. 14 https://social.samsunginter.net/@rzr
    Samsung Open Source Group 2019
    Q&A ?
    (or Extras?)
    Ask now or online:
    https://social.samsunginter.net/@rzr
    

    View Slide

15. Samsung Open Source Group 15 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Controlling real data & consuming OpenData
    https://youtu.be/OT0Ahuy3Cv4#webthing-iotjs-opendata-20190202rzr
    

    View Slide

16. 16 https://social.samsunginter.net/@rzr
    Samsung Open Source Group 2019
    Demo: IoT gateway, ESP MCU, WebApp
    https://youtu.be/vzoUJ-v5h38#webthing-esp8266-webapp-20180602rzr
    ●
    ●
    https://vimeo.com/273037442
    ●
    webthing-esp8266-webapp-20180602rzr
    ●
    https://youtu.be/vzoUJ-v5h38
    ●
    https://vimeo.com/user12599872/
    ●
    webthing-esp8266-webapp-20180602rzr
    

    View Slide

17. Samsung Open Source Group 17 https://social.samsunginter.net/@rzr #LavalVirtual2019
    WebThings-WebApp: Tizen or PWA (sbrowser)
    https://youtu.be/S_mHYu5-iYM#webthing-webapp-pwa-20180629rzr
    https://youtu.be/S
    _mHYu5-iYM#we
    bthing-webapp-p
    wa-20180629rzr
    

    View Slide

18. Samsung Open Source Group 18 https://social.samsunginter.net/@rzr #LavalVirtual2019
    *3D for the Web: Lessons from the past
    ●
    Several attempts before WebGL (2011)
    ●
    Y2K: Java applets ?
    – Software rendering
    ●
    No OpenGL
    ●
    Just Triangles, Lines or pixels: performance impact
    – Portable?
    ●
    Applet plugins weren’t opensource first
    – Until, gcj, OpenJDK
    ●
    All failed in integration into browsers
    – Bad performance
    – Security policies ?
    ●
    Same “death” as shockwave/adobe flash
    ●
    In 2019 we can assume that JS won
    

    View Slide

19. Samsung Open Source Group 19 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Dynamic 3D scenes
    ●
    VRML97 and VRML2
    – Various VRML plugins
    – Scene graph
    – Descriptive script (like PoV-ray)
    – OpenGL support
    ●
    HW acceleration
    – Most were closed sources
    ●
    Except openvrml
    – Superseded by Web3D (XML)
    ●
    Still Java applets for
    – Manipulating scene graph
    – Javascript eventually
    

    View Slide

20. Samsung Open Source Group 20 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Real time connectivity
    ●
    Before REST (2000)
    – and WebSockets (2011)
    ●
    Java RMI vs AJAX
    – Security policies
    – Custom ports
    – Proxy blockers?
    ●
    Sensors (before W3C G-sensors)
    – Costly
    – Closed source
    ●
    Drivers
    ●
    Middlewares
    ●
    Second life was native
    

    View Slide