Upgrade to Pro — share decks privately, control downloads, hide ads and more …

wotxr-20190320rzr

 wotxr-20190320rzr

https://social.samsunginter.net/@rzr/101788928870087007# #WoTxR : From #WebOfThings to #XR using #FLOSS : #IoTJs @MozillaIoT @AFrameVr ... @ #LavalVirtual2019

Phil "RzR" Coval

March 20, 2019
Tweet

More Decks by Phil "RzR" Coval

Other Decks in Programming

Transcript

  1. Samsung Open Source Group 1 https://social.samsunginter.net/@rzr #LavalVirtual2019
    WoTxR:
    From Web of Things to XR
    using Free Libre Open Source Software
    Philippe Coval
    Samsung Open Source Group / SRUK
    mailto:[email protected]
    The 21st international Exhibition and Conference
    https://is.gd/webxr #LavalVirtual2019, Laval, France <2019-03-20>

    View Slide

  2. Samsung Open Source Group 2 https://social.samsunginter.net/@rzr #LavalVirtual2019
    $ who is Philippe Coval

    Software engineer for Samsung OSG
    – Belongs to SRUK team, based in Rennes, France
    – Interest: Web of Things with “Privacy by Design”

    Not an XR expert but past 3D experiences*:
    – Studied 3D at University (Toulouse, 2000), more VR/AR at INRIA, Orange
    – Diet3D, OpenGL, VRML/EAI, ARToolkitPlus…

    In 2019 at FOSDEM, met famous XR expert Fabien Benetou:
    – And we hacked WoTxR PoC: bridge between WoT+XR (check @MozHacks)

    Ping me online: https://social.samsunginter.net/@rzr

    View Slide

  3. Samsung Open Source Group 3 https://social.samsunginter.net/@rzr #LavalVirtual2019
    WoTxR: Realtime webthings + XR twins
    https://youtu.be/s3r8pQtzhAU#wotxr-20190320rzr

    View Slide

  4. Samsung Open Source Group 4 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Javascript the language of Web (of Things)

    IoT.js an alternative runtime inspired by Node.js:
    – Powered by JerryScript engine designed for micro-controllers
    – Base features: IO (I2C, GPIO...), Network (HTTP/S, MQTT, WS) …

    Modules: mastodon-lite, generic-sensors-lite
    – Supporting: RT, GNU/Linux ...

    WebThings can be build using webthing-iotjs module:
    – Standalone HTTP servers exposing Mozilla Things API:

    RESTful architecture: read, update operations
    – Can be connected to MozIoT “PrivacyByDesign” gateway
    – Or queried from the browser in SVG, HTML or 3D !

    View Slide

  5. Samsung Open Source Group 5 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Run a “color sensor WebThing” with IoT.js

    Install IoT.js for WebThing-IotJs (GNU/Linux, TizenRT, WLS...)
    – https://github.com/rzr/webthing-iotjs/wiki/IotJs

    git clone https://github.com/samsunginternet/color-sensor-js
    – cd color-sensor-js
    – iotjs lib/tcs34725.js # => log: value=[7779,36778,11173,42766]
    – make -C example/color-sensor-webthing start
    – curl http://localhost:8888/properties/ # => {"color": "#af0695"}

    Or run webthing in the cloud:
    – https://color-sensor-webthing.glitch.me

    View Slide

  6. Samsung Open Source Group 6 https://social.samsunginter.net/@rzr #LavalVirtual2019

    function ColorProperty(thing) {
    webthing.Property.call(this, thing, 'color',
    new webthing.Value(‘#000000’),
    { '@type': 'colorProperty', type: 'string',
    readOnly: true });
    }
    //…

    var thing = new webthing.Thing
    (‘ColorSensor’[‘ColorControl’]);

    thing.addProperty(new ColorProperty(thing));
    //…

    server.start();
    Implementing WebThings
    $ curl http://localhost:8888
    { ...
    "properties":{"color": ...
    "links":[
    {"rel":"property",
    "href":"/properties/color"

    View Slide

  7. Samsung Open Source Group 7 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Sensor updating webthing
    var ColorSensor = require("color-sensor-js/example");
    function ColorProperty(thing) {
    var that = this;
    Webthing.Property.call(..., "color", new webthing.Value...
    this.sensor = new ColorSensor();
    this.sensor.onreading = function() {
    that.value.notifyOfExternalUpdate(self.sensor.color);
    };
    this.sensor.start();
    }
    $ curl http://localhost:8888/properties # => {"color": "#af0695"}

    View Slide

  8. Samsung Open Source Group 8 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Immersive Web

    Not new idea:
    – VRML97 plugins controlled by Java applets (EAI, RMI...)

    But a successful evolution:
    – HTML5, WebGL, Three.js, WebVR, OpenVR, WebXR, OpenXR...

    A-Frame: Free and Libre opensource framework
    – Originally from Mozilla, HTML for VR, Uses Web components
    – Support: VR headsets, GearVR, GearVR2017 (with controller)

    Using most workbrowsers: SamungInternet (svrbrowser), Firefox or chromium based...

    Exokit: Native VR/AR/XR engine for JavaScript (WebGL)

    View Slide

  9. Samsung Open Source Group 9 https://social.samsunginter.net/@rzr #LavalVirtual2019
    A-Frame’s scene in DOM

    <br/>






    radius=".6" position="0 -1 -5">



    View Slide

  10. Samsung Open Source Group 10 https://social.samsunginter.net/@rzr #LavalVirtual2019
    DOM manipulation from Javascript
    <br/>setTimeout(function(){<br/>var el = document.getElementById("color");<br/>el.setAttribute("color", "#00FF00");<br/>}, 1000);<br/>

    <br/>








    View Slide

  11. Samsung Open Source Group 11 https://social.samsunginter.net/@rzr #LavalVirtual2019

    // TODO: Update with your webthing IP in LAN : eg: http://localhost:8888

    const url = "https://samsunginternet-color-sensor-js.glitch.me";
    const name = "color";

    const el = document.getElementById(name); // The light bulb’s sphere
    fetch(url)
    .then(function(response) { return response.json(); })
    .then(function(json) { el.setAttribute(name, json[name]); });

    // Challenge: use PUT to update properties of actuators’
    Updating device property to his 3D twin

    View Slide

  12. Samsung Open Source Group 12 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Let’s boot digital worlds on the web of thing

    Build more webthings using your favorite language

    Install Mozilla gateway in your home network

    Connect your webthings to gateway
    – Control them using the progressive web app

    Try the alternate Progressive Web App to browse them in XR:
    – http://github.com/samsunginternet/webthings-webapp

    More to come:
    – https://github.com/rzr/webthing-iotjs/wiki/XR

    View Slide

  13. Samsung Open Source Group 13 https://social.samsunginter.net/@rzr #LavalVirtual2019
    References

    https://github.com/rzr/webthing-iotjs/wiki/
    – https://github.com/rzr/webthing-iotjs/wiki/Sensor

    https://github.com/SamsungInternet/color-sensor-js

    https://github.com/rzr/webthing-iotjs/wiki/XR
    – https://github.com/tizenteam/aframe-ui-widgets/pull/17#

    https://hacks.mozilla.org/2019/03/connecting-real-things-to-virtual-worlds-using-web/

    https://hub.samsunginter.net/
    – https://github.com/immersive-web/webvr-polyfill

    https://aframe.io/

    https://iot.mozilla.org/

    https://monado.dev/

    https://www.khronos.org/openxr

    https://github.com/webmixedreality/exokit

    https://www.laval-virtual.com/schedule-2019/

    View Slide

  14. 14 https://social.samsunginter.net/@rzr
    Samsung Open Source Group 2019
    Q&A ?
    (or Extras?)
    Ask now or online:
    https://social.samsunginter.net/@rzr

    View Slide

  15. Samsung Open Source Group 15 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Controlling real data & consuming OpenData
    https://youtu.be/OT0Ahuy3Cv4#webthing-iotjs-opendata-20190202rzr

    View Slide

  16. 16 https://social.samsunginter.net/@rzr
    Samsung Open Source Group 2019
    Demo: IoT gateway, ESP MCU, WebApp
    https://youtu.be/vzoUJ-v5h38#webthing-esp8266-webapp-20180602rzr


    https://vimeo.com/273037442

    webthing-esp8266-webapp-20180602rzr

    https://youtu.be/vzoUJ-v5h38

    https://vimeo.com/user12599872/

    webthing-esp8266-webapp-20180602rzr

    View Slide

  17. Samsung Open Source Group 17 https://social.samsunginter.net/@rzr #LavalVirtual2019
    WebThings-WebApp: Tizen or PWA (sbrowser)
    https://youtu.be/S_mHYu5-iYM#webthing-webapp-pwa-20180629rzr
    https://youtu.be/S
    _mHYu5-iYM#we
    bthing-webapp-p
    wa-20180629rzr

    View Slide

  18. Samsung Open Source Group 18 https://social.samsunginter.net/@rzr #LavalVirtual2019
    *3D for the Web: Lessons from the past

    Several attempts before WebGL (2011)

    Y2K: Java applets ?
    – Software rendering

    No OpenGL

    Just Triangles, Lines or pixels: performance impact
    – Portable?

    Applet plugins weren’t opensource first
    – Until, gcj, OpenJDK

    All failed in integration into browsers
    – Bad performance
    – Security policies ?

    Same “death” as shockwave/adobe flash

    In 2019 we can assume that JS won

    View Slide

  19. Samsung Open Source Group 19 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Dynamic 3D scenes

    VRML97 and VRML2
    – Various VRML plugins
    – Scene graph
    – Descriptive script (like PoV-ray)
    – OpenGL support

    HW acceleration
    – Most were closed sources

    Except openvrml
    – Superseded by Web3D (XML)

    Still Java applets for
    – Manipulating scene graph
    – Javascript eventually

    View Slide

  20. Samsung Open Source Group 20 https://social.samsunginter.net/@rzr #LavalVirtual2019
    Real time connectivity

    Before REST (2000)
    – and WebSockets (2011)

    Java RMI vs AJAX
    – Security policies
    – Custom ports
    – Proxy blockers?

    Sensors (before W3C G-sensors)
    – Costly
    – Closed source

    Drivers

    Middlewares

    Second life was native

    View Slide