wotxr-20190320rzr

https://social.samsunginter.net/@rzr/101788928870087007# #WoTxR : From #WebOfThings to #XR using #FLOSS : #IoTJs @MozillaIoT @AFrameVr ... @ #LavalVirtual2019

Transcript

1. Samsung Open Source Group 1 https://social.samsunginter.net/@rzr #LavalVirtual2019 WoTxR: From Web of Things to XR using Free Libre Open Source Software Philippe Coval Samsung Open Source Group / SRUK mailto:p.coval@samsung.com The 21st international Exhibition and Conference https://is.gd/webxr #LavalVirtual2019, Laval, France <2019-03-20>
2. Samsung Open Source Group 2 https://social.samsunginter.net/@rzr #LavalVirtual2019 $ who is Philippe Coval • Software engineer for Samsung OSG – Belongs to SRUK team, based in Rennes, France – Interest: Web of Things with “Privacy by Design” • Not an XR expert but past 3D experiences*: – Studied 3D at University (Toulouse, 2000), more VR/AR at INRIA, Orange – Diet3D, OpenGL, VRML/EAI, ARToolkitPlus… • In 2019 at FOSDEM, met famous XR expert Fabien Benetou: – And we hacked WoTxR PoC: bridge between WoT+XR (check @MozHacks) • Ping me online: https://social.samsunginter.net/@rzr
3. Samsung Open Source Group 3 https://social.samsunginter.net/@rzr #LavalVirtual2019 WoTxR: Realtime webthings + XR twins https://youtu.be/s3r8pQtzhAU#wotxr-20190320rzr
4. Samsung Open Source Group 4 https://social.samsunginter.net/@rzr #LavalVirtual2019 Javascript the language of Web (of Things) • IoT.js an alternative runtime inspired by Node.js: – Powered by JerryScript engine designed for micro-controllers – Base features: IO (I2C, GPIO...), Network (HTTP/S, MQTT, WS) … • Modules: mastodon-lite, generic-sensors-lite – Supporting: RT, GNU/Linux ... • WebThings can be build using webthing-iotjs module: – Standalone HTTP servers exposing Mozilla Things API: • RESTful architecture: read, update operations – Can be connected to MozIoT “PrivacyByDesign” gateway – Or queried from the browser in SVG, HTML or 3D !
5. Samsung Open Source Group 5 https://social.samsunginter.net/@rzr #LavalVirtual2019 Run a “color sensor WebThing” with IoT.js • Install IoT.js for WebThing-IotJs (GNU/Linux, TizenRT, WLS...) – https://github.com/rzr/webthing-iotjs/wiki/IotJs • git clone https://github.com/samsunginternet/color-sensor-js – cd color-sensor-js – iotjs lib/tcs34725.js # => log: value=[7779,36778,11173,42766] – make -C example/color-sensor-webthing start – curl http://localhost:8888/properties/ # => {"color": "#af0695"} • Or run webthing in the cloud: – https://color-sensor-webthing.glitch.me
6. Samsung Open Source Group 6 https://social.samsunginter.net/@rzr #LavalVirtual2019 • function ColorProperty(thing) { webthing.Property.call(this, thing, 'color', new webthing.Value(‘#000000’), { '@type': 'colorProperty', type: 'string', readOnly: true }); } //… • var thing = new webthing.Thing (‘ColorSensor’[‘ColorControl’]); • thing.addProperty(new ColorProperty(thing)); //… • server.start(); Implementing WebThings $ curl http://localhost:8888 { ... "properties":{"color": ... "links":[ {"rel":"property", "href":"/properties/color" …
7. Samsung Open Source Group 7 https://social.samsunginter.net/@rzr #LavalVirtual2019 Sensor updating webthing var ColorSensor = require("color-sensor-js/example"); function ColorProperty(thing) { var that = this; Webthing.Property.call(..., "color", new webthing.Value... this.sensor = new ColorSensor(); this.sensor.onreading = function() { that.value.notifyOfExternalUpdate(self.sensor.color); }; this.sensor.start(); } $ curl http://localhost:8888/properties # => {"color": "#af0695"}
8. Samsung Open Source Group 8 https://social.samsunginter.net/@rzr #LavalVirtual2019 Immersive Web • Not new idea: – VRML97 plugins controlled by Java applets (EAI, RMI...) • But a successful evolution: – HTML5, WebGL, Three.js, WebVR, OpenVR, WebXR, OpenXR... • A-Frame: Free and Libre opensource framework – Originally from Mozilla, HTML for VR, Uses Web components – Support: VR headsets, GearVR, GearVR2017 (with controller) • Using most workbrowsers: SamungInternet (svrbrowser), Firefox or chromium based... • Exokit: Native VR/AR/XR engine for JavaScript (WebGL)
9. Samsung Open Source Group 9 https://social.samsunginter.net/@rzr #LavalVirtual2019 A-Frame’s scene in DOM • <script src="https://aframe.io/releases/0.9.0/aframe.min.js"> </script> • <a-scene> • <a-sphere id="color" color="#FF0000" position="0 0 -5"> </a-sphere> • <a-cylinder id="base" color="#777" radius=".6" position="0 -1 -5"> </a-cylinder> • </a-scene>
10. Samsung Open Source Group 10 https://social.samsunginter.net/@rzr #LavalVirtual2019 DOM manipulation from Javascript <script> setTimeout(function(){ var el = document.getElementById("color"); el.setAttribute("color", "#00FF00"); }, 1000); </script> • <script src="https://aframe.io/releases/0.9.0/aframe.min.js"> </script> • <a-scene> • <a-sphere id="color" color="#FF0000" position="0 0 -5"> </a-sphere> • <a-cylinder ... • </a-scene>
11. Samsung Open Source Group 11 https://social.samsunginter.net/@rzr #LavalVirtual2019 • // TODO: Update with your webthing IP in LAN : eg: http://localhost:8888 • const url = "https://samsunginternet-color-sensor-js.glitch.me"; const name = "color"; • const el = document.getElementById(name); // The light bulb’s sphere fetch(url) .then(function(response) { return response.json(); }) .then(function(json) { el.setAttribute(name, json[name]); }); • // Challenge: use PUT to update properties of actuators’ Updating device property to his 3D twin
12. Samsung Open Source Group 12 https://social.samsunginter.net/@rzr #LavalVirtual2019 Let’s boot digital worlds on the web of thing • Build more webthings using your favorite language • Install Mozilla gateway in your home network • Connect your webthings to gateway – Control them using the progressive web app • Try the alternate Progressive Web App to browse them in XR: – http://github.com/samsunginternet/webthings-webapp • More to come: – https://github.com/rzr/webthing-iotjs/wiki/XR
13. Samsung Open Source Group 13 https://social.samsunginter.net/@rzr #LavalVirtual2019 References • https://github.com/rzr/webthing-iotjs/wiki/ – https://github.com/rzr/webthing-iotjs/wiki/Sensor • https://github.com/SamsungInternet/color-sensor-js • https://github.com/rzr/webthing-iotjs/wiki/XR – https://github.com/tizenteam/aframe-ui-widgets/pull/17# • https://hacks.mozilla.org/2019/03/connecting-real-things-to-virtual-worlds-using-web/ • https://hub.samsunginter.net/ – https://github.com/immersive-web/webvr-polyfill • https://aframe.io/ • https://iot.mozilla.org/ • https://monado.dev/ • https://www.khronos.org/openxr • https://github.com/webmixedreality/exokit • https://www.laval-virtual.com/schedule-2019/
14. 14 https://social.samsunginter.net/@rzr Samsung Open Source Group 2019 Q&A ? (or Extras?) Ask now or online: https://social.samsunginter.net/@rzr
15. Samsung Open Source Group 15 https://social.samsunginter.net/@rzr #LavalVirtual2019 Controlling real data & consuming OpenData https://youtu.be/OT0Ahuy3Cv4#webthing-iotjs-opendata-20190202rzr
16. 16 https://social.samsunginter.net/@rzr Samsung Open Source Group 2019 Demo: IoT gateway, ESP MCU, WebApp https://youtu.be/vzoUJ-v5h38#webthing-esp8266-webapp-20180602rzr • • https://vimeo.com/273037442 • webthing-esp8266-webapp-20180602rzr • https://youtu.be/vzoUJ-v5h38 • https://vimeo.com/user12599872/ • webthing-esp8266-webapp-20180602rzr
17. Samsung Open Source Group 17 https://social.samsunginter.net/@rzr #LavalVirtual2019 WebThings-WebApp: Tizen or PWA (sbrowser) https://youtu.be/S_mHYu5-iYM#webthing-webapp-pwa-20180629rzr https://youtu.be/S _mHYu5-iYM#we bthing-webapp-p wa-20180629rzr
18. Samsung Open Source Group 18 https://social.samsunginter.net/@rzr #LavalVirtual2019 *3D for the Web: Lessons from the past • Several attempts before WebGL (2011) • Y2K: Java applets ? – Software rendering • No OpenGL • Just Triangles, Lines or pixels: performance impact – Portable? • Applet plugins weren’t opensource first – Until, gcj, OpenJDK • All failed in integration into browsers – Bad performance – Security policies ? • Same “death” as shockwave/adobe flash • In 2019 we can assume that JS won
19. Samsung Open Source Group 19 https://social.samsunginter.net/@rzr #LavalVirtual2019 Dynamic 3D scenes • VRML97 and VRML2 – Various VRML plugins – Scene graph – Descriptive script (like PoV-ray) – OpenGL support • HW acceleration – Most were closed sources • Except openvrml – Superseded by Web3D (XML) • Still Java applets for – Manipulating scene graph – Javascript eventually
20. Samsung Open Source Group 20 https://social.samsunginter.net/@rzr #LavalVirtual2019 Real time connectivity • Before REST (2000) – and WebSockets (2011) • Java RMI vs AJAX – Security policies – Custom ports – Proxy blockers? • Sensors (before W3C G-sensors) – Costly – Closed source • Drivers • Middlewares • Second life was native