aframe-webthing-20190710rzr

Merging Realities
Using the Web to Bring the Internet of Things to High End Augmented Reality
FullStack Conference
#FullStackCon, London UK <2019-07-10>

Philippe Coval
Fabien Benetou

URL: http://purl.org/aframe-webthing

Transcript

1. #FullStackCon 2019 1 http://purl.org/aframe-webthing
   Philippe Coval
   Fabien Benetou
   Merging Realities
   Using the Web to Bring
   the Internet of Things
   to High End Augmented Reality
   FullStack Conference
   #FullStackCon, London UK <2019-07-10>
   

   View Slide

2. #FullStackCon 2019 2 http://purl.org/aframe-webthing
   $ who are we ?
   ●
   Philippe Coval
   – Software engineer for Samsung Open Source Group
   – Interest: Web of Things with “Privacy by Design”
   ●
   Fabien Benetou
   – WebXR developer consulting for the European Parliament / UNICEF
   – Mozilla Tech Speaker
   ●
   Met In 2019 at FOSDEM and mutual inspiration
   – Check @MozHacks
   

   View Slide

3. #FullStackCon 2019 3 http://purl.org/aframe-webthing
   Agenda
   1) AR and VR on the web
   2) WebThings connectivity
   3) Binding WebThings and XR
   

   View Slide

4. Samsung Open Source Group 4 https://fosdem.org/2018/schedule/event/tizen_rt/
   “Simplicity
   is the ultimate sophistication.”
   ~Leonardo da Vinci
   

   View Slide

5. #FullStackCon 2019 5 http://purl.org/aframe-webthing
   From HTML to 2D
   

   View Slide

6. #FullStackCon 2019 6 http://purl.org/aframe-webthing
   From 2D to 3D
   

   View Slide

7. #FullStackCon 2019 7 http://purl.org/aframe-webthing
   Making space : coordinate systems
   

   View Slide

8. #FullStackCon 2019 8 http://purl.org/aframe-webthing
   From a browser view to an immersive world
   

   View Slide

9. #FullStackCon 2019 9 http://purl.org/aframe-webthing
   Always on the web
   

   View Slide

10. #FullStackCon 2019 10 http://purl.org/aframe-webthing
    A view from nowhere?
    

    View Slide

11. #FullStackCon 2019 11 http://purl.org/aframe-webthing
    Turning heads
    

    View Slide

12. #FullStackCon 2019 12 http://purl.org/aframe-webthing
    Position but also rotation
    

    View Slide

13. #FullStackCon 2019 13 http://purl.org/aframe-webthing
    Pose to move in space
    

    View Slide

14. #FullStackCon 2019 14 http://purl.org/aframe-webthing
    Tracking : outside-in
    

    View Slide

15. #FullStackCon 2019 15 http://purl.org/aframe-webthing
    Tracking : Inside-out
    

    View Slide

16. #FullStackCon 2019 16 http://purl.org/aframe-webthing
    Interacting with objects
    

    View Slide

17. #FullStackCon 2019 17 http://purl.org/aframe-webthing
    Beyond primitives : what's a mesh
    

    View Slide

18. #FullStackCon 2019 18 http://purl.org/aframe-webthing
    Beyond static object : components
    

    View Slide

19. #FullStackCon 2019 19 http://purl.org/aframe-webthing
    WebXR examples, AR focus
    

    View Slide

20. #FullStackCon 2019 20 http://purl.org/aframe-webthing
    WebXR examples, VR focus
    

    View Slide

21. 21
    https://social.samsunginter.net/@rzr
    Samsung Open Source Group
    

    View Slide

22. #FullStackCon 2019 22 http://purl.org/aframe-webthing
    Web of Thing aka WoT
    ●
    W3C working group
    – Standardization
    ●
    Interaction with devices
    – using Web technologies:
    ●
    JSON, RESTful,
    ●
    HTTP, WebSockets, CoAP…
    ●
    Things description
    – JSON-TD / JSON-LD
    – Semantics: (iot.schema.org)
    Simplified Thing Description
    {
    "name":"Switch",
    "id":"urn:dev:wot:…:switch",
    "base":"https://…/switch",
    "properties": {
    "on": {
    "type":"boolean",
    // ...
    "href":"/on"
    }
    }
    // …
    "actions": // …
    "events": // …
    }…
    

    View Slide

23. Samsung Open Source Group 23 https://social.samsunginter.net/@rzr
    Mozilla WebThing project
    ●
    FLOSS Platform for IoT
    – using Web
    – with Privacy By Design
    ●
    WebThing API and Schemas
    – Can be implemented
    – using native (C/C++), Python,
    – JS (node.js or IoT.js)
    ●
    Things gateway
    – to connect, control (UI)
    – and automate webthings
    G
    D
    P
    R
    G
    D
    PR
    A
    rt. 25
    A
    rt.
    25
    

    View Slide

24. Samsung Open Source Group 24 https://social.samsunginter.net/@rzr
    Javascript the language of Web (of Twins)
    ●
    IoT.js: an alternative runtime inspired by Node.js:
    – Powered by JerryScript engine designed for micro-controllers
    – Feature: Networking (HTTP, MQTT...) & IO (I2C, GPIO, ADC, PWM)
    ●
    Things can be build using webthing-iotjs module:
    – Standalone HTTP servers exposing Mozilla Things API:
    ●
    RESTful architecture: read, update operations
    – Devices can be connected to MozIoT webthing gateway
    ●
    ACL (JWT), Remote access using pkgkite tunneling
    – Or Web services (OpenData, Social Media etc)
    

    View Slide

25. Samsung Open Source Group 25 https://fosdem.org/2018/schedule/event/tizen_rt/
    “The secret to getting ahead
    is getting started.”
    ~Mark Twain
    

    View Slide

26. #FullStackCon 2019 26 http://purl.org/aframe-webthing
    Example: A Robotic arm idea
    ●
    From concept:
    – Top level properties: Angles:
    ●
    Torso [-180, +180]
    ●
    Shoulder [0, +90]
    ●
    Arm [0, +90]
    ●
    Hand [0, +45]
    ●
    To early specifications:
    – Design Model CAD→ VR/AR
    ●
    Simulation
    ●
    → Identify integration issues:
    – (Location, constraints)
    – Implement embedded system
    ●
    Sourcing hardware
    ●
    Controller / Controllee
    ●
    Adjust and iterate
    – Thing description+API stay up
    

    View Slide

27. #FullStackCon 2019 27 http://purl.org/aframe-webthing
    WebThing to XR Integration
    ●
    A-Frame components can be accessed from DOM API
    ●
    So attributes can be updated from JS
    – Bind Webthing from URL
    ●
    Using HTTP polling (and/or WebSockets if supported)
    – If shared through gateway (server):
    ●
    Set URL + JWT token to requests (client app)
    – authorization issued from WebUI (devel menu)
    ●
    It works both way:
    – Sensors: (Real to XR) & Actuators (XR to Real)
    ●
    https://www.npmjs.com/package/aframe-webthing
    

    View Slide

28. Samsung Open Source Group 28 https://fosdem.org/2018/schedule/event/tizen_rt/
    “Any sufficiently
    advanced technology
    is indistinguishable
    from magic.”
    ~ Arthur C. Clarke
    

    View Slide

29. #FullStackCon 2019 29 http://purl.org/aframe-webthing
    Digital Twins with WebThing-IoTjs (on STM32)
    https://youtu.be/sUayRsjV1Ys
    

    View Slide

30. #FullStackCon 2019 30 http://purl.org/aframe-webthing
    Live control in 3D using A-Frame on GearVR:
    https://youtu.be/s3r8pQtzhAU#wotxr-20190320rzr
    

    View Slide

31. #FullStackCon 2019 31 http://purl.org/aframe-webthing
    Lets build the Web of Twins ?
    ●
    Digital twins
    – Introduced by Dr M. Grieves
    (2002)
    ●
    Real time (or deferred) connectivity:
    – Between the physical component
    ●
    and its digital counterpart
    ●
    “Devices as service” concept:
    ●
    Applies to many industries:
    – City, manufacturing, health,
    transport…
    ●
    Useful for:
    – Re/Co/Design
    – Monitoring, Quality tracking
    – Impact analysis:
    ●
    Dependency, process,
    lifecycle, financial...
    – Digital traces for analytic
    – Simulation, AI/ML etc
    – Improve decision making
    

    View Slide

32. #FullStackCon 2019 32 http://purl.org/aframe-webthing
    Contributors welcome !
    ●
    Open Source:
    – https://github.com/rzr/aframe-webthing
    – https://github.com/rzr/twins
    – https://github.com/rzr/webthing-iotjs/wiki
    – https://github.com/SamsungInternet/color-sensor-js
    ●
    Infos:
    – https://social.samsunginter.net/@rzr/102139995659879619
    – https://hacks.mozilla.org/2019/03/connecting-real-things-to-virtual-worlds-using-web/
    – https://skillsmatter.com/conferences/11213-fullstack-london-2019-the-conference-on-javascript-node-and-internet-of-things#program
    

    View Slide

33. #FullStackCon 2019 33 http://purl.org/aframe-webthing
    Summary
    ●
    3D in browser has never been so easy
    – Using A-frame framework supported by browsers and devices
    ●
    Ready for JavaScript developers
    – Develop for low end microcontrollers using IoT.js
    – And support WebThing API to be interacted from the web
    ●
    Direct or connected to gateway (shared securely to apps)
    ●
    Let’s develop the Web of Twins:
    – built on FLOSS tools with privacy by design
    

    View Slide

34. #FullStackCon 2019 34 http://purl.org/aframe-webthing
    Summary
    1) AR and VR on the web ✓
    

    View Slide

35. #FullStackCon 2019 35 http://purl.org/aframe-webthing
    Summary
    1) AR and VR on the web ✓
    2) WebThings connectivity ✓
    

    View Slide

36. #FullStackCon 2019 36 http://purl.org/aframe-webthing
    Summary
    1) AR and VR on the web ✓
    2) WebThings connectivity ✓
    3) Binding WebThings and XR ✓
    

    View Slide

37. 37
    https://social.samsunginter.net/@rzr
    Samsung Open Source Group
    Q&A (or Extras?)
    Ask now or online:
    https://social.samsunginter.net/@rzr
    https://twitter.com/@utopiah
    

    View Slide

38. #FullStackCon 2019 38 http://purl.org/aframe-webthing
    Community rules @ #FullStackCon
    

    View Slide

39. 39
    https://social.samsunginter.net/@rzr
    Samsung Open Source Group
    Generic Sensors addon adapter (I2C)
    https://purl.org/rzr/mozilla-iot-gateway-sensors-20180406rzr
    ●
    mozilla-iot-gateway-sensors-20180406rzr
    ●
    https://youtu.be/4haKrPetGmg
    

    View Slide

40. 40
    https://social.samsunginter.net/@rzr
    Samsung Open Source Group
    Social Web of Things demo (#MozFest)
    https://purl.org/rzr/webthing-iotjs-20181027rzr
    

    View Slide

41. #FullStackCon 2019 41 http://purl.org/aframe-webthing
    Controlling real devices / consuming OpenData
    https://purl.org/rzr/webthing-iotjs-opendata-20190202rzr
    

    View Slide

42. 42
    https://social.samsunginter.net/@rzr
    Samsung Open Source Group
    Thanks !
    Resources:
    Flaticons CC,
    PixBay.com
    https://Social.SamsungInter.net/@rzr
    https://twitter.com/@utopiah
    

    View Slide