Upgrade to Pro — share decks privately, control downloads, hide ads and more …

aframe-webthing-20190710rzr

 aframe-webthing-20190710rzr

Merging Realities
Using the Web to Bring the Internet of Things to High End Augmented Reality
FullStack Conference
#FullStackCon, London UK <2019-07-10>

Philippe Coval
Fabien Benetou

URL: http://purl.org/aframe-webthing

Phil "RzR" Coval

July 10, 2019
Tweet

More Decks by Phil "RzR" Coval

Other Decks in Technology

Transcript

  1. #FullStackCon 2019 1 http://purl.org/aframe-webthing
    Philippe Coval
    Fabien Benetou
    Merging Realities
    Using the Web to Bring
    the Internet of Things
    to High End Augmented Reality
    FullStack Conference
    #FullStackCon, London UK <2019-07-10>

    View Slide

  2. #FullStackCon 2019 2 http://purl.org/aframe-webthing
    $ who are we ?

    Philippe Coval
    – Software engineer for Samsung Open Source Group
    – Interest: Web of Things with “Privacy by Design”

    Fabien Benetou
    – WebXR developer consulting for the European Parliament / UNICEF
    – Mozilla Tech Speaker

    Met In 2019 at FOSDEM and mutual inspiration
    – Check @MozHacks

    View Slide

  3. #FullStackCon 2019 3 http://purl.org/aframe-webthing
    Agenda
    1) AR and VR on the web
    2) WebThings connectivity
    3) Binding WebThings and XR

    View Slide

  4. Samsung Open Source Group 4 https://fosdem.org/2018/schedule/event/tizen_rt/
    “Simplicity
    is the ultimate sophistication.”
    ~Leonardo da Vinci

    View Slide

  5. #FullStackCon 2019 5 http://purl.org/aframe-webthing
    From HTML to 2D

    View Slide

  6. #FullStackCon 2019 6 http://purl.org/aframe-webthing
    From 2D to 3D

    View Slide

  7. #FullStackCon 2019 7 http://purl.org/aframe-webthing
    Making space : coordinate systems

    View Slide

  8. #FullStackCon 2019 8 http://purl.org/aframe-webthing
    From a browser view to an immersive world

    View Slide

  9. #FullStackCon 2019 9 http://purl.org/aframe-webthing
    Always on the web

    View Slide

  10. #FullStackCon 2019 10 http://purl.org/aframe-webthing
    A view from nowhere?

    View Slide

  11. #FullStackCon 2019 11 http://purl.org/aframe-webthing
    Turning heads

    View Slide

  12. #FullStackCon 2019 12 http://purl.org/aframe-webthing
    Position but also rotation

    View Slide

  13. #FullStackCon 2019 13 http://purl.org/aframe-webthing
    Pose to move in space

    View Slide

  14. #FullStackCon 2019 14 http://purl.org/aframe-webthing
    Tracking : outside-in

    View Slide

  15. #FullStackCon 2019 15 http://purl.org/aframe-webthing
    Tracking : Inside-out

    View Slide

  16. #FullStackCon 2019 16 http://purl.org/aframe-webthing
    Interacting with objects

    View Slide

  17. #FullStackCon 2019 17 http://purl.org/aframe-webthing
    Beyond primitives : what's a mesh

    View Slide

  18. #FullStackCon 2019 18 http://purl.org/aframe-webthing
    Beyond static object : components

    View Slide

  19. #FullStackCon 2019 19 http://purl.org/aframe-webthing
    WebXR examples, AR focus

    View Slide

  20. #FullStackCon 2019 20 http://purl.org/aframe-webthing
    WebXR examples, VR focus

    View Slide

  21. 21
    https://social.samsunginter.net/@rzr
    Samsung Open Source Group

    View Slide

  22. #FullStackCon 2019 22 http://purl.org/aframe-webthing
    Web of Thing aka WoT

    W3C working group
    – Standardization

    Interaction with devices
    – using Web technologies:

    JSON, RESTful,

    HTTP, WebSockets, CoAP…

    Things description
    – JSON-TD / JSON-LD
    – Semantics: (iot.schema.org)
    Simplified Thing Description
    {
    "name":"Switch",
    "id":"urn:dev:wot:…:switch",
    "base":"https://…/switch",
    "properties": {
    "on": {
    "type":"boolean",
    // ...
    "href":"/on"
    }
    }
    // …
    "actions": // …
    "events": // …
    }…

    View Slide

  23. Samsung Open Source Group 23 https://social.samsunginter.net/@rzr
    Mozilla WebThing project

    FLOSS Platform for IoT
    – using Web
    – with Privacy By Design

    WebThing API and Schemas
    – Can be implemented
    – using native (C/C++), Python,
    – JS (node.js or IoT.js)

    Things gateway
    – to connect, control (UI)
    – and automate webthings
    G
    D
    P
    R
    G
    D
    PR
    A
    rt. 25
    A
    rt.
    25

    View Slide

  24. Samsung Open Source Group 24 https://social.samsunginter.net/@rzr
    Javascript the language of Web (of Twins)

    IoT.js: an alternative runtime inspired by Node.js:
    – Powered by JerryScript engine designed for micro-controllers
    – Feature: Networking (HTTP, MQTT...) & IO (I2C, GPIO, ADC, PWM)

    Things can be build using webthing-iotjs module:
    – Standalone HTTP servers exposing Mozilla Things API:

    RESTful architecture: read, update operations
    – Devices can be connected to MozIoT webthing gateway

    ACL (JWT), Remote access using pkgkite tunneling
    – Or Web services (OpenData, Social Media etc)

    View Slide

  25. Samsung Open Source Group 25 https://fosdem.org/2018/schedule/event/tizen_rt/
    “The secret to getting ahead
    is getting started.”
    ~Mark Twain

    View Slide

  26. #FullStackCon 2019 26 http://purl.org/aframe-webthing
    Example: A Robotic arm idea

    From concept:
    – Top level properties: Angles:

    Torso [-180, +180]

    Shoulder [0, +90]

    Arm [0, +90]

    Hand [0, +45]

    To early specifications:
    – Design Model CAD→ VR/AR

    Simulation

    → Identify integration issues:
    – (Location, constraints)
    – Implement embedded system

    Sourcing hardware

    Controller / Controllee

    Adjust and iterate
    – Thing description+API stay up

    View Slide

  27. #FullStackCon 2019 27 http://purl.org/aframe-webthing
    WebThing to XR Integration

    A-Frame components can be accessed from DOM API

    So attributes can be updated from JS
    – Bind Webthing from URL

    Using HTTP polling (and/or WebSockets if supported)
    – If shared through gateway (server):

    Set URL + JWT token to requests (client app)
    – authorization issued from WebUI (devel menu)

    It works both way:
    – Sensors: (Real to XR) & Actuators (XR to Real)

    https://www.npmjs.com/package/aframe-webthing

    View Slide

  28. Samsung Open Source Group 28 https://fosdem.org/2018/schedule/event/tizen_rt/
    “Any sufficiently
    advanced technology
    is indistinguishable
    from magic.”
    ~ Arthur C. Clarke

    View Slide

  29. #FullStackCon 2019 29 http://purl.org/aframe-webthing
    Digital Twins with WebThing-IoTjs (on STM32)
    https://youtu.be/sUayRsjV1Ys

    View Slide

  30. #FullStackCon 2019 30 http://purl.org/aframe-webthing
    Live control in 3D using A-Frame on GearVR:
    https://youtu.be/s3r8pQtzhAU#wotxr-20190320rzr

    View Slide

  31. #FullStackCon 2019 31 http://purl.org/aframe-webthing
    Lets build the Web of Twins ?

    Digital twins
    – Introduced by Dr M. Grieves
    (2002)

    Real time (or deferred) connectivity:
    – Between the physical component

    and its digital counterpart

    “Devices as service” concept:

    Applies to many industries:
    – City, manufacturing, health,
    transport…

    Useful for:
    – Re/Co/Design
    – Monitoring, Quality tracking
    – Impact analysis:

    Dependency, process,
    lifecycle, financial...
    – Digital traces for analytic
    – Simulation, AI/ML etc
    – Improve decision making

    View Slide

  32. #FullStackCon 2019 32 http://purl.org/aframe-webthing
    Contributors welcome !

    Open Source:
    – https://github.com/rzr/aframe-webthing
    – https://github.com/rzr/twins
    – https://github.com/rzr/webthing-iotjs/wiki
    – https://github.com/SamsungInternet/color-sensor-js

    Infos:
    – https://social.samsunginter.net/@rzr/102139995659879619
    – https://hacks.mozilla.org/2019/03/connecting-real-things-to-virtual-worlds-using-web/
    – https://skillsmatter.com/conferences/11213-fullstack-london-2019-the-conference-on-javascript-node-and-internet-of-things#program

    View Slide

  33. #FullStackCon 2019 33 http://purl.org/aframe-webthing
    Summary

    3D in browser has never been so easy
    – Using A-frame framework supported by browsers and devices

    Ready for JavaScript developers
    – Develop for low end microcontrollers using IoT.js
    – And support WebThing API to be interacted from the web

    Direct or connected to gateway (shared securely to apps)

    Let’s develop the Web of Twins:
    – built on FLOSS tools with privacy by design

    View Slide

  34. #FullStackCon 2019 34 http://purl.org/aframe-webthing
    Summary
    1) AR and VR on the web ✓

    View Slide

  35. #FullStackCon 2019 35 http://purl.org/aframe-webthing
    Summary
    1) AR and VR on the web ✓
    2) WebThings connectivity ✓

    View Slide

  36. #FullStackCon 2019 36 http://purl.org/aframe-webthing
    Summary
    1) AR and VR on the web ✓
    2) WebThings connectivity ✓
    3) Binding WebThings and XR ✓

    View Slide

  37. 37
    https://social.samsunginter.net/@rzr
    Samsung Open Source Group
    Q&A (or Extras?)
    Ask now or online:
    https://social.samsunginter.net/@rzr
    https://twitter.com/@utopiah

    View Slide

  38. #FullStackCon 2019 38 http://purl.org/aframe-webthing
    Community rules @ #FullStackCon

    View Slide

  39. 39
    https://social.samsunginter.net/@rzr
    Samsung Open Source Group
    Generic Sensors addon adapter (I2C)
    https://purl.org/rzr/mozilla-iot-gateway-sensors-20180406rzr

    mozilla-iot-gateway-sensors-20180406rzr

    https://youtu.be/4haKrPetGmg

    View Slide

  40. 40
    https://social.samsunginter.net/@rzr
    Samsung Open Source Group
    Social Web of Things demo (#MozFest)
    https://purl.org/rzr/webthing-iotjs-20181027rzr

    View Slide

  41. #FullStackCon 2019 41 http://purl.org/aframe-webthing
    Controlling real devices / consuming OpenData
    https://purl.org/rzr/webthing-iotjs-opendata-20190202rzr

    View Slide

  42. 42
    https://social.samsunginter.net/@rzr
    Samsung Open Source Group
    Thanks !
    Resources:
    Flaticons CC,
    PixBay.com
    https://Social.SamsungInter.net/@rzr
    https://twitter.com/@utopiah

    View Slide