Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Getting Started with WebAR for HoloLens2 and Meta Quest

Getting Started with WebAR for HoloLens2 and Meta Quest

TakashiYoshinaga

August 13, 2021
Tweet

More Decks by TakashiYoshinaga

Other Decks in Technology

Transcript

  1. Getting Started with WebAR
    for HoloLens2 and Oculus Quest

    View full-size slide

  2. Download Sample
    http://arfukuoka.lolipop.jp/
    WebXR2021/Sample.zip

    View full-size slide

  3. Goal
    Let's try hand tracking using WebXR on HoloLens2 and Oculus Quest.
    https://youtu.be/y4WD2wgzKYQ

    View full-size slide

  4. Letʼs Try
    You can try demo if setting to use
    WebXR is finished on your device.
    https://webxr-hand-drawing.glitch.me

    View full-size slide

  5. Duplicate the Template
    https://glitch.com/~webxr-hand-template
    GET STARTED

    View full-size slide

  6. Duplicate the Template
    Scroll

    View full-size slide

  7. Duplicate the Template
    Remix Your Own

    View full-size slide

  8. Check the Template
    Click on index.html to make sure the code is displayed.
    index.html

    View full-size slide

  9. Check the Template
    Editor Preview
    Click on index.html to make sure the code is displayed.

    View full-size slide

  10. Change the Content Name (URL)
    Click on the content name on the upper left corner of the screen.
    Click here

    View full-size slide

  11. Change the Content Name (URL)
    Change each string to a name that is easier to understand.
    Change to a simple name

    View full-size slide

  12. Check the URL
    Confirm the URL by clicking [Change URL] on the top
    of the preview screen.
    Change URL
    URL

    View full-size slide

  13. Check on the Device
    Enter the URL
    AR Button
    Enter the URL
    VR Button
    HoloLens2 Quest

    View full-size slide

  14. How it works (HoloLens2)
    A Cube will appear.

    View full-size slide

  15. Tutorial Procedures
    1.Explain the overview of the sample.
    2.Add hand tracking feature.
    3.Prepare work with hand tracking by javascript.
    4.Describe the action of the right hand.
    a) Spawn an object when pinching starts.
    b) Spawn objects to the trajectory of the hand.
    c) Set the color of the next object to be added.
    when the pinch ends.
    5.Describe the action of the left hand.
    a) Delete all objects at the start of pinch
    6.Extra
    a) Always track the position of the index finger

    View full-size slide

  16. Tutorial Procedures
    1.Explain the overview of the sample.
    2.Add hand tracking feature.
    3.Prepare work with hand tracking by javascript.
    4.Describe the action of the right hand.
    a) Spawn an object when pinching starts.
    b) Spawn objects to the trajectory of the hand.
    c) Set the color of the next object to be added.
    when the pinch ends.
    5.Describe the action of the left hand.
    a) Delete all objects at the start of pinch
    6.Extra
    a) Always track the position of the index finger

    View full-size slide

  17. Overview of the Sample
    Lesson01

    View full-size slide

  18. Overview of the Sample
    Loading A-Frame
    Describes content to be
    shown in 3D space.

    View full-size slide

  19. Overview of the Sample

    color="#4CC3D9" shadow>

    a-box
    position
    0 1.5 -2

    View full-size slide

  20. Overview of the Sample
    Describe the
    behavior of
    the hand later.

    View full-size slide

  21. Overview of the Sample
    <br/>window.onload = function() {<br/>initRightHand(); //右⼿の初期化<br/>initLeftHand(); //左⼿の初期化<br/>};<br/>//右⼿の初期化を⾏う関数<br/>function initRightHand(){<br/>};<br/>//左⼿の初期化を⾏う関数<br/>function initLeftHand(){<br/>};<br/>
    Automatically called
    once the page has
    loaded.
    Function prepared to
    describe the behavior
    of the right hand.
    Function prepared to
    describe the behavior
    of the left hand.

    View full-size slide

  22. Tutorial Procedures
    1.Explain the overview of the sample.
    2.Add hand tracking feature.
    3.Prepare work with hand tracking by javascript.
    4.Describe the action of the right hand.
    a) Spawn an object when pinching starts.
    b) Spawn objects to the trajectory of the hand.
    c) Set the color of the next object to be added.
    when the pinch ends.
    5.Describe the action of the left hand.
    a) Delete all objects at the start of pinch
    6.Extra
    a) Always track the position of the index finger

    View full-size slide

  23. Show Hand Objects
    Edit here

    View full-size slide

  24. Show Hand Objects

    color="#4CC3D9" shadow>

    hand-tracking-controls="hand: left; modelStyle: dots;">

    hand-tracking-controls="hand: right; modelStyle: dots;">

    【hand-tracking-controls】
    hand: left or right
    modelStyle: mesh pr dots
    (mesh doesnʼt work)
    modelColor: color of hand
    (Default is white) Lesson02

    View full-size slide

  25. Check on the Device
    Be sure to click the Reload button.
    Reload
    Reload

    View full-size slide

  26. Check on the Device

    View full-size slide

  27. Tutorial Procedures
    1.Explain the overview of the sample.
    2.Add hand tracking feature.
    3.Prepare work with hand tracking by javascript.
    4.Describe the action of the right hand.
    a) Spawn an object when pinching starts.
    b) Spawn objects to the trajectory of the hand.
    c) Set the color of the next object to be added.
    when the pinch ends.
    5.Describe the action of the left hand.
    a) Delete all objects at the start of pinch
    6.Extra
    a) Always track the position of the index finger

    View full-size slide

  28. Prepare Work with Hand Tracking
    Edit here

    View full-size slide

  29. Prepare Work with Hand Tracking
    <br/>window.onload = function() {<br/>initRightHand(); //右⼿の初期化<br/>initLeftHand(); //左⼿の初期化<br/>};<br/>//右⼿の初期化を⾏う関数<br/>function initRightHand(){<br/>};<br/>//左⼿の初期化を⾏う関数<br/>function initLeftHand(){<br/>};<br/>
    Add variables here

    View full-size slide

  30. Prepare Work with Hand Tracking
    var scene; //3D space to add 3D objects
    var rightHand; //Variable to access the right hand.
    var leftHand; //Variable to access the left hand.
    window.onload = function() {
    initRightHand(); //右⼿の初期化
    initLeftHand(); //左⼿の初期化
    };
    //右⼿の初期化を⾏う関数
    function initRightHand(){
    };
    //左⼿の初期化を⾏う関数
    function initLeftHand(){
    }; Lesson03

    View full-size slide

  31. Prepare Work with Hand Tracking
    var scene; //3D space to add 3D objects
    var rightHand; //Variable to access the right hand
    var leftHand; //Variable to access the left hand
    window.onload = function() {
    // Keeps information in 3D space as variables.
    scene = document.querySelector('a-scene');
    initRightHand(); //右⼿の初期化
    initLeftHand(); //左⼿の初期化
    };
    //右⼿の初期化を⾏う関数
    function initRightHand(){
    };
    //左⼿の初期化を⾏う関数
    function initLeftHand(){ };
    Lesson04

    View full-size slide

  32. Tutorial Procedures
    1.Explain the overview of the sample.
    2.Add hand tracking feature.
    3.Prepare work with hand tracking by javascript.
    4.Describe the action of the right hand.
    a) Spawn an object when pinching starts.
    b) Spawn objects to the trajectory of the hand.
    c) Set the color of the next object to be added.
    when the pinch ends.
    5.Describe the action of the left hand.
    a) Delete all objects at the start of pinch
    6.Extra
    a) Always track the position of the index finger

    View full-size slide

  33. Create Object by Right Hand Operation
    var scene; //描画する3D空間
    var rightHand; //右⼿
    var leftHand; //左⼿
    window.onload = function() {
    //3D空間を変数として保持
    scene = document.querySelector('a-scene');
    initRightHand(); //右⼿の初期化
    initLeftHand(); //左⼿の初期化
    };
    //右⼿の初期化を⾏う関数
    function initRightHand(){
    };
    //左⼿の初期化を⾏う関数
    function initLeftHand(){ };
    Edit here

    View full-size slide

  34. Create Object by Right Hand Operation
    function initRightHand(){
    //Get the right hand object (id=rightHand)
    rightHand = document.getElementById("rightHand");
    //Begin pinch
    rightHand.addEventListener('pinchstarted', function (e) {
    });
    //End pinch
    rightHand.addEventListener('pinchended', function (e) {
    });
    //Move hand while pinching
    rightHand.addEventListener('pinchmoved', function (e) {
    });
    }; Lesson05

    View full-size slide

  35. Getting the Pinch Start
    function initRightHand(){
    //Get the right hand object (id=rightHand)
    rightHand = document.getElementById("rightHand");
    //Begin pinch
    rightHand.addEventListener('pinchstarted', function (e) {
    });
    //End pinch
    rightHand.addEventListener('pinchended', function (e) {
    });
    // Move hand while pinching
    rightHand.addEventListener('pinchmoved', function (e) {
    });
    };
    Edit here

    View full-size slide

  36. Getting the Pinch Start
    function initRightHand(){
    // Get the right hand object (id=rightHand)
    rightHand = document.getElementById("rightHand");
    // Begin pinch
    rightHand.addEventListener('pinchstarted', function (e) {
    var position = e.detail.position;
    //Calling the addBOX function (explained in the next page)
    addBox(position);
    });
    // End pinch
    rightHand.addEventListener('pinchended', function (e) {
    });
    // Move hand while pinching
    rightHand.addEventListener('pinchmoved', function (e) {
    });
    }; Lesson06

    View full-size slide

  37. Generating Box Object
    Add the function here

    View full-size slide

  38. Generating Box Object
    //左⼿の初期化を⾏う関数
    function initLeftHand(){
    };
    function addBox(position){
    //Create an element named a-box
    var box = document.createElement('a-box');
    // Set the position, size, and color.
    box.setAttribute('position', position);
    box.setAttribute('scale', '0.02 0.02 0.02');
    box.setAttribute('color', '#0062C6');
    // Set class names for easy deletion later.
    box.setAttribute('class', 'box');
    // Add a box in 3D space.
    scene.appendChild(box);
    }
    Lesson07

    View full-size slide

  39. Check on the Device

    View full-size slide

  40. Tutorial Procedures
    1.Explain the overview of the sample.
    2.Add hand tracking feature.
    3.Prepare work with hand tracking by javascript.
    4.Describe the action of the right hand.
    a) Spawn an object when pinching starts.
    b) Spawn objects to the trajectory of the hand.
    c) Set the color of the next object to be added.
    when the pinch ends.
    5.Describe the action of the left hand.
    a) Delete all objects at the start of pinch
    6.Extra
    a) Always track the position of the index finger

    View full-size slide

  41. Follows the Movement of the Hand
    function initRightHand(){
    // Get the right hand object (id=rightHand)
    rightHand = document.getElementById("rightHand");
    // Begin pinch
    rightHand.addEventListener('pinchstarted', function (e) {
    /*omitted*/
    });
    // End pinch
    rightHand.addEventListener('pinchended', function (e) {
    });
    // Move hand while pinching
    rightHand.addEventListener('pinchmoved', function (e) {
    });
    };
    Edit here

    View full-size slide

  42. Follows the Movement of the Hand
    // Get the right hand object (id=rightHand)
    rightHand = document.getElementById("rightHand");
    // Begin pinch
    rightHand.addEventListener(ʻpinchstartedʼ, function (e) {
    /*omitted*/
    });
    // End pinch
    rightHand.addEventListener('pinchended', function (e) {
    });
    // Move hand while pinching
    rightHand.addEventListener('pinchmoved', function (e) {
    var position = e.detail.position;
    //generate a box
    addBox(position);
    }); Lesson08

    View full-size slide

  43. Check on the Device

    View full-size slide

  44. Problem
    If you generate the box too densely, it will quickly consume memory

    View full-size slide

  45. Reduce the Density of Boxes
    <br/>var scene;<br/>var rightHand;<br/>var leftHand;<br/>var prev;//To store the position where the last box was placed.<br/>window.onload = function() {<br/>//3D空間を変数として保持<br/>scene = document.querySelector('a-scene');<br/>initRightHand();<br/>initLeftHand();<br/>};<br/>//右⼿の初期化を⾏う関数<br/>function initRightHand(){<br/>/*以下省略*/<br/>Lesson09<br/>

    View full-size slide

  46. Reduce the Density of Boxes
    var prev;//最後にBox置いた位置を記憶
    window.onload = function() {
    /*omitted*/
    };
    //右⼿の初期化を⾏う関数
    function initRightHand(){
    /*omitted*/
    };
    //左⼿の初期化を⾏う関数
    function initLeftHand(){
    };
    //Function to add box
    function addBox(position){
    }
    Edit here

    View full-size slide

  47. Reduce the Density of Boxes
    function addBox(position){
    // Create an element named a-box
    var box = document.createElement('a-box');
    // Set the position, size, and color
    box.setAttribute('position', position);
    box.setAttribute('scale', '0.02 0.02 0.02');
    box.setAttribute('color', '#0062C6');
    // Set class names for easy deletion later
    box.setAttribute('class', 'box');
    //Add a box in 3D space.
    scene.appendChild(box);
    // Memorize the position where the box was placed.
    prev=box.object3D.position;
    }
    Lesson09

    View full-size slide

  48. Reduce the Density of Boxes
    var prev;//最後にBox置いた位置を記憶
    window.onload = function() {
    /*omitted*/
    };
    //右⼿の初期化を⾏う関数
    function initRightHand(){
    /*omitted*/
    };
    //左⼿の初期化を⾏う関数
    function initLeftHand(){
    };
    //Boxの追加を⾏う関数
    function addBox(position){
    }
    Edit here

    View full-size slide

  49. Reduce the Density of Boxes
    function initRightHand(){
    //Get the right hand object (id=rightHand)
    rightHand = document.getElementById("rightHand");
    // Begin pinch
    rightHand.addEventListener('pinchstarted', function (e) {
    /*omitted*/
    });
    // End pinch
    rightHand.addEventListener('pinchended', function (e) {
    });
    // Move hand while pinching
    rightHand.addEventListener('pinchmoved', function (e) {
    });
    };
    Edit here

    View full-size slide

  50. Reduce the Density of Boxes
    rightHand.addEventListener('pinchmoved', function (e) {
    var position = e.detail.position;
    // Distance between the last position of the box and
    // the current position of the hand
    var length= prev.distanceTo(position);
    if(length>=0.02){
    //Generate a box
    addBox(position);
    }
    });
    Lesson10
    prev
    position
    一定距離以上

    View full-size slide

  51. Check on the Device

    View full-size slide

  52. Tutorial Procedures
    1.Explain the overview of the sample.
    2.Add hand tracking feature.
    3.Prepare work with hand tracking by javascript.
    4.Describe the action of the right hand.
    a) Spawn an object when pinching starts.
    b) Spawn objects to the trajectory of the hand.
    c) Set the color of the next object to be added.
    when the pinch ends.
    5.Describe the action of the left hand.
    a) Delete all objects at the start of pinch
    6.Extra
    a) Always track the position of the index finger

    View full-size slide

  53. Switch Color of Box
    var scene;
    var rightHand;
    var leftHand;
    var prev;
    // Array of colors to assign to a cube
    var colors = ['#0062C6', '#00AB00', '#FFCF00', '#7F00FF',
    '#FFFFFF', '#FF2000', '#F52394', '#593110'];
    // Index of color array to use
    var index=0;
    window.onload = function() {
    //3D空間を変数として保持
    scene = document.querySelector('a-scene');
    initRightHand(); //右⼿の初期化
    initLeftHand(); //左⼿の初期化
    }; Lesson11

    View full-size slide

  54. Switch Color of Box
    function initRightHand(){
    // Get the right hand object (id=rightHand)
    rightHand = document.getElementById("rightHand");
    // Begin pinch
    rightHand.addEventListener('pinchstarted', function (e) {
    /*omitted*/
    });
    // End pinch
    rightHand.addEventListener('pinchended', function (e) {
    });
    // Move hand while pinching
    rightHand.addEventListener('pinchmoved', function (e) {
    /*omitted*/
    });
    };
    Edit here

    View full-size slide

  55. Switch Color of Box
    function initRightHand(){
    // Get the right hand object (id=rightHand)
    rightHand = document.getElementById("rightHand");
    // Begin pinch
    rightHand.addEventListener('pinchstarted', function (e) {
    var position = e.detail.position;
    //Generate a box
    addBox(position);
    });
    // End pinch
    rightHand.addEventListener('pinchended', function (e) {
    // Add up the number of colors.
    index++;
    index%=colors.length;
    });
    Lesson12

    View full-size slide

  56. Switch Color of Box
    window.onload = function() {
    /*省略*/
    };
    //右⼿の初期化を⾏う関数
    function initRightHand(){
    /*省略*/
    };
    //左⼿の初期化を⾏う関数
    function initLeftHand(){
    };
    //Function to generate a box
    function addBox(position){
    }
    Edit here

    View full-size slide

  57. Switch Color of Box
    function addBox(position){
    // Create an element named a-box
    var box = document.createElement('a-box');
    // Set the position, size, and color
    box.setAttribute('position', position);
    box.setAttribute('scale', '0.02 0.02 0.02ʼ);
    box.setAttribute('color', '#0062C6');
    box.setAttribute('color', colors[index]);
    // Set class names for easy deletion later
    box.setAttribute('class', 'box');
    // Add a box in 3D space
    scene.appendChild(box);
    // Memorize the position where the box was placed
    prev=box.object3D.position;
    } Lesson13
    box.setAttribute('color', '#0062C6');

    View full-size slide

  58. Chek on the Device

    View full-size slide

  59. Tutorial Procedures
    1.Explain the overview of the sample.
    2.Add hand tracking feature.
    3.Prepare work with hand tracking by javascript.
    4.Describe the action of the right hand.
    a) Spawn an object when pinching starts.
    b) Spawn objects to the trajectory of the hand.
    c) Set the color of the next object to be added.
    when the pinch ends.
    5.Describe the action of the left hand.
    a) Delete all objects at the start of pinch
    6.Extra
    a) Always track the position of the index finger

    View full-size slide

  60. Delete All Boxes
    var prev;//最後にBox置いた位置を記憶
    window.onload = function() {
    /*omitted*/
    };
    //右⼿の初期化を⾏う関数
    function initRightHand(){
    /* omitted */
    };
    //左⼿の初期化を⾏う関数
    function initLeftHand(){
    };
    //Function to generate a box
    function addBox(position){
    /*omitted */
    }
    Edit here

    View full-size slide

  61. Delete All Boxes
    function initLeftHand(){
    // Get the left hand object.
    leftHand = document.getElementById("leftHand");
    // Behavior at the start of a pinch
    leftHand.addEventListener('pinchstarted', function (e) {
    // Get all objects to which the box class is assigned.
    var els = document.querySelectorAll('.boxʼ);
    // Delete the boxes one by one.
    for (var i = 0; i < els.length; i++) {
    els[i].parentNode.removeChild(els[i]);
    }
    });
    };
    Lesson14

    View full-size slide

  62. Tutorial Procedures
    1.Explain the overview of the sample.
    2.Add hand tracking feature.
    3.Prepare work with hand tracking by javascript.
    4.Describe the action of the right hand.
    a) Spawn an object when pinching starts.
    b) Spawn objects to the trajectory of the hand.
    c) Set the color of the next object to be added.
    when the pinch ends.
    5.Describe the action of the left hand.
    a) Delete all objects at the start of pinch
    6.Extra
    a) Always track the position of the index finger

    View full-size slide

  63. Getting the Position of Finger Tip
    Edit inside a-scene

    View full-size slide

  64. Getting the Position of Finger Tip

    rotation="0 45 0" color="#4CC3D9" shadow>



    color="#0062C6">

    Lesson15

    View full-size slide

  65. Write Scripts
    See
    javascript

    View full-size slide

  66. Getting the Position of Finger Tip
    // Array of colors to assign to a cube
    var colors = ['#0062C6', '#00AB00', '#FFCF00', '#7F00FF',
    '#FFFFFF', '#FF2000', '#F52394', '#593110'];
    // Index of color array to use
    var index=0;
    //Sphere representing the tip of the finger
    var tip;
    window.onload = function() {
    //3D空間を変数として保持
    scene = document.querySelector('a-scene');
    //Get the sphere representing the fingertip position
    tip = document.getElementById("tip");
    initRightHand(); //右⼿の初期化
    initLeftHand(); //左⼿の初期化
    };
    Lesson16

    View full-size slide

  67. Getting the Position of Finger Tip
    var prev;//最後にBox置いた位置を記憶
    window.onload = function() {
    /*省略*/
    };
    //右⼿の初期化を⾏う関数
    function initRightHand(){
    /*omitted*/
    };
    //左⼿の初期化を⾏う関数
    function initLeftHand(){
    /*省略*/
    };
    //Boxの追加を⾏う関数
    function addBox(position){
    /*省略*/
    }
    Edit here

    View full-size slide

  68. Getting the Position of Finger Tip
    function initRightHand(){
    // Get the right hand object (id=rightHand)
    rightHand = document.getElementById("rightHand");
    // Track finger tips in real time (to be implemented later)
    tipTracking(rightHand.components['hand-tracking-controls']);
    // Begin pinch
    rightHand.addEventListener('pinchstarted', function (e) {
    /*omitted*/
    });
    // End pinch
    rightHand.addEventListener('pinchended', function (e) {
    /*omitted*/
    });
    Lesson17

    View full-size slide

  69. Getting the Position of Finger Tip
    window.onload = function() {
    /*omitted*/
    };
    function initRightHand(){
    /* omitted(右⼿の初期化を⾏う関数)*/
    };
    function initLeftHand(){
    /* omitted(左⼿の初期化を⾏う関数)*/
    };
    function addBox(position){
    /* omitted(Boxの追加を⾏う関数)*/
    }
    Add a function for tracking
    finger position here

    View full-size slide

  70. Getting the Position of Finger Tip
    function initLeftHand(){
    /*omitted(左⼿の初期化を⾏う関数)*/
    };
    function addBox(position){
    /*omitted(Boxの追加を⾏う関数)*/
    }
    function tipTracking(ctrl){
    // Finger tip position
    var position = ctrl.indexTipPosition;
    // Assign finger position to the ball
    tip.setAttribute('position', position);
    // Call tipTracking again after 30ms
    setTimeout(function(){tipTracking(ctrl);}, 30);
    };
    Lesson18

    View full-size slide

  71. Assign the Box Color to the Fingertip
    function initRightHand(){
    //右⼿のオブジェクト(id=rightHand)を取得
    rightHand = document.getElementById("rightHand");
    //ピンチ開始
    rightHand.addEventListener('pinchstarted', function (e) {
    /*中略*/
    });
    //End pinch
    rightHand.addEventListener('pinchended', function (e) {
    });
    //ピンチしながら⼿を移動
    rightHand.addEventListener('pinchmoved', function (e) {
    /*中略*/
    });
    };
    Edit here

    View full-size slide

  72. 次のBoxの⾊を指先の球に反映
    // Begin pinch
    rightHand.addEventListener('pinchstarted', function (e) {
    var position = e.detail.position;
    //BOX⽣成関数を呼び出す
    addBox(position);
    });
    // End pinch
    rightHand.addEventListener('pinchended', function (e) {
    //⾊の番号を加算してboxの⾊に反映
    index++;
    index%=colors.length;
    // Assing the color to the ball at the tip of the finger
    tip.setAttribute('color', colors[index]);
    });
    Lesson19

    View full-size slide

  73. Extra part was
    completed!

    View full-size slide