Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Getting Started with WebAR for HoloLens2 and Meta Quest

Getting Started with WebAR for HoloLens2 and Meta Quest

357c9e58ecce2865f9eb748192e5143f?s=128

TakashiYoshinaga

August 13, 2021
Tweet

More Decks by TakashiYoshinaga

Other Decks in Technology

Transcript

  1. Getting Started with WebAR for HoloLens2 and Oculus Quest

  2. Download Sample http://arfukuoka.lolipop.jp/ WebXR2021/Sample.zip

  3. Goal Let's try hand tracking using WebXR on HoloLens2 and

    Oculus Quest. https://youtu.be/y4WD2wgzKYQ
  4. Letʼs Try You can try demo if setting to use

    WebXR is finished on your device. https://webxr-hand-drawing.glitch.me
  5. Duplicate the Template https://glitch.com/~webxr-hand-template GET STARTED

  6. Duplicate the Template Scroll

  7. Duplicate the Template Remix Your Own

  8. Check the Template Click on index.html to make sure the

    code is displayed. index.html
  9. Check the Template Editor Preview Click on index.html to make

    sure the code is displayed.
  10. Change the Content Name (URL) Click on the content name

    on the upper left corner of the screen. Click here
  11. Change the Content Name (URL) Change each string to a

    name that is easier to understand. Change to a simple name
  12. Check the URL Confirm the URL by clicking [Change URL]

    on the top of the preview screen. Change URL URL
  13. Check on the Device Enter the URL AR Button Enter

    the URL VR Button HoloLens2 Quest
  14. How it works (HoloLens2) A Cube will appear.

  15. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  16. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  17. Overview of the Sample Lesson01

  18. Overview of the Sample Loading A-Frame Describes content to be

    shown in 3D space.
  19. Overview of the Sample <a-scene background="color: #ECECEC"> <a-box position="0 1.5

    -2" scale="0.5 0.5 0.5" rotation="0 45 0" color="#4CC3D9" shadow></a-box> </a-scene> a-box position 0 1.5 -2
  20. Overview of the Sample Describe the behavior of the hand

    later.
  21. Overview of the Sample <script> window.onload = function() { initRightHand();

    //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; </script> Automatically called once the page has loaded. Function prepared to describe the behavior of the right hand. Function prepared to describe the behavior of the left hand.
  22. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  23. Show Hand Objects Edit here

  24. Show Hand Objects <a-scene background="color: #ECECEC"> <a-box position="0 1.5 -2"

    scale="0.5 0.5 0.5" rotation="0 45 0" color="#4CC3D9" shadow></a-box> <!-- Drawing the left hand --> <a-entity id="leftHand" hand-tracking-controls="hand: left; modelStyle: dots;"></a-entity> <!-- Drawing the rihgt hand --> <a-entity id="rightHand" hand-tracking-controls="hand: right; modelStyle: dots;"></a-entity> </a-scene> 【hand-tracking-controls】 hand: left or right modelStyle: mesh pr dots (mesh doesnʼt work) modelColor: color of hand (Default is white) Lesson02
  25. Check on the Device Be sure to click the Reload

    button. Reload Reload
  26. Check on the Device

  27. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  28. Prepare Work with Hand Tracking Edit here

  29. Prepare Work with Hand Tracking <script> window.onload = function() {

    initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; </script> Add variables here
  30. Prepare Work with Hand Tracking var scene; //3D space to

    add 3D objects var rightHand; //Variable to access the right hand. var leftHand; //Variable to access the left hand. window.onload = function() { initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; Lesson03
  31. Prepare Work with Hand Tracking var scene; //3D space to

    add 3D objects var rightHand; //Variable to access the right hand var leftHand; //Variable to access the left hand window.onload = function() { // Keeps information in 3D space as variables. scene = document.querySelector('a-scene'); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; Lesson04
  32. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  33. Create Object by Right Hand Operation var scene; //描画する3D空間 var

    rightHand; //右⼿ var leftHand; //左⼿ window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; Edit here
  34. Create Object by Right Hand Operation function initRightHand(){ //Get the

    right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); //Begin pinch rightHand.addEventListener('pinchstarted', function (e) { }); //End pinch rightHand.addEventListener('pinchended', function (e) { }); //Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Lesson05
  35. Getting the Pinch Start function initRightHand(){ //Get the right hand

    object (id=rightHand) rightHand = document.getElementById("rightHand"); //Begin pinch rightHand.addEventListener('pinchstarted', function (e) { }); //End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Edit here
  36. Getting the Pinch Start function initRightHand(){ // Get the right

    hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { var position = e.detail.position; //Calling the addBOX function (explained in the next page) addBox(position); }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Lesson06
  37. Generating Box Object Add the function here

  38. Generating Box Object //左⼿の初期化を⾏う関数 function initLeftHand(){ }; function addBox(position){ //Create

    an element named a-box var box = document.createElement('a-box'); // Set the position, size, and color. box.setAttribute('position', position); box.setAttribute('scale', '0.02 0.02 0.02'); box.setAttribute('color', '#0062C6'); // Set class names for easy deletion later. box.setAttribute('class', 'box'); // Add a box in 3D space. scene.appendChild(box); } </script> Lesson07
  39. Check on the Device

  40. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  41. Follows the Movement of the Hand function initRightHand(){ // Get

    the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Edit here
  42. Follows the Movement of the Hand // Get the right

    hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener(ʻpinchstartedʼ, function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { var position = e.detail.position; //generate a box addBox(position); }); Lesson08
  43. Check on the Device

  44. Problem If you generate the box too densely, it will

    quickly consume memory
  45. Reduce the Density of Boxes <script> var scene; var rightHand;

    var leftHand; var prev;//To store the position where the last box was placed. window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); initRightHand(); initLeftHand(); }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*以下省略*/ Lesson09
  46. Reduce the Density of Boxes var prev;//最後にBox置いた位置を記憶 window.onload = function()

    { /*omitted*/ }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*omitted*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Function to add box function addBox(position){ } Edit here
  47. Reduce the Density of Boxes function addBox(position){ // Create an

    element named a-box var box = document.createElement('a-box'); // Set the position, size, and color box.setAttribute('position', position); box.setAttribute('scale', '0.02 0.02 0.02'); box.setAttribute('color', '#0062C6'); // Set class names for easy deletion later box.setAttribute('class', 'box'); //Add a box in 3D space. scene.appendChild(box); // Memorize the position where the box was placed. prev=box.object3D.position; } Lesson09
  48. Reduce the Density of Boxes var prev;//最後にBox置いた位置を記憶 window.onload = function()

    { /*omitted*/ }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*omitted*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Boxの追加を⾏う関数 function addBox(position){ } Edit here
  49. Reduce the Density of Boxes function initRightHand(){ //Get the right

    hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Edit here
  50. Reduce the Density of Boxes rightHand.addEventListener('pinchmoved', function (e) { var

    position = e.detail.position; // Distance between the last position of the box and // the current position of the hand var length= prev.distanceTo(position); if(length>=0.02){ //Generate a box addBox(position); } }); Lesson10 prev position 一定距離以上
  51. Check on the Device

  52. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  53. Switch Color of Box var scene; var rightHand; var leftHand;

    var prev; // Array of colors to assign to a cube var colors = ['#0062C6', '#00AB00', '#FFCF00', '#7F00FF', '#FFFFFF', '#FF2000', '#F52394', '#593110']; // Index of color array to use var index=0; window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; Lesson11
  54. Switch Color of Box function initRightHand(){ // Get the right

    hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { /*omitted*/ }); }; Edit here
  55. Switch Color of Box function initRightHand(){ // Get the right

    hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { var position = e.detail.position; //Generate a box addBox(position); }); // End pinch rightHand.addEventListener('pinchended', function (e) { // Add up the number of colors. index++; index%=colors.length; }); Lesson12
  56. Switch Color of Box window.onload = function() { /*省略*/ };

    //右⼿の初期化を⾏う関数 function initRightHand(){ /*省略*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Function to generate a box function addBox(position){ } Edit here
  57. Switch Color of Box function addBox(position){ // Create an element

    named a-box var box = document.createElement('a-box'); // Set the position, size, and color box.setAttribute('position', position); box.setAttribute('scale', '0.02 0.02 0.02ʼ); box.setAttribute('color', '#0062C6'); box.setAttribute('color', colors[index]); // Set class names for easy deletion later box.setAttribute('class', 'box'); // Add a box in 3D space scene.appendChild(box); // Memorize the position where the box was placed prev=box.object3D.position; } Lesson13 box.setAttribute('color', '#0062C6');
  58. Chek on the Device

  59. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  60. Delete All Boxes var prev;//最後にBox置いた位置を記憶 window.onload = function() { /*omitted*/

    }; //右⼿の初期化を⾏う関数 function initRightHand(){ /* omitted */ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Function to generate a box function addBox(position){ /*omitted */ } Edit here
  61. Delete All Boxes function initLeftHand(){ // Get the left hand

    object. leftHand = document.getElementById("leftHand"); // Behavior at the start of a pinch leftHand.addEventListener('pinchstarted', function (e) { // Get all objects to which the box class is assigned. var els = document.querySelectorAll('.boxʼ); // Delete the boxes one by one. for (var i = 0; i < els.length; i++) { els[i].parentNode.removeChild(els[i]); } }); }; Lesson14
  62. Completed!

  63. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  64. Getting the Position of Finger Tip Edit inside a-scene

  65. Getting the Position of Finger Tip <a-scene background="color: #ECECEC"> <a-box

    position="0 1.5 -2" scale="0.5 0.5 0.5" rotation="0 45 0" color="#4CC3D9" shadow></a-box> <a-entity id="leftHand" hand-tracking-controls="hand: left; modelStyle: dots;"></a-entity> <a-entity id="rightHand" hand-tracking-controls="hand: right; modelStyle: dots;"></a-entity> <!-- Sphere to be displayed at fingertip --> <a-sphere id="tip" position="0 0 0" radius="0.015" color="#0062C6"></a-sphere> </a-scene> Lesson15
  66. Write Scripts See javascript

  67. Getting the Position of Finger Tip // Array of colors

    to assign to a cube var colors = ['#0062C6', '#00AB00', '#FFCF00', '#7F00FF', '#FFFFFF', '#FF2000', '#F52394', '#593110']; // Index of color array to use var index=0; //Sphere representing the tip of the finger var tip; window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); //Get the sphere representing the fingertip position tip = document.getElementById("tip"); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; Lesson16
  68. Getting the Position of Finger Tip var prev;//最後にBox置いた位置を記憶 window.onload =

    function() { /*省略*/ }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*omitted*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ /*省略*/ }; //Boxの追加を⾏う関数 function addBox(position){ /*省略*/ } Edit here
  69. Getting the Position of Finger Tip function initRightHand(){ // Get

    the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Track finger tips in real time (to be implemented later) tipTracking(rightHand.components['hand-tracking-controls']); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { /*omitted*/ }); Lesson17
  70. Getting the Position of Finger Tip window.onload = function() {

    /*omitted*/ }; function initRightHand(){ /* omitted(右⼿の初期化を⾏う関数)*/ }; function initLeftHand(){ /* omitted(左⼿の初期化を⾏う関数)*/ }; function addBox(position){ /* omitted(Boxの追加を⾏う関数)*/ } Add a function for tracking finger position here
  71. Getting the Position of Finger Tip function initLeftHand(){ /*omitted(左⼿の初期化を⾏う関数)*/ };

    function addBox(position){ /*omitted(Boxの追加を⾏う関数)*/ } function tipTracking(ctrl){ // Finger tip position var position = ctrl.indexTipPosition; // Assign finger position to the ball tip.setAttribute('position', position); // Call tipTracking again after 30ms setTimeout(function(){tipTracking(ctrl);}, 30); }; Lesson18
  72. Assign the Box Color to the Fingertip function initRightHand(){ //右⼿のオブジェクト(id=rightHand)を取得

    rightHand = document.getElementById("rightHand"); //ピンチ開始 rightHand.addEventListener('pinchstarted', function (e) { /*中略*/ }); //End pinch rightHand.addEventListener('pinchended', function (e) { }); //ピンチしながら⼿を移動 rightHand.addEventListener('pinchmoved', function (e) { /*中略*/ }); }; Edit here
  73. 次のBoxの⾊を指先の球に反映 // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { var position

    = e.detail.position; //BOX⽣成関数を呼び出す addBox(position); }); // End pinch rightHand.addEventListener('pinchended', function (e) { //⾊の番号を加算してboxの⾊に反映 index++; index%=colors.length; // Assing the color to the ball at the tip of the finger tip.setAttribute('color', colors[index]); }); Lesson19
  74. Extra part was completed!