Getting Started with WebAR for HoloLens2 and Meta Quest

Transcript

1. Getting Started with WebAR for HoloLens2 and Oculus Quest
2. Download Sample http://arfukuoka.lolipop.jp/ WebXR2021/Sample.zip
3. Goal Let's try hand tracking using WebXR on HoloLens2 and Oculus Quest. https://youtu.be/y4WD2wgzKYQ
4. Letʼs Try You can try demo if setting to use WebXR is finished on your device. https://webxr-hand-drawing.glitch.me
5. Duplicate the Template https://glitch.com/~webxr-hand-template GET STARTED
6. Duplicate the Template Scroll
7. Duplicate the Template Remix Your Own
8. Check the Template Click on index.html to make sure the code is displayed. index.html
9. Check the Template Editor Preview Click on index.html to make sure the code is displayed.
10. Change the Content Name (URL) Click on the content name on the upper left corner of the screen. Click here
11. Change the Content Name (URL) Change each string to a name that is easier to understand. Change to a simple name
12. Check the URL Confirm the URL by clicking [Change URL] on the top of the preview screen. Change URL URL
13. Check on the Device Enter the URL AR Button Enter the URL VR Button HoloLens2 Quest
14. How it works (HoloLens2) A Cube will appear.
15. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
16. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
17. Overview of the Sample Lesson01
18. Overview of the Sample Loading A-Frame Describes content to be shown in 3D space.
19. Overview of the Sample <a-scene background="color: #ECECEC"> <a-box position="0 1.5 -2" scale="0.5 0.5 0.5" rotation="0 45 0" color="#4CC3D9" shadow></a-box> </a-scene> a-box position 0 1.5 -2
20. Overview of the Sample Describe the behavior of the hand later.
21. Overview of the Sample <script> window.onload = function() { initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; </script> Automatically called once the page has loaded. Function prepared to describe the behavior of the right hand. Function prepared to describe the behavior of the left hand.
22. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
23. Show Hand Objects Edit here
24. Show Hand Objects <a-scene background="color: #ECECEC"> <a-box position="0 1.5 -2" scale="0.5 0.5 0.5" rotation="0 45 0" color="#4CC3D9" shadow></a-box> <!-- Drawing the left hand --> <a-entity id="leftHand" hand-tracking-controls="hand: left; modelStyle: dots;"></a-entity> <!-- Drawing the rihgt hand --> <a-entity id="rightHand" hand-tracking-controls="hand: right; modelStyle: dots;"></a-entity> </a-scene> 【hand-tracking-controls】 hand: left or right modelStyle: mesh pr dots (mesh doesnʼt work) modelColor: color of hand (Default is white) Lesson02
25. Check on the Device Be sure to click the Reload button. Reload Reload
26. Check on the Device
27. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
28. Prepare Work with Hand Tracking Edit here
29. Prepare Work with Hand Tracking <script> window.onload = function() { initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; </script> Add variables here
30. Prepare Work with Hand Tracking var scene; //3D space to add 3D objects var rightHand; //Variable to access the right hand. var leftHand; //Variable to access the left hand. window.onload = function() { initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; Lesson03
31. Prepare Work with Hand Tracking var scene; //3D space to add 3D objects var rightHand; //Variable to access the right hand var leftHand; //Variable to access the left hand window.onload = function() { // Keeps information in 3D space as variables. scene = document.querySelector('a-scene'); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; Lesson04
32. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
33. Create Object by Right Hand Operation var scene; //描画する3D空間 var rightHand; //右⼿ var leftHand; //左⼿ window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; Edit here
34. Create Object by Right Hand Operation function initRightHand(){ //Get the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); //Begin pinch rightHand.addEventListener('pinchstarted', function (e) { }); //End pinch rightHand.addEventListener('pinchended', function (e) { }); //Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Lesson05
35. Getting the Pinch Start function initRightHand(){ //Get the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); //Begin pinch rightHand.addEventListener('pinchstarted', function (e) { }); //End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Edit here
36. Getting the Pinch Start function initRightHand(){ // Get the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { var position = e.detail.position; //Calling the addBOX function (explained in the next page) addBox(position); }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Lesson06
37. Generating Box Object Add the function here
38. Generating Box Object //左⼿の初期化を⾏う関数 function initLeftHand(){ }; function addBox(position){ //Create an element named a-box var box = document.createElement('a-box'); // Set the position, size, and color. box.setAttribute('position', position); box.setAttribute('scale', '0.02 0.02 0.02'); box.setAttribute('color', '#0062C6'); // Set class names for easy deletion later. box.setAttribute('class', 'box'); // Add a box in 3D space. scene.appendChild(box); } </script> Lesson07
39. Check on the Device
40. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
41. Follows the Movement of the Hand function initRightHand(){ // Get the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Edit here
42. Follows the Movement of the Hand // Get the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener(ʻpinchstartedʼ, function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { var position = e.detail.position; //generate a box addBox(position); }); Lesson08
43. Check on the Device
44. Problem If you generate the box too densely, it will quickly consume memory
45. Reduce the Density of Boxes <script> var scene; var rightHand; var leftHand; var prev;//To store the position where the last box was placed. window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); initRightHand(); initLeftHand(); }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*以下省略*/ Lesson09
46. Reduce the Density of Boxes var prev;//最後にBox置いた位置を記憶 window.onload = function() { /*omitted*/ }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*omitted*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Function to add box function addBox(position){ } Edit here
47. Reduce the Density of Boxes function addBox(position){ // Create an element named a-box var box = document.createElement('a-box'); // Set the position, size, and color box.setAttribute('position', position); box.setAttribute('scale', '0.02 0.02 0.02'); box.setAttribute('color', '#0062C6'); // Set class names for easy deletion later box.setAttribute('class', 'box'); //Add a box in 3D space. scene.appendChild(box); // Memorize the position where the box was placed. prev=box.object3D.position; } Lesson09
48. Reduce the Density of Boxes var prev;//最後にBox置いた位置を記憶 window.onload = function() { /*omitted*/ }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*omitted*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Boxの追加を⾏う関数 function addBox(position){ } Edit here
49. Reduce the Density of Boxes function initRightHand(){ //Get the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Edit here
50. Reduce the Density of Boxes rightHand.addEventListener('pinchmoved', function (e) { var position = e.detail.position; // Distance between the last position of the box and // the current position of the hand var length= prev.distanceTo(position); if(length>=0.02){ //Generate a box addBox(position); } }); Lesson10 prev position 一定距離以上
51. Check on the Device
52. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
53. Switch Color of Box var scene; var rightHand; var leftHand; var prev; // Array of colors to assign to a cube var colors = ['#0062C6', '#00AB00', '#FFCF00', '#7F00FF', '#FFFFFF', '#FF2000', '#F52394', '#593110']; // Index of color array to use var index=0; window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; Lesson11
54. Switch Color of Box function initRightHand(){ // Get the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { /*omitted*/ }); }; Edit here
55. Switch Color of Box function initRightHand(){ // Get the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { var position = e.detail.position; //Generate a box addBox(position); }); // End pinch rightHand.addEventListener('pinchended', function (e) { // Add up the number of colors. index++; index%=colors.length; }); Lesson12
56. Switch Color of Box window.onload = function() { /*省略*/ }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*省略*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Function to generate a box function addBox(position){ } Edit here
57. Switch Color of Box function addBox(position){ // Create an element named a-box var box = document.createElement('a-box'); // Set the position, size, and color box.setAttribute('position', position); box.setAttribute('scale', '0.02 0.02 0.02ʼ); box.setAttribute('color', '#0062C6'); box.setAttribute('color', colors[index]); // Set class names for easy deletion later box.setAttribute('class', 'box'); // Add a box in 3D space scene.appendChild(box); // Memorize the position where the box was placed prev=box.object3D.position; } Lesson13 box.setAttribute('color', '#0062C6');
58. Chek on the Device
59. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
60. Delete All Boxes var prev;//最後にBox置いた位置を記憶 window.onload = function() { /*omitted*/ }; //右⼿の初期化を⾏う関数 function initRightHand(){ /* omitted */ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Function to generate a box function addBox(position){ /*omitted */ } Edit here
61. Delete All Boxes function initLeftHand(){ // Get the left hand object. leftHand = document.getElementById("leftHand"); // Behavior at the start of a pinch leftHand.addEventListener('pinchstarted', function (e) { // Get all objects to which the box class is assigned. var els = document.querySelectorAll('.boxʼ); // Delete the boxes one by one. for (var i = 0; i < els.length; i++) { els[i].parentNode.removeChild(els[i]); } }); }; Lesson14
62. Completed!
63. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
64. Getting the Position of Finger Tip Edit inside a-scene
65. Getting the Position of Finger Tip <a-scene background="color: #ECECEC"> <a-box position="0 1.5 -2" scale="0.5 0.5 0.5" rotation="0 45 0" color="#4CC3D9" shadow></a-box> <a-entity id="leftHand" hand-tracking-controls="hand: left; modelStyle: dots;"></a-entity> <a-entity id="rightHand" hand-tracking-controls="hand: right; modelStyle: dots;"></a-entity> <!-- Sphere to be displayed at fingertip --> <a-sphere id="tip" position="0 0 0" radius="0.015" color="#0062C6"></a-sphere> </a-scene> Lesson15
66. Write Scripts See javascript
67. Getting the Position of Finger Tip // Array of colors to assign to a cube var colors = ['#0062C6', '#00AB00', '#FFCF00', '#7F00FF', '#FFFFFF', '#FF2000', '#F52394', '#593110']; // Index of color array to use var index=0; //Sphere representing the tip of the finger var tip; window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); //Get the sphere representing the fingertip position tip = document.getElementById("tip"); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; Lesson16
68. Getting the Position of Finger Tip var prev;//最後にBox置いた位置を記憶 window.onload = function() { /*省略*/ }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*omitted*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ /*省略*/ }; //Boxの追加を⾏う関数 function addBox(position){ /*省略*/ } Edit here
69. Getting the Position of Finger Tip function initRightHand(){ // Get the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Track finger tips in real time (to be implemented later) tipTracking(rightHand.components['hand-tracking-controls']); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { /*omitted*/ }); Lesson17
70. Getting the Position of Finger Tip window.onload = function() { /*omitted*/ }; function initRightHand(){ /* omitted(右⼿の初期化を⾏う関数)*/ }; function initLeftHand(){ /* omitted(左⼿の初期化を⾏う関数)*/ }; function addBox(position){ /* omitted(Boxの追加を⾏う関数)*/ } Add a function for tracking finger position here
71. Getting the Position of Finger Tip function initLeftHand(){ /*omitted(左⼿の初期化を⾏う関数)*/ }; function addBox(position){ /*omitted(Boxの追加を⾏う関数)*/ } function tipTracking(ctrl){ // Finger tip position var position = ctrl.indexTipPosition; // Assign finger position to the ball tip.setAttribute('position', position); // Call tipTracking again after 30ms setTimeout(function(){tipTracking(ctrl);}, 30); }; Lesson18
72. Assign the Box Color to the Fingertip function initRightHand(){ //右⼿のオブジェクト(id=rightHand)を取得 rightHand = document.getElementById("rightHand"); //ピンチ開始 rightHand.addEventListener('pinchstarted', function (e) { /*中略*/ }); //End pinch rightHand.addEventListener('pinchended', function (e) { }); //ピンチしながら⼿を移動 rightHand.addEventListener('pinchmoved', function (e) { /*中略*/ }); }; Edit here
73. 次のBoxの⾊を指先の球に反映 // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { var position = e.detail.position; //BOX⽣成関数を呼び出す addBox(position); }); // End pinch rightHand.addEventListener('pinchended', function (e) { //⾊の番号を加算してboxの⾊に反映 index++; index%=colors.length; // Assing the color to the ball at the tip of the finger tip.setAttribute('color', colors[index]); }); Lesson19
74. Extra part was completed!