Slide 1
Slide 1 text
Getting Started with WebAR
for HoloLens2 and Oculus Quest
Slide 2
Slide 2 text
Download Sample
http://arfukuoka.lolipop.jp/
WebXR2021/Sample.zip
Slide 3
Slide 3 text
Goal
Let's try hand tracking using WebXR on HoloLens2 and Oculus Quest.
https://youtu.be/y4WD2wgzKYQ
Slide 4
Slide 4 text
Letʼs Try
You can try demo if setting to use
WebXR is finished on your device.
https://webxr-hand-drawing.glitch.me
Slide 5
Slide 5 text
Duplicate the Template
https://glitch.com/~webxr-hand-template
GET STARTED
Slide 6
Slide 6 text
Duplicate the Template
Scroll
Slide 7
Slide 7 text
Duplicate the Template
Remix Your Own
Slide 8
Slide 8 text
Check the Template
Click on index.html to make sure the code is displayed.
index.html
Slide 9
Slide 9 text
Check the Template
Editor Preview
Click on index.html to make sure the code is displayed.
Slide 10
Slide 10 text
Change the Content Name (URL)
Click on the content name on the upper left corner of the screen.
Click here
Slide 11
Slide 11 text
Change the Content Name (URL)
Change each string to a name that is easier to understand.
Change to a simple name
Slide 12
Slide 12 text
Check the URL
Confirm the URL by clicking [Change URL] on the top
of the preview screen.
Change URL
URL
Slide 13
Slide 13 text
Check on the Device
Enter the URL
AR Button
Enter the URL
VR Button
HoloLens2 Quest
Slide 14
Slide 14 text
How it works (HoloLens2)
A Cube will appear.
Slide 15
Slide 15 text
Tutorial Procedures
1.Explain the overview of the sample.
2.Add hand tracking feature.
3.Prepare work with hand tracking by javascript.
4.Describe the action of the right hand.
a) Spawn an object when pinching starts.
b) Spawn objects to the trajectory of the hand.
c) Set the color of the next object to be added.
when the pinch ends.
5.Describe the action of the left hand.
a) Delete all objects at the start of pinch
6.Extra
a) Always track the position of the index finger
Slide 16
Slide 16 text
Tutorial Procedures
1.Explain the overview of the sample.
2.Add hand tracking feature.
3.Prepare work with hand tracking by javascript.
4.Describe the action of the right hand.
a) Spawn an object when pinching starts.
b) Spawn objects to the trajectory of the hand.
c) Set the color of the next object to be added.
when the pinch ends.
5.Describe the action of the left hand.
a) Delete all objects at the start of pinch
6.Extra
a) Always track the position of the index finger
Slide 17
Slide 17 text
Overview of the Sample
Lesson01
Slide 18
Slide 18 text
Overview of the Sample
Loading A-Frame
Describes content to be
shown in 3D space.
Slide 19
Slide 19 text
Overview of the Sample
a-box
position
0 1.5 -2
Slide 20
Slide 20 text
Overview of the Sample
Describe the
behavior of
the hand later.
Slide 21
Slide 21 text
Overview of the Sample
window.onload = function() {
initRightHand(); //右⼿の初期化
initLeftHand(); //左⼿の初期化
};
//右⼿の初期化を⾏う関数
function initRightHand(){
};
//左⼿の初期化を⾏う関数
function initLeftHand(){
};
Automatically called
once the page has
loaded.
Function prepared to
describe the behavior
of the right hand.
Function prepared to
describe the behavior
of the left hand.
Slide 22
Slide 22 text
Tutorial Procedures
1.Explain the overview of the sample.
2.Add hand tracking feature.
3.Prepare work with hand tracking by javascript.
4.Describe the action of the right hand.
a) Spawn an object when pinching starts.
b) Spawn objects to the trajectory of the hand.
c) Set the color of the next object to be added.
when the pinch ends.
5.Describe the action of the left hand.
a) Delete all objects at the start of pinch
6.Extra
a) Always track the position of the index finger
Slide 23
Slide 23 text
Show Hand Objects
Edit here
Slide 24
Slide 24 text
Show Hand Objects
【hand-tracking-controls】
hand: left or right
modelStyle: mesh pr dots
(mesh doesnʼt work)
modelColor: color of hand
(Default is white) Lesson02
Slide 25
Slide 25 text
Check on the Device
Be sure to click the Reload button.
Reload
Reload
Slide 26
Slide 26 text
Check on the Device
Slide 27
Slide 27 text
Tutorial Procedures
1.Explain the overview of the sample.
2.Add hand tracking feature.
3.Prepare work with hand tracking by javascript.
4.Describe the action of the right hand.
a) Spawn an object when pinching starts.
b) Spawn objects to the trajectory of the hand.
c) Set the color of the next object to be added.
when the pinch ends.
5.Describe the action of the left hand.
a) Delete all objects at the start of pinch
6.Extra
a) Always track the position of the index finger
Slide 28
Slide 28 text
Prepare Work with Hand Tracking
Edit here
Slide 29
Slide 29 text
Prepare Work with Hand Tracking
window.onload = function() {
initRightHand(); //右⼿の初期化
initLeftHand(); //左⼿の初期化
};
//右⼿の初期化を⾏う関数
function initRightHand(){
};
//左⼿の初期化を⾏う関数
function initLeftHand(){
};
Add variables here
Slide 30
Slide 30 text
Prepare Work with Hand Tracking
var scene; //3D space to add 3D objects
var rightHand; //Variable to access the right hand.
var leftHand; //Variable to access the left hand.
window.onload = function() {
initRightHand(); //右⼿の初期化
initLeftHand(); //左⼿の初期化
};
//右⼿の初期化を⾏う関数
function initRightHand(){
};
//左⼿の初期化を⾏う関数
function initLeftHand(){
}; Lesson03
Slide 31
Slide 31 text
Prepare Work with Hand Tracking
var scene; //3D space to add 3D objects
var rightHand; //Variable to access the right hand
var leftHand; //Variable to access the left hand
window.onload = function() {
// Keeps information in 3D space as variables.
scene = document.querySelector('a-scene');
initRightHand(); //右⼿の初期化
initLeftHand(); //左⼿の初期化
};
//右⼿の初期化を⾏う関数
function initRightHand(){
};
//左⼿の初期化を⾏う関数
function initLeftHand(){ };
Lesson04
Slide 32
Slide 32 text
Tutorial Procedures
1.Explain the overview of the sample.
2.Add hand tracking feature.
3.Prepare work with hand tracking by javascript.
4.Describe the action of the right hand.
a) Spawn an object when pinching starts.
b) Spawn objects to the trajectory of the hand.
c) Set the color of the next object to be added.
when the pinch ends.
5.Describe the action of the left hand.
a) Delete all objects at the start of pinch
6.Extra
a) Always track the position of the index finger
Slide 33
Slide 33 text
Create Object by Right Hand Operation
var scene; //描画する3D空間
var rightHand; //右⼿
var leftHand; //左⼿
window.onload = function() {
//3D空間を変数として保持
scene = document.querySelector('a-scene');
initRightHand(); //右⼿の初期化
initLeftHand(); //左⼿の初期化
};
//右⼿の初期化を⾏う関数
function initRightHand(){
};
//左⼿の初期化を⾏う関数
function initLeftHand(){ };
Edit here
Slide 34
Slide 34 text
Create Object by Right Hand Operation
function initRightHand(){
//Get the right hand object (id=rightHand)
rightHand = document.getElementById("rightHand");
//Begin pinch
rightHand.addEventListener('pinchstarted', function (e) {
});
//End pinch
rightHand.addEventListener('pinchended', function (e) {
});
//Move hand while pinching
rightHand.addEventListener('pinchmoved', function (e) {
});
}; Lesson05
Slide 35
Slide 35 text
Getting the Pinch Start
function initRightHand(){
//Get the right hand object (id=rightHand)
rightHand = document.getElementById("rightHand");
//Begin pinch
rightHand.addEventListener('pinchstarted', function (e) {
});
//End pinch
rightHand.addEventListener('pinchended', function (e) {
});
// Move hand while pinching
rightHand.addEventListener('pinchmoved', function (e) {
});
};
Edit here
Slide 36
Slide 36 text
Getting the Pinch Start
function initRightHand(){
// Get the right hand object (id=rightHand)
rightHand = document.getElementById("rightHand");
// Begin pinch
rightHand.addEventListener('pinchstarted', function (e) {
var position = e.detail.position;
//Calling the addBOX function (explained in the next page)
addBox(position);
});
// End pinch
rightHand.addEventListener('pinchended', function (e) {
});
// Move hand while pinching
rightHand.addEventListener('pinchmoved', function (e) {
});
}; Lesson06
Slide 37
Slide 37 text
Generating Box Object
Add the function here
Slide 38
Slide 38 text
Generating Box Object
//左⼿の初期化を⾏う関数
function initLeftHand(){
};
function addBox(position){
//Create an element named a-box
var box = document.createElement('a-box');
// Set the position, size, and color.
box.setAttribute('position', position);
box.setAttribute('scale', '0.02 0.02 0.02');
box.setAttribute('color', '#0062C6');
// Set class names for easy deletion later.
box.setAttribute('class', 'box');
// Add a box in 3D space.
scene.appendChild(box);
}
Lesson07
Slide 39
Slide 39 text
Check on the Device
Slide 40
Slide 40 text
Tutorial Procedures
1.Explain the overview of the sample.
2.Add hand tracking feature.
3.Prepare work with hand tracking by javascript.
4.Describe the action of the right hand.
a) Spawn an object when pinching starts.
b) Spawn objects to the trajectory of the hand.
c) Set the color of the next object to be added.
when the pinch ends.
5.Describe the action of the left hand.
a) Delete all objects at the start of pinch
6.Extra
a) Always track the position of the index finger
Slide 41
Slide 41 text
Follows the Movement of the Hand
function initRightHand(){
// Get the right hand object (id=rightHand)
rightHand = document.getElementById("rightHand");
// Begin pinch
rightHand.addEventListener('pinchstarted', function (e) {
/*omitted*/
});
// End pinch
rightHand.addEventListener('pinchended', function (e) {
});
// Move hand while pinching
rightHand.addEventListener('pinchmoved', function (e) {
});
};
Edit here
Slide 42
Slide 42 text
Follows the Movement of the Hand
// Get the right hand object (id=rightHand)
rightHand = document.getElementById("rightHand");
// Begin pinch
rightHand.addEventListener(ʻpinchstartedʼ, function (e) {
/*omitted*/
});
// End pinch
rightHand.addEventListener('pinchended', function (e) {
});
// Move hand while pinching
rightHand.addEventListener('pinchmoved', function (e) {
var position = e.detail.position;
//generate a box
addBox(position);
}); Lesson08
Slide 43
Slide 43 text
Check on the Device
Slide 44
Slide 44 text
Problem
If you generate the box too densely, it will quickly consume memory
Slide 45
Slide 45 text
Reduce the Density of Boxes
var scene;
var rightHand;
var leftHand;
var prev;//To store the position where the last box was placed.
window.onload = function() {
//3D空間を変数として保持
scene = document.querySelector('a-scene');
initRightHand();
initLeftHand();
};
//右⼿の初期化を⾏う関数
function initRightHand(){
/*以下省略*/
Lesson09
Slide 46
Slide 46 text
Reduce the Density of Boxes
var prev;//最後にBox置いた位置を記憶
window.onload = function() {
/*omitted*/
};
//右⼿の初期化を⾏う関数
function initRightHand(){
/*omitted*/
};
//左⼿の初期化を⾏う関数
function initLeftHand(){
};
//Function to add box
function addBox(position){
}
Edit here
Slide 47
Slide 47 text
Reduce the Density of Boxes
function addBox(position){
// Create an element named a-box
var box = document.createElement('a-box');
// Set the position, size, and color
box.setAttribute('position', position);
box.setAttribute('scale', '0.02 0.02 0.02');
box.setAttribute('color', '#0062C6');
// Set class names for easy deletion later
box.setAttribute('class', 'box');
//Add a box in 3D space.
scene.appendChild(box);
// Memorize the position where the box was placed.
prev=box.object3D.position;
}
Lesson09
Slide 48
Slide 48 text
Reduce the Density of Boxes
var prev;//最後にBox置いた位置を記憶
window.onload = function() {
/*omitted*/
};
//右⼿の初期化を⾏う関数
function initRightHand(){
/*omitted*/
};
//左⼿の初期化を⾏う関数
function initLeftHand(){
};
//Boxの追加を⾏う関数
function addBox(position){
}
Edit here
Slide 49
Slide 49 text
Reduce the Density of Boxes
function initRightHand(){
//Get the right hand object (id=rightHand)
rightHand = document.getElementById("rightHand");
// Begin pinch
rightHand.addEventListener('pinchstarted', function (e) {
/*omitted*/
});
// End pinch
rightHand.addEventListener('pinchended', function (e) {
});
// Move hand while pinching
rightHand.addEventListener('pinchmoved', function (e) {
});
};
Edit here
Slide 50
Slide 50 text
Reduce the Density of Boxes
rightHand.addEventListener('pinchmoved', function (e) {
var position = e.detail.position;
// Distance between the last position of the box and
// the current position of the hand
var length= prev.distanceTo(position);
if(length>=0.02){
//Generate a box
addBox(position);
}
});
Lesson10
prev
position
一定距離以上
Slide 51
Slide 51 text
Check on the Device
Slide 52
Slide 52 text
Tutorial Procedures
1.Explain the overview of the sample.
2.Add hand tracking feature.
3.Prepare work with hand tracking by javascript.
4.Describe the action of the right hand.
a) Spawn an object when pinching starts.
b) Spawn objects to the trajectory of the hand.
c) Set the color of the next object to be added.
when the pinch ends.
5.Describe the action of the left hand.
a) Delete all objects at the start of pinch
6.Extra
a) Always track the position of the index finger
Slide 53
Slide 53 text
Switch Color of Box
var scene;
var rightHand;
var leftHand;
var prev;
// Array of colors to assign to a cube
var colors = ['#0062C6', '#00AB00', '#FFCF00', '#7F00FF',
'#FFFFFF', '#FF2000', '#F52394', '#593110'];
// Index of color array to use
var index=0;
window.onload = function() {
//3D空間を変数として保持
scene = document.querySelector('a-scene');
initRightHand(); //右⼿の初期化
initLeftHand(); //左⼿の初期化
}; Lesson11
Slide 54
Slide 54 text
Switch Color of Box
function initRightHand(){
// Get the right hand object (id=rightHand)
rightHand = document.getElementById("rightHand");
// Begin pinch
rightHand.addEventListener('pinchstarted', function (e) {
/*omitted*/
});
// End pinch
rightHand.addEventListener('pinchended', function (e) {
});
// Move hand while pinching
rightHand.addEventListener('pinchmoved', function (e) {
/*omitted*/
});
};
Edit here
Slide 55
Slide 55 text
Switch Color of Box
function initRightHand(){
// Get the right hand object (id=rightHand)
rightHand = document.getElementById("rightHand");
// Begin pinch
rightHand.addEventListener('pinchstarted', function (e) {
var position = e.detail.position;
//Generate a box
addBox(position);
});
// End pinch
rightHand.addEventListener('pinchended', function (e) {
// Add up the number of colors.
index++;
index%=colors.length;
});
Lesson12
Slide 56
Slide 56 text
Switch Color of Box
window.onload = function() {
/*省略*/
};
//右⼿の初期化を⾏う関数
function initRightHand(){
/*省略*/
};
//左⼿の初期化を⾏う関数
function initLeftHand(){
};
//Function to generate a box
function addBox(position){
}
Edit here
Slide 57
Slide 57 text
Switch Color of Box
function addBox(position){
// Create an element named a-box
var box = document.createElement('a-box');
// Set the position, size, and color
box.setAttribute('position', position);
box.setAttribute('scale', '0.02 0.02 0.02ʼ);
box.setAttribute('color', '#0062C6');
box.setAttribute('color', colors[index]);
// Set class names for easy deletion later
box.setAttribute('class', 'box');
// Add a box in 3D space
scene.appendChild(box);
// Memorize the position where the box was placed
prev=box.object3D.position;
} Lesson13
box.setAttribute('color', '#0062C6');
Slide 58
Slide 58 text
Chek on the Device
Slide 59
Slide 59 text
Tutorial Procedures
1.Explain the overview of the sample.
2.Add hand tracking feature.
3.Prepare work with hand tracking by javascript.
4.Describe the action of the right hand.
a) Spawn an object when pinching starts.
b) Spawn objects to the trajectory of the hand.
c) Set the color of the next object to be added.
when the pinch ends.
5.Describe the action of the left hand.
a) Delete all objects at the start of pinch
6.Extra
a) Always track the position of the index finger
Slide 60
Slide 60 text
Delete All Boxes
var prev;//最後にBox置いた位置を記憶
window.onload = function() {
/*omitted*/
};
//右⼿の初期化を⾏う関数
function initRightHand(){
/* omitted */
};
//左⼿の初期化を⾏う関数
function initLeftHand(){
};
//Function to generate a box
function addBox(position){
/*omitted */
}
Edit here
Slide 61
Slide 61 text
Delete All Boxes
function initLeftHand(){
// Get the left hand object.
leftHand = document.getElementById("leftHand");
// Behavior at the start of a pinch
leftHand.addEventListener('pinchstarted', function (e) {
// Get all objects to which the box class is assigned.
var els = document.querySelectorAll('.boxʼ);
// Delete the boxes one by one.
for (var i = 0; i < els.length; i++) {
els[i].parentNode.removeChild(els[i]);
}
});
};
Lesson14
Slide 62
Slide 62 text
Completed!
Slide 63
Slide 63 text
Tutorial Procedures
1.Explain the overview of the sample.
2.Add hand tracking feature.
3.Prepare work with hand tracking by javascript.
4.Describe the action of the right hand.
a) Spawn an object when pinching starts.
b) Spawn objects to the trajectory of the hand.
c) Set the color of the next object to be added.
when the pinch ends.
5.Describe the action of the left hand.
a) Delete all objects at the start of pinch
6.Extra
a) Always track the position of the index finger
Slide 64
Slide 64 text
Getting the Position of Finger Tip
Edit inside a-scene
Slide 65
Slide 65 text
Getting the Position of Finger Tip
Lesson15
Slide 66
Slide 66 text
Write Scripts
See
javascript
Slide 67
Slide 67 text
Getting the Position of Finger Tip
// Array of colors to assign to a cube
var colors = ['#0062C6', '#00AB00', '#FFCF00', '#7F00FF',
'#FFFFFF', '#FF2000', '#F52394', '#593110'];
// Index of color array to use
var index=0;
//Sphere representing the tip of the finger
var tip;
window.onload = function() {
//3D空間を変数として保持
scene = document.querySelector('a-scene');
//Get the sphere representing the fingertip position
tip = document.getElementById("tip");
initRightHand(); //右⼿の初期化
initLeftHand(); //左⼿の初期化
};
Lesson16
Slide 68
Slide 68 text
Getting the Position of Finger Tip
var prev;//最後にBox置いた位置を記憶
window.onload = function() {
/*省略*/
};
//右⼿の初期化を⾏う関数
function initRightHand(){
/*omitted*/
};
//左⼿の初期化を⾏う関数
function initLeftHand(){
/*省略*/
};
//Boxの追加を⾏う関数
function addBox(position){
/*省略*/
}
Edit here
Slide 69
Slide 69 text
Getting the Position of Finger Tip
function initRightHand(){
// Get the right hand object (id=rightHand)
rightHand = document.getElementById("rightHand");
// Track finger tips in real time (to be implemented later)
tipTracking(rightHand.components['hand-tracking-controls']);
// Begin pinch
rightHand.addEventListener('pinchstarted', function (e) {
/*omitted*/
});
// End pinch
rightHand.addEventListener('pinchended', function (e) {
/*omitted*/
});
Lesson17
Slide 70
Slide 70 text
Getting the Position of Finger Tip
window.onload = function() {
/*omitted*/
};
function initRightHand(){
/* omitted(右⼿の初期化を⾏う関数)*/
};
function initLeftHand(){
/* omitted(左⼿の初期化を⾏う関数)*/
};
function addBox(position){
/* omitted(Boxの追加を⾏う関数)*/
}
Add a function for tracking
finger position here
Slide 71
Slide 71 text
Getting the Position of Finger Tip
function initLeftHand(){
/*omitted(左⼿の初期化を⾏う関数)*/
};
function addBox(position){
/*omitted(Boxの追加を⾏う関数)*/
}
function tipTracking(ctrl){
// Finger tip position
var position = ctrl.indexTipPosition;
// Assign finger position to the ball
tip.setAttribute('position', position);
// Call tipTracking again after 30ms
setTimeout(function(){tipTracking(ctrl);}, 30);
};
Lesson18
Slide 72
Slide 72 text
Assign the Box Color to the Fingertip
function initRightHand(){
//右⼿のオブジェクト(id=rightHand)を取得
rightHand = document.getElementById("rightHand");
//ピンチ開始
rightHand.addEventListener('pinchstarted', function (e) {
/*中略*/
});
//End pinch
rightHand.addEventListener('pinchended', function (e) {
});
//ピンチしながら⼿を移動
rightHand.addEventListener('pinchmoved', function (e) {
/*中略*/
});
};
Edit here
Slide 73
Slide 73 text
次のBoxの⾊を指先の球に反映
// Begin pinch
rightHand.addEventListener('pinchstarted', function (e) {
var position = e.detail.position;
//BOX⽣成関数を呼び出す
addBox(position);
});
// End pinch
rightHand.addEventListener('pinchended', function (e) {
//⾊の番号を加算してboxの⾊に反映
index++;
index%=colors.length;
// Assing the color to the ball at the tip of the finger
tip.setAttribute('color', colors[index]);
});
Lesson19
Slide 74
Slide 74 text
Extra part was
completed!