Getting Started with ARCore 1.7

Slide 1

Slide 1 text

Getting Started with ARCore 1.7 #AR_Fukuoka

Slide 2

Slide 2 text

Name： Takashi Yoshinaga Affiliation: Institute of Systems,Information Technologies and Nanotechnologies Field of work： Application of AR for med. and edu. SNS: Twitter -> @Tks_Yoshinaga LinkedIn -> tks-yoshinaga

Slide 3

Slide 3 text

ARCore 【Features】 (1) Motion Tracking for realize marker less AR. (2) Light Estimation which can estimate light strength and color. (3) Environmental Understanding to recognize floor and wall. (4) Augmented Image to recognize predefined image marker. (5) Cloud Anchor for sharing AR experience among multiple users. (6) Augmented Faces to recognize face & pose estimation.

Slide 4

Slide 4 text

Today’s Tutorial 【Features】 (1) Motion Tracking for realize marker less AR. (2) Light Estimation which can estimate light strength and color. (3) Environmental Understanding to recognize floor and wall. (4) Augmented Image to recognize predefined image marker. (5) Cloud Anchor for sharing AR experience among multiple users. (6) Augmented Faces to recognize face & pose estimation.

Slide 5

Slide 5 text

See also below if you want to know Augmented Image and Cloud Anchor. https://speakerdeck.com/takashiyoshin aga/lets-start-ar-with-arcore-and-unity

Slide 6

Slide 6 text

Today’s Tutorial ① Superimposing CG ② Plane Detection and Placing CG ③ 3D Drawing with Motion Tracking ④ How to Use Augmented Faces

Slide 7

Slide 7 text

Preparation • Unity2017.4.15 or later • ARCore SDK 1.7 https://github.com/google-ar/arcore- unity-sdk/releases/tag/v1.7.0 • Sample http://arfukuoka.lolipop.jp/arcore_pan asonic/sample.zip

Slide 8

Slide 8 text

Creating Unity Project (1/2) Click New button after starting Unity New

Slide 9

Slide 9 text

Creating Unity Project (2/2) Click Create Project button after inputting project name. 3D Project Name Save Directory Create Project

Slide 10

Slide 10 text

Importing ARCore SDK to Unity ①Assets ②Import Package → Custom Package ③arcore-unity-sdk-xxx ④開く

Slide 11

Slide 11 text

Tutorial 1: Motion Tracking

Slide 12

Slide 12 text

Setting up the Camera for ARCore (1/2) Delete Main Camera

Slide 13

Slide 13 text

Setting up the Camera for ARCore (2/2) ①GoogleARCore → Prefabs ② ARCore Device ③Drag & Drop

Slide 14

Slide 14 text

Putting a Virtual Object (1/2) ①GoogleARCore → Examples → Common → Prefabs ② Andy Green Diffuse ③Drag & Drop

Slide 15

Slide 15 text

Putting a Virtual Object (2/2) ①AndyGreenDiffuse ※Origin of the CG is position of smartphone when application was launched ②Put Andy in front of user. Position (x,y,z)=(0,0,0.5)[m]

Slide 16

Slide 16 text

Save the Project ①File ②Save Scene as...

Slide 17

Slide 17 text

Save the Project ①New Folder ③Open Sample1 folder and save as sample1 ④Save After this operation the content can be saved with Ctrl + S ②Name new folder as Sample1

Slide 18

Slide 18 text

Let’s install app to smartphone

Slide 19

Slide 19 text

Build Setting ①File ② Build Settings

Slide 20

Slide 20 text

Build Setting ②Switch Platform ① Android

Slide 21

Slide 21 text

Build Setting ②Player Settings ① Internal

Slide 22

Slide 22 text

Build Setting ①Input Product Name ② Other Settings ③Disable Multithreaded Rendering

Slide 23

Slide 23 text

Build Setting ①Input Package Name 例) com.yourname.test1 ② Set Minimum API Level to Android 7.0

Slide 24

Slide 24 text

Build Setting ①XR Settings ② Enable ARCore Supported

Slide 25

Slide 25 text

Build and Install ①File ② Build & Run

Slide 26

Slide 26 text

Build and Install ①Input a name of apk ② Save

Slide 27

Slide 27 text

Run & Confirm

Slide 28

Slide 28 text

Tutorial 2: Environmental Understanding

Slide 29

Slide 29 text

Environmental Understanding (1/5) ①Sample1

Slide 30

Slide 30 text

Environmental Understanding (2/5) ①Right Click ②Create ③GoogleARCore ④SessionConfig

Slide 31

Slide 31 text

Environmental Understanding (3/5) Change the file name (ex. sample1)

Slide 32

Slide 32 text

Environmental Understanding (4/5) ①Click sample1.asset 【Plane Finding Mode】 (1)Disabled (2)Horizontal And Vertical (3)Horizontal (4)Vertical Choose a mode from(2)~(4)

Slide 33

Slide 33 text

Environmental Understanding (5/5) ②Click ARCore Device ①Sample1 ③sample1.asset ④Drag & Drop to Session Config

Slide 34

Slide 34 text

Visualization of Detected Planes (1/4) ①Right Click ② Create Empty

Slide 35

Slide 35 text

Visualization of Detected Planes (2/4) Change the name of GameObject to Controller

Slide 36

Slide 36 text

Visualization of Detected Planes (3/4) ①Click Controller ②AddComponent ③Detected Plane Generator

Slide 37

Slide 37 text

Visualization of Detected Planes (4/4) ①GoogleARCore → Examples → Common → Prefabs ③DetectedPlaneVisualizer ②Controller ④Drag & Drop to Detected Plane Prefab

Slide 38

Slide 38 text

Run & Confirmation Horizontal Plane Vertical Plane

Slide 39

Slide 39 text

Putting Object to the Tapped Place ①Controller ②Add Component

Slide 40

Slide 40 text

Putting Object to the Tapped Place ①New Script ③Create and Add ②PutScript

Slide 41

Slide 41 text

Putting Object to the Tapped Place ①Controller ②Double Click PutScript

Slide 42

Slide 42 text

Script using System.Collections; using System.Collections.Generic; using UnityEngine; using GoogleARCore; public class PutScript : MonoBehaviour { public GameObject andy; //Variable to handle CG(Andy) void Start () { } void Update () { //(1) Detect tap. //(2) Transform 2D position to 3D position of real world. //(3) Put Andy there. } }

Slide 43

Slide 43 text

Script void Update () { //Return if screen touch is not detected if (Input.touchCount < 1 ){ return; } Touch touch = Input.GetTouch(0); //Return if touch state is not swipe. if (touch.phase != TouchPhase.Moved ){ return;} //Calculate touched position of detected plane. TrackableHit hit; TrackableHitFlags filter = TrackableHitFlags.PlaneWithinPolygon; if (Frame.Raycast(touch.position.x, touch.position.y, filter, out hit)) { //Move Andy to pointed position. (Next page) } } touch.position hit

Slide 44

Slide 44 text

Script if (Frame.Raycast(touch.position.x, touch.position.y, filter, out hit)) { // If pointed trackable is plane if (hit.Trackable is DetectedPlane ) { //Set the position and the angle of Andy andy.transform.position = hit.Pose.position; andy.transform.rotation = hit.Pose.rotation; andy.transform.Rotate(0, 180, 0, Space.Self); //Set the anchor to fix Andy object to real space. var anchor = hit.Trackable.CreateAnchor(hit.Pose); andy.transform.parent = anchor.transform; } }

Slide 45

Slide 45 text

Coupling Virtual Object and Variable ②AndyGreenDiffuse ①Controller ③ Drag & Drop to Put Script’s Andy

Slide 46

Slide 46 text

Run & Confirm You can place Andy on the touched position of a detected plane.

Slide 47

Slide 47 text

Save this project by Ctrl+S

Slide 48

Slide 48 text

Tutorial 3: 3D Drawing with Motion Tracking

Slide 49

Slide 49 text

Preparation for Next Tutorial (1/6) Select Sample1 & Ctrl +D

Slide 50

Slide 50 text

Preparation for Next Tutorial (2/6) ②Rename each file to sample2 ①sample2

Slide 51

Slide 51 text

Preparation for Next Tutorial (3/6) ①Double click sample2.unity ②sample2 will appear

Slide 52

Slide 52 text

Preparation for Next Tutorial (4/6) ①ARCoreDevice ②sample2.asset ③Drag & Drop into Session Config

Slide 53

Slide 53 text

Preparation for Next Tutorial (5/6) Delete Andy Dffuse

Slide 54

Slide 54 text

Preparation for Next Tutorial (6/6) ①Controller ②Click of PutScript ③Remove Component

Slide 55

Slide 55 text

Run & Confirmation Horizontal Plane Vertical Plane

Slide 56

Slide 56 text

Making Material of Line ①Right Click ②Create ③Material

Slide 57

Slide 57 text

Making Material of Line ①New Material ②Create white box and choose color

Slide 58

Slide 58 text

Drawing 3D Line with TrailRenderer ①Right Click ②Create Empty

Slide 59

Slide 59 text

Drawing 3D Line with TrailRenderer ①GameObject ②Add Component

Slide 60

Slide 60 text

Drawing 3D Line with TrailRenderer Search “trail” 【Trail Renderer】 ■Material Drag & Drop NewMaterial into Element0 ■Time Change to Infinity ■MinVertexDistance 0.03 ■Width 0.01 Double click Trail Renderer

Slide 61

Slide 61 text

Drawing 3D Line with TrailRenderer Change Shader to Sprites/Default ①GameObject

Slide 62

Slide 62 text

Run & Confirm ①Scene Line appears while dragging & moving GameObject ②GameObject

Slide 63

Slide 63 text

Enable Drawing Multi-line ②GameObject ①Sample2 ③Drag & Drop

Slide 64

Slide 64 text

Enable Drawing Multi-line Delete GameObjec

Slide 65

Slide 65 text

Enable Drawing Multi-line ①Controller ②AddComponent

Slide 66

Slide 66 text

Enable Drawing Multi-line ②New Script ④Create and Add ③DrawScript ①Clear form

Slide 67

Slide 67 text

Enable Drawing Multi-line ①Controller ②Double Click DrawScript

Slide 68

Slide 68 text

Script using System.Collections; using System.Collections.Generic; using UnityEngine; using GoogleARCore; public class DrawScript : MonoBehaviour { public GameObject obj; //Template of trail drawing object GameObject drawObj; // Object used for actual trail drawing void Start () { } void Update () { //Detect Tap //Instantiate drawing object when screen touch is started. //Draw trail line while user touching screen. } }

Slide 69

Slide 69 text

Script void Update () { if (Input.touchCount == 1) { //Calculate relative position from camera. 0 0 0.1 [m] Vector3 p = Camera.main.transform.TransformPoint(0,0,0.1f); //Touch is started if ( Input.GetTouch(0).phase == TouchPhase.Began) { drawObj = GameObject.Instantiate(obj, p, Quaternion.identity); } //While touching. else if (Input.GetTouch(0).phase == TouchPhase.Stationary){ drawObj.transform.position = p; } } }

Slide 70

Slide 70 text

Coupling Virtual Object and Variable ②Controller ①Sample2 ④Drag & Drop into Obj ③GameObject

Slide 71

Slide 71 text

Run & Confirmation

Slide 72

Slide 72 text

Enable Deleting Line List lines = new List(); void Update () { if (Input.touchCount == 1) {//カメラ手前10cmの位置を取得 Vector3 p = Camera.main.transform.TransformPoint(0,0,0.1f);//タ if ( Input.GetTouch(0).phase == TouchPhase.Began) { drawObj = GameObject.Instantiate(obj, p, Quaternion.identity); GameObject tmp = GameObject.Instantiate(obj, p, Quaternion.identity); lines.Add(tmp); drawObj = tmp; } //押下中 else if (Input.GetTouch(0).phase == TouchPhase.Stationary){ drawObj.transform.position = p; } } }

Slide 73

Slide 73 text

Enable Deleting Line void Update () { if (Input.touchCount == 1) { //Code for writing trail lines./カメラ手前10cmの位置を取得 } else if (Input.touchCount == 2) { if (Input.GetTouch(0).phase == TouchPhase.Ended) { for(int i = 0; i < lines.Count; i++) { Destroy(lines[i]); lines[i] = null; } lines.Clear(); } } }

Slide 74

Slide 74 text

Lines will be deleted when you tap a screen of smartphone with two fingers!

Slide 75

Slide 75 text

Save the project by Ctrl+S

Slide 76

Slide 76 text

Tutorial 4：Augmented Faces

Slide 77

Slide 77 text

Preparation for Next Tutorial (1/6) Select Sample2 & Ctrl +D

Slide 78

Slide 78 text

Preparation for Next Tutorial (2/6) ②Change each name to sample3 ①sample3

Slide 79

Slide 79 text

Preparation for Next Tutorial (3/6) ①Double click sample3.unity ②sample3 will appear

Slide 80

Slide 80 text

Preparation for Next Tutorial (4/6) ①ARCoreDevice ②sample3.asset ③Drag & Drop to Session Config

Slide 81

Slide 81 text

Preparation for Next Tutorial (5/6) Delete GameObject

Slide 82

Slide 82 text

Preparation for Next Tutorial (6/6) Delete Controller

Slide 83

Slide 83 text

Enable Front Camera ①ARCoreDevice ②Chanbe Device Camera Direction to Front Facing

Slide 84

Slide 84 text

Enable Face Detection ②sample3.asset ①sample3 ③Augmented Face Mode

Slide 85

Slide 85 text

Attach a Mesh to Face ②CreateEmpty ①右クリック

Slide 86

Slide 86 text

Attach a Mesh to Face Change the name of GameObject to FaceMesh

Slide 87

Slide 87 text

Attach a Mesh to Face ①FaceMesh ②Add Component

Slide 88

Slide 88 text

Attach a Mesh to Face ①Search Face ②ARCore Augmented Face Mesh Filter

Slide 89

Slide 89 text

Attach a Mesh to Face Check Auto Bind ※Enable auto attachment of mesh to face

Slide 90

Slide 90 text

Adding Mesh Renderer ①FaceMesh ②Add Coponent

Slide 91

Slide 91 text

Adding Mesh Renderer ②Mesh ③MeshRenderer ①Clear form

Slide 92

Slide 92 text

Setting Up Material of Face Mesh ①Sample3 ③Create ②Right Click ④Material

Slide 93

Slide 93 text

Setting Up Material of Face Mesh ①NewMaterial ②Shader

Slide 94

Slide 94 text

Augmented Faces ②NewMaterial ①FaceMesh ②Materials内のElement0にドラッグ&ドロップ

Slide 95

Slide 95 text

Run & Confirm

Slide 96

Slide 96 text

Applying a Texture Image to Material ②Texture ※You can use image with alpha channel by choosing Transparent ①Unlit

Slide 97

Slide 97 text

Applying a Texture Image to Material ③Drag&Drop ①Sample Folder ①Sample3

Slide 98

Slide 98 text

Applying a Texture Image to Material ①NewMaterial

Slide 99

Slide 99 text

Applying a Texture Image to Material ②Drag&Drop ①Mesh

Slide 100

Slide 100 text

Run & Confirm You can estimate positional relationship between face & texture image.

Slide 101

Slide 101 text

Using Position of Face Parts FOREHEAD_LEFT FOREHEAD_RIGHT NOSE_TIP 3 parts, named FOREHEAD_LEFT/RIGHT & NOSE_TIP, are available in SDK.

Slide 102

Slide 102 text

Using Position of Face Parts ②CreateEmpty ①Right Click

Slide 103

Slide 103 text

Using Position of Face Parts ①Change name of GameObject to FaceParts

Slide 104

Slide 104 text

Using Position of Face Parts ①FaceParts ②Add Compornent

Slide 105

Slide 105 text

Using Position of Face Parts ①Search “Face” ②ARCore Augmented Face Rig

Slide 106

Slide 106 text

Using Position of Face Parts ①Auto Bind

Slide 107

Slide 107 text

Augmented Faces ①Open FaceParts ②You can see NOSE_TIP, FOREHEAD_LFT/RIHGT

Slide 108

Slide 108 text

Using Position of Face Parts ①Right click NOSE_TIP ②3D Object ③Sphere

Slide 109

Slide 109 text

Using Position of Face Parts ①Sphere ②Edit Transform Position: all 0 Scale: all 0.04

Slide 110

Slide 110 text

Using Position of Face Parts Add sphere as child of FOREHEAD_RIGHT and LEFT by the same procedure

Slide 111

Slide 111 text

Using Position of Face Parts

Slide 112

Slide 112 text

Modification of Sphere’s Position ① Select Sphere being child of FOREHEAD_LEFT ② PositionのY=-0.04

Slide 113

Slide 113 text

Modification of Sphere’s Position ① Select Sphere being child of FOREHEAD_RIGHT ② PositionのY=-0.04

Slide 114

Slide 114 text

Complete