Getting Started with Non-Programming AR Development with MRTK v2.4.0

by TakashiYoshinaga

Slide 1

Slide 1 text

Getting Started with Non-Programming AR Development with MRTK

Slide 2

Slide 2 text

Speaker Name︓吉永崇(Takashi Yoshinaga) Affiliation︓Steampunk Digital ISIT Community: AR Fukuoka

Slide 3

Slide 3 text

SNS https://www.linkedin.com/in/tks-yoshinaga/ https://twitter.com/Taka_Yoshinaga LinkedIn Twitter

Slide 4

Slide 4 text

#AR_Fukuoka Hashtag

Slide 5

Slide 5 text

Download Sample Data http://arfukuoka.lolipop.jp/mrtk_np/Sample.zip

Slide 6

Slide 6 text

Download MRTK v2.4.0 (1/3) https://github.com/Microsoft/MixedRealityToolkit-Unity Releases

Slide 7

Slide 7 text

Download MRTK v2.4.0 (2/3) MRTK v2.4.0 is used for this tutorial Scroll down

Slide 8

Slide 8 text

Download MRTK v2.4.0 (3/3) Download only Foundation.2.4.0.unitypackage Foundation.2.4.0.unitypackage

Slide 9

Slide 9 text

Todayʼs Goal Show and Manipulate Virtual Object without Programming https://youtu.be/Yae3bOtidTk

Slide 10

Slide 10 text

Creating Project (1/6) Launch Unity Hub and click Project プロジェクト

Slide 11

Slide 11 text

Creating Project (2/6) Click ▽ aligned on the right side of NEW button. 新規作成横の▽

Slide 12

Slide 12 text

Creating Project (3/6) Select 2018.4.xxf1 (2018.4.23f1 is selected in this deck) 2018.4.xxf1

Slide 13

Slide 13 text

Creating Project (4/6) ①3D ②Set the project name (ex.ARFukuoka_MRTK) ③Select the directory to save the project

Slide 14

Slide 14 text

Creating Project (5/6) Create

Slide 15

Slide 15 text

Creating Project (6/6) Unity Editor will be shown

Slide 16

Slide 16 text

Installation of MRTK (1/3) Double click the unitypackage. Click Import button of Import Unity Package dialog. Microsoft.MixedReality. Toolkit.Unity.Foundation .2.4.0.unitypackage Import

Slide 17

Slide 17 text

Installation of MRTK (2/3) Apply Click Apply button to switch the setting for XR.

Slide 18

Slide 18 text

Installation of MRTK (3/3) Close Close MRTK Project Configurator dialog.

Slide 19

Slide 19 text

Adding 3D Object into the Space (1/3) Right click blank of Hierarchy

Slide 20

Slide 20 text

Adding 3D Object into the Space (2/3) ①3D Object ②Quad

Slide 21

Slide 21 text

Adding 3D Object into the Space (3/3) Quad is shown ※Backside is invisible

Slide 22

Slide 22 text

Adjust Viewpoint of Scene [←] [→]: Move to Left/RIght [↑][↓]: Zoom In/Out [Alt]+Drag: Rotation +Drag: Move to ←→↑↓

Slide 23

Slide 23 text

Adjust Viewpoint of Scene Object can be shown on the center of Scene tab. Double click Quad

Slide 24

Slide 24 text

Run カメラから⾒た空間が表⽰される Click to play Click it again to stop preview

Slide 25

Slide 25 text

Modification of Objectʼs Transform Click Quad

Slide 26

Slide 26 text

Modification of Objectʼs Transform Move Rotate Scale

Slide 27

Slide 27 text

Inputting Values to Transform Object Click Quad

Slide 28

Slide 28 text

Inputting Values to Transform Object Inspector p Properties of each object can be added/edited in Inspector p Position/Rotation/Scale are editable in Transform.

Slide 29

Slide 29 text

Inputting Values to Transform Object p Properties of each object can be added/edited in Inspector p Position/Rotation/Scale are editable in Transform. Set as following Positionを 0 0 1 Rotation 0 0 0 Scale 0.3 0.3 0.3

Slide 30

Slide 30 text

Adjust Viewpoint of Scene Letʼs set the object at the center of scene tab

Slide 31

Slide 31 text

Adjust Viewpoint of Scene Double Click Quad

Slide 32

Slide 32 text

Next Step Not only Transform but also appearance and behavior and properties can be edited in Inspector

Slide 33

Slide 33 text

Changing Color of Cube (1/4) ① Quad ② See Mesh Renderer ③ Open Materials

Slide 34

Slide 34 text

Changing Color of Cube (2/4) Element0

Slide 35

Slide 35 text

Changing Color of Cube (3/4) Appearance information is described in Material file

Slide 36

Slide 36 text

Changing Color of Cube (4/4) Detail of Material is shown at the bottom of Inspector. But Default-Material is not editable.

Slide 37

Slide 37 text

Itʼs necessary to create Material file to edit appearance of an object.

Slide 38

Slide 38 text

Creating Material (1/3) Create Material to describe color of Quad. ①Assets ②Right Click

Slide 39

Slide 39 text

Creating Material (2/3) ①Create ②Material Create Material to describe color of Quad.

Slide 40

Slide 40 text

Creating Material (3/3) Create Material to describe color of Quad. New Material should appear

Slide 41

Slide 41 text

Application of Material (1/3) ①Quad Open Materials of Quad in Inspector ② See Default-Material

Slide 42

Slide 42 text

Application of Material (2/3) Replace Default-Material with New Material. Drag & Drop into Default-Material

Slide 43

Slide 43 text

Application of Material (3/3) ①Quad ②Material should be New Material

Slide 44

Slide 44 text

Next Step...

Slide 45

Slide 45 text

Application of Texture (1/8) ②Open Dropdown Menu ③Unlit ④Transparent ①See New Material

Slide 46

Slide 46 text

Application of Texture (2/8) ①Click ▼ to open NewMateria ②Texture can be set here

Slide 47

Slide 47 text

Application of Texture (3/8) ①Assets ②Import New Asset

Slide 48

Slide 48 text

Application of Texture (4/8) ①arfukuoka.png ②Import

Slide 49

Slide 49 text

Application of Texture (5/8) ①Quad ②Open New Materia

Slide 50

Slide 50 text

Application of Texture (6/8) ①Assets ② see arfukuoka.png (Do NOT click)

Slide 51

Slide 51 text

Application of Texture (7/8) Drag & Drop arfukuoka.png into None(Texture)

Slide 52

Slide 52 text

Application of Texture (8/8) Texture is applied

Slide 53

Slide 53 text

How to Save this Scene (1/4) File

Slide 54

Slide 54 text

How to Save this Scene (2/4) SaveAs...

Slide 55

Slide 55 text

How to Save this Scene (3/4) ①Name as Sample1 ②Save

Slide 56

Slide 56 text

How to Save this Scene (4/4) Sample1 should be appear in Assets folder

Slide 57

Slide 57 text

You can override the scene with current info by Ctrl / Command + S

Slide 58

Slide 58 text

Letʼs use MRTK from now!

Slide 59

Slide 59 text

What Youʼll Do with MRTK at First? Prepare moving around and using hand in UnityEditor

Slide 60

Slide 60 text

Using Camera of MRTK (1/3) MixedRealityToolkit

Slide 61

Slide 61 text

Using Camera of MRTK (2/3) Add to Scene and Configure...

Slide 62

Slide 62 text

Using Camera of MRTK (3/3) MixedRealityToolKit and MixedRealityPlayspace should appear

Slide 63

Slide 63 text

Play (1/4) Click Play

Slide 64

Slide 64 text

Play (2/4) [A][D] :Move to Left and Right [W][S] :Zoom In/Out [Q][E] :Move to Up and Down RightClick + Drag :Rotation

Slide 65

Slide 65 text

Play (3/4) ※Object canʼt be manipulated so far [Shift] :Show left hand [Space] :Show right hand While hand is appearing+ [Click] :Pinch [Scroll] :Forward/Back

Slide 66

Slide 66 text

Play (4/4) Stop before going to next step

Slide 67

Slide 67 text

Letʼs manipulate Quad

Slide 68

Slide 68 text

Making Quad Manipulatable (1/3) ①Quad ②Add Component

Slide 69

Slide 69 text

Making Quad Manipulatable (2/3) Search [manipulation] Select ManipulationHandler

Slide 70

Slide 70 text

Making Quad Manipulatable (3/3) ①Quad ②ManipulationHandler is added

Slide 71

Slide 71 text

Play (1/4) Playをクリック

Slide 72

Slide 72 text

One Hand Manipulation

Slide 73

Slide 73 text

Play (2/4) Shift or Space

Slide 74

Slide 74 text

Play (3/4) Put the cursor on Quad

Slide 75

Slide 75 text

Play (4/4) Drag and move Quad

Slide 76

Slide 76 text

Two Hand Manipulation

Slide 77

Slide 77 text

Play (1/5) Show left hand by pushing Shift

Slide 78

Slide 78 text

Play (2/5) ①Click after putting cursor on Quad ② Hit [T] to fix left hand

Slide 79

Slide 79 text

Play (3/5) Show right hand by pushing Space

Slide 80

Slide 80 text

Play (4/5) Drag and move

Slide 81

Slide 81 text

Play (5/5) You can Rotate/Scale Cube

Slide 82

Slide 82 text

For Smartphone

Slide 83

Slide 83 text

Play (1/2) Put the cursor shown in the center of view on Quad

Slide 84

Slide 84 text

Play (2/2) Drag to move

Slide 85

Slide 85 text

Letʼs Add Another Object

Slide 86

Slide 86 text

Adding Obj File(1/6) Assets

Slide 87

Slide 87 text

Adding Obj File(2/6) Sample Folder → Model

Slide 88

Slide 88 text

Adding Obj File (3/6) Drag & Drop Model into Assets

Slide 89

Slide 89 text

Adding Obj File (4/6) Model

Slide 90

Slide 90 text

Adding Obj File (5/6) model.obj

Slide 91

Slide 91 text

Adding Obj File (6/6) Drag & Drop model.obj into Hierarchy

Slide 92

Slide 92 text

Adjusting Object (1/3) ①model Put chair as following Position 0.5 0 1

Slide 93

Slide 93 text

Adjusting Object (2/3) Click ▼ at the left of mode to open child hierarchy

Slide 94

Slide 94 text

Adjusting Object (3/3) model under upper level of model Set trans form as bellow Position 0 -0.45 0 Rotation 0 180 0 It is possible to adjust the Transform using only the model in the upper level, but for the sake of efficiency, I divided the operation into two steps

Slide 95

Slide 95 text

In order to enable chair to detect contact with hand, Collider should be added to model.obj

Slide 96

Slide 96 text

Adding Collider (1/5) ①model (上の階層の⽅) ②Add Component

Slide 97

Slide 97 text

Adding Collider (2/5) ②BoxCollider ①Search box

Slide 98

Slide 98 text

Adding Collider (3/5) Bounding box will appear

Slide 99

Slide 99 text

Adding Collider (4/5) ①model ②Edit Collider

Slide 100

Slide 100 text

Adding Collider (5/5) Adjust position of each face by moving■

Slide 101

Slide 101 text

Make Chair Manipulatable (1/3) ①model ②Add Component

Slide 102

Slide 102 text

Make Chair Manipulatable (2/3) ②ManipulationHandler ① Searchmanipulation

Slide 103

Slide 103 text

Make Chair Manipulatable (3/3) ①model ② Manipulation Handler is added

Slide 104

Slide 104 text

Allow Only Scaling for 2 Hand Operation ①model ②Set Scale for Two Hand Manipulation Type

Slide 105

Slide 105 text

Play

Slide 106

Slide 106 text

Play Stop

Slide 107

Slide 107 text

Remove Performance Viewer Can be deleted

Slide 108

Slide 108 text

Remove Performance Viewer ①MixedRealityToolKit ② Open dropdown menu under Mixed Reality Toolkit

Slide 109

Slide 109 text

Remove Performance Viewer ①MixedRealityToolKit ②DefaultHololens2ConfigureProfile

Slide 110

Slide 110 text

Remove Performance Viewer Clone

Slide 111

Slide 111 text

Remove Performance Viewer ②Clone ①Change Profile Name to Sample1 MixedRealityToolkit...

Slide 112

Slide 112 text

Remove Performance Viewer ①Diagnostics ②Turn Enable Diagnostics System into Off

Slide 113

Slide 113 text

Play Performance Viewer become invisible

Slide 114

Slide 114 text

Play Stop

Slide 115

Slide 115 text

p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

Slide 116

Slide 116 text

p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

Slide 117

Slide 117 text

Build Setting for HoloLens2 File

Slide 118

Slide 118 text

Build Setting for HoloLens2 Build Setting

Slide 119

Slide 119 text

Build Setting for HoloLens2 ①Universal Windows Platform ②Switch Platform

Slide 120

Slide 120 text

Build Setting for HoloLens2 Apply

Slide 121

Slide 121 text

Build Setting for HoloLens2 Player Settings...

Slide 122

Slide 122 text

Build Setting for HoloLens2 ①XR Settings ②Select 16-bit depth ad Depth Format

Slide 123

Slide 123 text

Build for HoloLens2 Build

Slide 124

Slide 124 text

Build for HoloLens2 New Folder

Slide 125

Slide 125 text

Build for HoloLens2 Rename new folder to build

Slide 126

Slide 126 text

Build for HoloLens2 ①build ②Select Folder

Slide 127

Slide 127 text

Wait for finishing build

Slide 128

Slide 128 text

Build for HoloLens2 Open build

Slide 129

Slide 129 text

Build for HoloLens2 Open solution (*.sln) with VisualStudio2019

Slide 130

Slide 130 text

Build for HoloLens2 Release ARM

Slide 131

Slide 131 text

Build for HoloLens2 ▼

Slide 132

Slide 132 text

Build for HoloLens2 Device

Slide 133

Slide 133 text

Connect HoloLens2 and PC by using USB cable

Slide 134

Slide 134 text

Build for HoloLens2 ①デバッグ ②Start without debug

Slide 135

Slide 135 text

動作の様⼦

Slide 136

Slide 136 text

p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

Slide 137

Slide 137 text

p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

Slide 138

Slide 138 text

Installation of ARFoundation Window

Slide 139

Slide 139 text

Installation of ARFoundation Package Manager

Slide 140

Slide 140 text

Installation of ARFoundation Advanced

Slide 141

Slide 141 text

Installation of ARFoundation Turn ON Show preview packages

Slide 142

Slide 142 text

Installation of ARFoundation ①AR Foundation ②Open version list

Slide 143

Slide 143 text

Installation of ARFoundation ①All Version ②1.5.0-preview.6

Slide 144

Slide 144 text

Installation of ARFoundation Install

Slide 145

Slide 145 text

Setting procedure for iOS is from next page.

Slide 146

Slide 146 text

Installation of ARKit (for iOS) ①ARKit XR Plugin ②2.1.2

Slide 147

Slide 147 text

Installation of ARKit (for iOS) Install

Slide 148

Slide 148 text

Setting procedure for Android is from next page.

Slide 149

Slide 149 text

Installation of ARCore (for Android) ①ARCore XR Plugin ②2.1.2

Slide 150

Slide 150 text

Installation of ARCore (for Android) Install

Slide 151

Slide 151 text

Setting procedure for both ARCore/ARKi is from here.

Slide 152

Slide 152 text

Build Setting for ARFoundation MixedRealityToolkit

Slide 153

Slide 153 text

Build Setting for ARFoundation ①Camera ②Clone

Slide 154

Slide 154 text

Build Setting for ARFoundation ②Clone ①Set ProfileName as Mobile MixedRealityToolkit...

Slide 155

Slide 155 text

Build Setting for ARFoundation Camera Setting Providers

Slide 156

Slide 156 text

Build Setting for ARFoundation Windows Mixed Reality Camera Settings

Slide 157

Slide 157 text

Build Setting for ARFoundation See Type

Slide 158

Slide 158 text

Build Setting for ARFoundation Open dropdown menu

Slide 159

Slide 159 text

Build Setting for ARFoundation ①MicrosoftMixedReality.Toolkit.Experimental.UnityAR ②UnityARCameraSetting

Slide 160

Slide 160 text

Remove Performance Viewer Turn Off Enable Diagnostics System

Slide 161

Slide 161 text

Build for ARFoundation File

Slide 162

Slide 162 text

Build for ARFoundation Build Settings...

Slide 163

Slide 163 text

Build for ARKit(iOS) ①iOS ②Switch Platform

Slide 164

Slide 164 text

Build for ARCore(Android) ①Android ②Switch Platform

Slide 165

Slide 165 text

Build for ARFoundation Apply

Slide 166

Slide 166 text

Build for ARFoundation Player Settings...

Slide 167

Slide 167 text

Setting procedure for iOS is from next page.

Slide 168

Slide 168 text

Build for ARKit(iOS) Other Settings

Slide 169

Slide 169 text

Build for ARKit(iOS) Set Bundle Identifier ex) com.Yoshinaga.ARFukuoka

Slide 170

Slide 170 text

Build for ARKit(iOS) Strip Engine CodeをOff

Slide 171

Slide 171 text

Setting procedure for Android is from next page.

Slide 172

Slide 172 text

Build for ARCore(Android) Set Package Name ex)com.yourname.arfukuoka

Slide 173

Slide 173 text

Build and Run for ARCore/ARKit

Slide 174

Slide 174 text

動作の様⼦

Slide 175

Slide 175 text

Ctrl / Command + S 現状を保存

Slide 176

Slide 176 text

Transform with Bounding Box Next Step...

Slide 177

Slide 177 text

Next Step…

Slide 178

Slide 178 text

Applying Bounding Box ①model ②Add Component

Slide 179

Slide 179 text

Applying Bounding Box ①Search [bounding] ②Click BoundingBox

Slide 180

Slide 180 text

動作確認

Slide 181

Slide 181 text

Store Objects in the Dock Next Step...

Slide 182

Slide 182 text

No content

Slide 183

Slide 183 text

Creating Dock Right Click Hierarchy

Slide 184

Slide 184 text

Creating Dock Create Empty

Slide 185

Slide 185 text

Creating Dock ①GameObject ②Rename as Dock1

Slide 186

Slide 186 text

Creating Dock Add Component

Slide 187

Slide 187 text

Creating Dock Search [sphere] SphereCollider

Slide 188

Slide 188 text

Creating Dock SphereCollider is added

Slide 189

Slide 189 text

Creating Dock Set radius of collider to 0.1m

Slide 190

Slide 190 text

Creating Dock Add Component

Slide 191

Slide 191 text

Creating Dock Search [dock] DockPosition

Slide 192

Slide 192 text

Creating Dock DockPosition is added

Slide 193

Slide 193 text

Creating Dock ①Right click Dock1 ②3D Object ③Quad

Slide 194

Slide 194 text

Creating Dock Set translation Position: 0, -0.1, 0 Rotation: 90, 0, 0 Scale: All 0.2