Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Getting Started With MRTK (for Beginner)

Getting Started With MRTK (for Beginner)

357c9e58ecce2865f9eb748192e5143f?s=128

TakashiYoshinaga

June 20, 2020
Tweet

More Decks by TakashiYoshinaga

Other Decks in Technology

Transcript

  1. Getting Started With MRTK for Beginner

  2. #AR_Fukuoka Hashtag

  3. Goal Learn about object manipulation as the basic usage of

    MRTK https://youtu.be/qUQXZACp35A
  4. Sample Data http://arfukuoka.lolipop.jp/mrtk/Sample.zip

  5. Downloading MRTK (1/3) https://github.com/Microsoft/MixedRealityToolkit-Unity releases

  6. Downloading MRTK (2/3) Microsoft Mixed Reality Toolkit v2.4.0 is used

    for this tutorial 少し下にスクロール
  7. Downloading MRTK (3/3) Download Foundation.2.4.0.unitypackage Foundation.2.4.0.unitypackage

  8. Creating Project (1/6) Launch UnityHub and click Project Project

  9. Creating Project (2/6) Click ▽ aligned on the right side

    of NEW button. ▽
  10. Creating Project (3/6) Select 2018.4.xxf1 (2018.4.23f1 is selected in this

    deck) 2018.4.xxf1
  11. Creating Project (4/6) ①3D ②Set the project name (ex.ARFukuoka_MRTK) ③Select

    the directory to save the project
  12. Creating Project (5/6) Create

  13. Creating Project (6/6) Unity Editor will appear

  14. Installation of MRTK (1/3) Double click the unitypackage. Click Import

    button of Import Unity Package dialog. Microsoft.MixedReality. Toolkit.Unity.Foundation .2.4.0.unitypackage Import
  15. Installation of MRTK (2/3) Apply Click Apply button to switch

    the setting to xR.
  16. Installation of MRTK (3/3) Close Close MRTK Project Configurator dialog.

  17. Adding 3D Object into the Space (1/3) Right click blank

    of Hierarchy
  18. Adding 3D Object into the Space (2/3) ①3D Object ②Cube

  19. Adding 3D Object into the Space (3/3) Cube should appear

  20. Run View from camera will be shown. Click to play

    Click it again to stop preview
  21. Modification of Objectʼs Transform Click Cube

  22. Modification of Objectʼs Transform Move Rotate Scale

  23. Inputting Values to Transform Object Click Cube

  24. Inputting Values to Transform Object p Properties of each object

    can be added/edited in Inspector p Position/Rotation/Scale are editable in Transform. Inspector
  25. Inputting Values to Transform Object Set as following Positionを 0

    0 1 Rotation 0 0 0 Scale 0.3 0.3 0.3 p Properties of each object can be added/edited in Inspector p Position/Rotation/Scale are editable in Transform.
  26. Controlling Viewpoint of Scene Letʼs set the object at the

    center of scene tab
  27. Controlling Viewpoint of Scene Double Click Cube

  28. Controlling Viewpoint of Scene Object can be shown on the

    center of Scene tab.
  29. Controlling Viewpoint of Scene [←] [→]: Move to Left/RIght [↑][↓]:

    Zoom In/Out [Alt]+Drag: Rotation +Drag: Move to ←→↑↓
  30. Next Step Not only Transform but also appearance and behavior

    and properties can be edited in Inspector
  31. Changing Color of Cube (1/4) ① Cube ② See Mesh

    Renderer ③ Open Materials
  32. Changing Color of Cube (2/4) Element0

  33. Changing Color of Cube (3/4) Appearance information is described in

    Material file
  34. Changing Color of Cube (4/4) Detail of Material is shown

    at the bottom of Inspector. But Default-Material is not editable.
  35. Itʼs necessary to create Material file to edit appearance of

    a object.
  36. Creating Material (1/3) Create Material to describe color of Cube.

    ①Assets ②Right Click
  37. Creating Material (2/3) Create Material to describe color of Cube.

    ①Create ②Material
  38. Creating Material (3/3) New Material should appear Create Material to

    describe color of Cube.
  39. Application of Material (1/2) Open Materials of Cube in Inspector

    ② See Default-Material ①Cube
  40. Application of Material (2/2) Replace Default-Material with New Material. Drag

    & Drop into Default-Material
  41. Editing Color of Cube (1/3) ①Cube ②Material should be New

    Material
  42. Editing Color of Cube (2/3) Open NewMaterial Click □ next

    to Albedo
  43. Editing Color of Cube (3/3) Color should be changed

  44. How to Save this Scene (1/4) File

  45. How to Save this Scene (2/4) SaveAs...

  46. How to Save this Scene (3/4) ①Name as Sample1 ②Save

  47. How to Save this Scene (4/4) Sample1 should be appear

    in Assets
  48. You can override the scene with current info by Ctrl

    / Command + S
  49. Letʼs use MRTK from now!

  50. What Youʼll Do at First? Prepare moving around and using

    hand in UnityEditor
  51. Using Camera of MRTK (1/3) MixedRealityToolkit

  52. Using Camera of MRTK (2/3) Add to Scene and Configure...

  53. Using Camera of MRTK (3/3) MixedRealityToolKit and MixedRealityPlayspace should appear

  54. Play (1/4) Click Play

  55. Play (2/4) [A][D] :Move to Left and Right [W][S] :Zoom

    In/Out [Q][E] :Move to Up and Down RightClick + Drag :Rotation
  56. Play (3/4) [Shift] :Show left hand [Space] :Show right hand

    While hand is appearing+ [Click] :Pinch [Scroll] :Forward/Back
  57. Play (4/4) Stop before going to next step

  58. Letʼs manipulate Cube

  59. Making Cube Manipulatable (1/3) ①Cube ②Add Component

  60. Making Cube Manipulatable (2/3) Search [manipulation] Select ManipulationHandler

  61. Making Cube Manipulatable (3/3) ①Cube ②ManipulationHandler should be added

  62. Enable Near Interaction ①Cube ②Add Component

  63. Enable Near Interaction ①Search [Near] ②NearInteractionGrabbable

  64. Enable Near Interaction ①Cube ②NearInteractionGrabbable should be added

  65. Play (1/4) Click Play

  66. One Hand Manipulation

  67. Play (2/4) Shift or Space

  68. Play (3/4) Put the cursor on Cube

  69. Play (4/4) Drag and move Cube

  70. Two Hand Manipulation

  71. Play (1/5) Show left hand by pushing Shift

  72. Play (2/5) ①Click after putting cursor on Cube ② Hit

    [T] to fix left hand
  73. Play (3/5) Show right hand by pushing Space

  74. Play (4/5) Drag and move

  75. Play (5/5) You can Rotate/Scale Cube

  76. Letʼs Add Another Object

  77. Adding Obj File(1/7) Assets

  78. Adding Obj File (2/7) Sample Folder → Model

  79. Adding Obj File (3/7) Drag & Drop Model into Assets

  80. Adding Obj File (4/7) Model

  81. Adding Obj File (5/7) model.obj

  82. Adding Obj File (6/7) Drag & Drop model.obj into Hierarchy

  83. Adding Obj File (7/7) ①model Set chair as following nbv

    Position 0.5 -0.3 1 Rotation 0 180 0
  84. In order to enable chair to detect contact with hand,

    Collider should be added to model.obj
  85. Adding Collider (1/5) ①model ②Add Component

  86. Adding Collider (2/5) ①Search box ②BoxCollider

  87. Adding Collider (3/5) Bounding box will appear

  88. Adding Collider (4/5) ①model ②Edit Collider

  89. Adding Collider (5/5) Adjust position of each face by moving▪

  90. Make Chair Manipulatable (1/5) ①model ②Add Component

  91. Make Chair Manipulatable (2/5) ① Searchmanipulation ②ManipulationHandler

  92. Make Chair Manipulatable (3/5) ①model ②Add Component

  93. Make Chair Manipulatable (4/5) ①Search near ②NearInteractionGrabbable

  94. Make Chair Manipulatable (5/5) ①model ②NearInteractionGrabbable should be appear

  95. Restrict the Direction of the Chair ①model ②Set One Hand

    Rotation Mode Near/Far to Gravity Aligned Maintain Rotation to User
  96. Allow Only Rotation for 2 Hand Operation ①model ②Set Rotate

    for Two Hand Manipulation Type ③ Set Y Axis Only for Constraint On Rotation
  97. Play Put cursor on chair Drag to move Rotate with

    to hands
  98. Play Stop

  99. Remove Performance Viewer Can be deleted

  100. Remove Performance Viewer ①MixedRealityToolKit ②Copy & Customize

  101. Remove Performance Viewer ①Change Profile Name to Sample1 MixedRealityToolkit... ②Clone

  102. Remove Performance Viewer ①Diagnostics ②Turn Enable Diagnostics System into Off

  103. Play

  104. Play Stop

  105. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

  106. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

  107. Build Setting for HoloLens2 File

  108. Build Setting for HoloLens2 Build Setting

  109. Build Setting for HoloLens2 ①Universal Windows Platform ②Switch Platform

  110. Build Setting for HoloLens2 Apply

  111. Build Setting for HoloLens2 Player Settings...

  112. Build Setting for HoloLens2 ①XR Settings ②Select 16-bit depth ad

    Depth Format
  113. Build for HoloLens2 Build

  114. Build for HoloLens2 New Folder

  115. Build for HoloLens2 Rename new folder to build

  116. Build for HoloLens2 ①build ②Select Folder

  117. Wait for a moment...

  118. Build for HoloLens2 Open build

  119. Build for HoloLens2 Open solution (*.sln) with VisualStudio2019

  120. Build for HoloLens2 Release ARM

  121. Build for HoloLens2 ▼

  122. Build for HoloLens2 Device

  123. Connect HoloLens2 and PC by using USB cable

  124. Build for HoloLens2 ① Debug ②Start without debug

  125. None
  126. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

  127. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

  128. Instalation of ARFoundation Window

  129. Instalation of ARFoundation Package Manager

  130. Instalation of ARFoundation ①AR Foundation ②Open version list

  131. Instalation of ARFoundation ①All Version ②1.5.0-preview.6

  132. Instalation of ARFoundation Install

  133. Setting procedure for iOS is from next page.

  134. Installation of ARKit (for iOS) ①ARKit XR Plugin ②2.1.2

  135. Installation of ARKit (for iOS) Install

  136. Setting procedure for Android is from next page.

  137. Installation of ARCore (for Android) ①ARCore XR Plugin ②2.1.2

  138. Installation of ARCore (for Android) Install

  139. Setting procedure for both ARCore/ARKi is from here.

  140. Build Setting for ARFoundation ①MixedRealityToolkit ②DefaultHoloLens2Configuration

  141. Build Setting for ARFoundation Copy & Customize

  142. Build Setting for ARFoundation ①Set ProfileName as Mobile MixedRealityToolkit... ②Clone

  143. Build Setting for ARFoundation ①Camera ②Clone

  144. Build Setting for ARFoundation ①Set ProfileName as Mobile MixedRealityCameraProfile ②Clone

  145. Build Setting for ARFoundation Camera Setting Providers

  146. Build Setting for ARFoundation -

  147. Build Setting for ARFoundation AddCameraSettingsProvider

  148. Build Setting for ARFoundation Newdataprovider 0

  149. Build Setting for ARFoundation Open Taypeを開く

  150. Build Setting for ARFoundation ①MicrosoftMixedReality.Toolkit.Experimental.UnityAR ②UnityARCameraSetting

  151. Remofe Performance Viewer Diagnostics

  152. Remofe Performance Viewer Turn Off Enable Diagnostics System

  153. Build for ARFoundation File

  154. Build for ARFoundation Build Settings...

  155. Build for ARKit(iOS) ①iOS ②Switch Platform

  156. Build for ARCore(Android) ①Android ②Switch Platform

  157. Build for ARFoundation Apply

  158. Build for ARFoundation Player Settings...

  159. Setting procedure for iOS is from next page.

  160. Build for ARKit(iOS) Other Settings

  161. Build for ARKit(iOS) Set Bundle Identifier ex) com.Yoshinaga.ARFukuoka

  162. Build for ARKit(iOS) Strip Engine CodeをOff

  163. Setting procedure for Android is from next page.

  164. Build for ARCore(Android) Set Package Name ex)com.yourname.arfukuoka

  165. Build and Run for ARCore/ARKit

  166. 動作の様⼦

  167. Ctrl / Command + S Save

  168. Letʼs Use Spatial Awareness Next Step is...

  169. Use room mesh model

  170. Duplicate Current Scene File

  171. Duplicate Current Scene Save As...

  172. Duplicate Current Scene Save As...

  173. Import of Spatial Mesh Model Assets

  174. Import of Spatial Mesh Model Import New Asset

  175. Import of Spatial Mesh Model ①room.obj ②import

  176. Import of Spatial Mesh Model room should appear

  177. Import of Spatial Mesh Model MixedRealityToolkit Clone

  178. Import of Spatial Mesh Model ①Set ProfileName as Sample2 MixedRealityToolkit...

    ②Clone
  179. Visualization of Spatial Mesh Model ①MixedRealityToolkit ②Spatial Awareness

  180. Visualization of Spatial Mesh Model Clone

  181. Visualization of Spatial Mesh Model ①Set ProfileName as Sample2MixedRealityCameraProfile ②Clone

  182. Visualization of Spatial Mesh Model Windows Mixed Reality Spatial Mesh

    Observer
  183. Visualization of Spatial Mesh Model Typeを開く

  184. Visualization of Spatial Mesh Model ①Microsoft.MixedReality.Toolkit.SpatialObjectMeshObserver ②SpatialObjectMeshObserver

  185. Visualization of Spatial Mesh Model Clone

  186. Visualization of Spatial Mesh Model ①Set ProfileName as Sample2SpatialObjectMeshObserver... ②Clone

  187. Visualization of Spatial Mesh Model See Spatial Mesh Object

  188. Visualization of Spatial Mesh Model See room (Do NOT click)

  189. Visualization of Spatial Mesh Model Drag&Drop into ExampleSpatialMesh

  190. Visualization of Spatial Mesh Model Replaced to room

  191. Replace Material of Spatial Mesh Click ◦ aligned on the

    right of MRTK_ Wireframe
  192. Replace Material of Spatial Mesh Defaultで検索 Defaultを選択 ※他のMtaterialでもOK

  193. Replace Material of Spatial Mesh Example Spatial Mesh Default Material

  194. Using Gravity ①model ②Add Component

  195. Using Gravity ①Search rigid ②Rigidbody

  196. Using Gravity ①model ②Rigidbody ③Turn ON Use Gravity

  197. Run

  198. Depending on the development environment, the chair may fall off

    before the mesh appears on the scene. Keep the chair in the air at first and allow it to fall when you let go of your hand.
  199. Turn Off Gravity ①model ② See Rigidbody

  200. Turn Off Gravity ①Use GravityをOff ②Is KinematicをOn Note︓ Turn Is

    Kinematic On to prevent physical interference. Prevents the chair from hitting the mesh and flying away when the app is launched.
  201. Turn On Gravity After Manipulation model The ManipulationHandler has OnManipulation

    Started/Ended, which makes it easy to get the operation started and finished.
  202. Turn On Gravity After Manipulation ①model ②Add Component

  203. Turn On Gravity After Manipulation ①Clear Search Area ③Set Script

    Name ex)PhysicsScript ②New script ④Create and Add
  204. Letʼs Write Script ①model ②Physics Script should appear.

  205. Letʼs Write Script ①model ②Open Physics Script

  206. using System.Collections; using System.Collections.Generic; using UnityEngine; public class PhysicsScript :

    MonoBehaviour { // Start is called before the first frame update void Start() { } // Update is called once per frame void Update() { } } Editing Script
  207. using System.Collections; using System.Collections.Generic; using UnityEngine; public class PhysicsScript :

    MonoBehaviour { Delete Start and Update function } Editing Script
  208. using System.Collections; using System.Collections.Generic; using UnityEngine; public class PhysicsScript :

    MonoBehaviour { //SetGravityにtrue/falseを渡すことで重⼒および物理⼲渉のOn/Offを切り替える public void SetGravity(bool flag) { //重⼒のOn/Offを設定 gameObject.GetComponent<Rigidbody>().useGravity = flag; //物理的な⼲渉のOn/Offを設定 gameObject.GetComponent<Rigidbody>().isKinematic = !flag; } } Function to Switch On/Off Physics
  209. Turn On Gravity when Manipulation is Ended ①model ②On Manipulation

    Ended ③Click +
  210. Turn On Gravity when Manipulation is Ended ①model Drag &

    Drop into area of shown as None
  211. Turn On Gravity when Manipulation is Ended ①model ②Click No

    Function
  212. Turn On Gravity when Manipulation is Ended ②SetGravity(bool) ①PhysicsScript

  213. Turn On Gravity when Manipulation is Ended ①model ③Turn On

    ②See Manipulation Ended
  214. Setting Hand Gesture ①model

  215. Setting Hand Gesture ①SetManipulation Type as One Handed Only ②Set

    Constraint on Rotation as None
  216. The object (chair) will rebound and roll when spatial meshes

    are refreshed on the actual HoloLens. Turn off gravity as soon as collision with spatial mesh
  217. using System.Collections; using System.Collections.Generic; using UnityEngine; public class HandScript :

    MonoBehaviour { //SetGravityにtrue/falseを渡すことで重⼒および物理⼲渉のOn/Offを切り替える public void SetGravity(bool flag) { gameObject.GetComponent<Rigidbody>().useGravity = flag; gameObject.GetComponent<Rigidbody>().isKinematic = !flag; } //Called when collision is occured private void OnCollisionEnter(Collision collision) { SetGravity(false); } }
  218. Delete Unused Object Remove Cube

  219. Delete Unused Object

  220. Play on Unity Editor

  221. Letʼs build and deploy this application into xR device.

  222. p HoloLens2 p ARFoundation(ARKit/ARCore) Run On Each Device

  223. p HoloLens2 p ARFoundation(ARKit/ARCore) Run On Each Device

  224. Setting for HoloLens2 ① model ② Spatial Awareness

  225. Setting for HoloLens2 See Windows Mixed Reality Spatial Mesh Observer

  226. Setting for HoloLens2 Open dropdown menu aligned on right of

    Type
  227. Setting for HoloLens2 ①MicrosoftMixedReality.Toolkit.WindowsMixedReality.SpatialAwareness ②MicrosoftMixedReality.Spatial.MeshObserver

  228. Build -> Deploy -> Run Installation

  229. None
  230. p HoloLens2 p ARFoundation(ARKit/ARCore) Run On Each Device

  231. p HoloLens2 p ARFoundation(ARKit/ARCore) Run On Each Device

  232. Setting for AR Foundation ①MixedRealityToolkit ②See Mixed Reality Toolkit

  233. Setting for AR Foundation MobileMixedRealityToolkitConfiguration (The setting made in the

    first half of tutorial)
  234. Letʼs use plane detection instead of spatial awareness.

  235. Using Plane Detection ①MixedRealityPlayspace ②Add Component

  236. Using Plane Detection ①SearchAR Plane ②AR Plane Manager

  237. Using Plane Detection ①MixedRealityPlayspace AR Session Origin & AR Plane

    Manager should be added
  238. Using Plane Detection Right click on the blank of Hierarchy

  239. Using Plane Detection ①XR ②AR Default Plane

  240. Using Plane Detection ①See AR Default Plane ②Assets

  241. Using Plane Detection Drag&Drop AR Default Plane into Assets

  242. Using Plane Detection Delete AR Default Plane

  243. Using Plane Detection ①MixedRealityPlayspace ②See Plane Prefab

  244. Using Plane Detection ①SeeAR Default Plane (※ Do NOT click)

    ②Drag & Drop into None aligned right of Plane Prefab
  245. Using Plane Detection Plane Prefab should be AR Default Plane

  246. Build -> Deploy -> Run Installation

  247. None
  248. Fin.