Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Getting Started with Non-Programming AR Development with MRTK v2.4.0

Getting Started with Non-Programming AR Development with MRTK v2.4.0

Slide deck of AR Fukuoka



August 13, 2020

More Decks by TakashiYoshinaga

Other Decks in Technology


  1. Getting Started with Non-Programming AR Development with MRTK

  2. Speaker Name︓吉永崇(Takashi Yoshinaga) Affiliation︓Steampunk Digital ISIT Community: AR Fukuoka

  3. SNS https://www.linkedin.com/in/tks-yoshinaga/ https://twitter.com/Taka_Yoshinaga LinkedIn Twitter

  4. #AR_Fukuoka Hashtag

  5. Download Sample Data http://arfukuoka.lolipop.jp/mrtk_np/Sample.zip

  6. Download MRTK v2.4.0 (1/3) https://github.com/Microsoft/MixedRealityToolkit-Unity Releases

  7. Download MRTK v2.4.0 (2/3) MRTK v2.4.0 is used for this

    tutorial Scroll down
  8. Download MRTK v2.4.0 (3/3) Download only Foundation.2.4.0.unitypackage Foundation.2.4.0.unitypackage

  9. Todayʼs Goal Show and Manipulate Virtual Object without Programming https://youtu.be/Yae3bOtidTk

  10. Creating Project (1/6) Launch Unity Hub and click Project プロジェクト

  11. Creating Project (2/6) Click ▽ aligned on the right side

    of NEW button. 新規作成横の▽
  12. Creating Project (3/6) Select 2018.4.xxf1 (2018.4.23f1 is selected in this

    deck) 2018.4.xxf1
  13. Creating Project (4/6) ①3D ②Set the project name (ex.ARFukuoka_MRTK) ③Select

    the directory to save the project
  14. Creating Project (5/6) Create

  15. Creating Project (6/6) Unity Editor will be shown

  16. Installation of MRTK (1/3) Double click the unitypackage. Click Import

    button of Import Unity Package dialog. Microsoft.MixedReality. Toolkit.Unity.Foundation .2.4.0.unitypackage Import
  17. Installation of MRTK (2/3) Apply Click Apply button to switch

    the setting for XR.
  18. Installation of MRTK (3/3) Close Close MRTK Project Configurator dialog.

  19. Adding 3D Object into the Space (1/3) Right click blank

    of Hierarchy
  20. Adding 3D Object into the Space (2/3) ①3D Object ②Quad

  21. Adding 3D Object into the Space (3/3) Quad is shown

    ※Backside is invisible
  22. Adjust Viewpoint of Scene [←] [→]: Move to Left/RIght [↑][↓]:

    Zoom In/Out [Alt]+Drag: Rotation +Drag: Move to ←→↑↓
  23. Adjust Viewpoint of Scene Object can be shown on the

    center of Scene tab. Double click Quad
  24. Run カメラから⾒た空間 が表⽰される Click to play Click it again to

    stop preview
  25. Modification of Objectʼs Transform Click Quad

  26. Modification of Objectʼs Transform Move Rotate Scale

  27. Inputting Values to Transform Object Click Quad

  28. Inputting Values to Transform Object Inspector p Properties of each

    object can be added/edited in Inspector p Position/Rotation/Scale are editable in Transform.
  29. Inputting Values to Transform Object p Properties of each object

    can be added/edited in Inspector p Position/Rotation/Scale are editable in Transform. Set as following Positionを 0 0 1 Rotation 0 0 0 Scale 0.3 0.3 0.3
  30. Adjust Viewpoint of Scene Letʼs set the object at the

    center of scene tab
  31. Adjust Viewpoint of Scene Double Click Quad

  32. Next Step Not only Transform but also appearance and behavior

    and properties can be edited in Inspector
  33. Changing Color of Cube (1/4) ① Quad ② See Mesh

    Renderer ③ Open Materials
  34. Changing Color of Cube (2/4) Element0

  35. Changing Color of Cube (3/4) Appearance information is described in

    Material file
  36. Changing Color of Cube (4/4) Detail of Material is shown

    at the bottom of Inspector. But Default-Material is not editable.
  37. Itʼs necessary to create Material file to edit appearance of

    an object.
  38. Creating Material (1/3) Create Material to describe color of Quad.

    ①Assets ②Right Click
  39. Creating Material (2/3) ①Create ②Material Create Material to describe color

    of Quad.
  40. Creating Material (3/3) Create Material to describe color of Quad.

    New Material should appear
  41. Application of Material (1/3) ①Quad Open Materials of Quad in

    Inspector ② See Default-Material
  42. Application of Material (2/3) Replace Default-Material with New Material. Drag

    & Drop into Default-Material
  43. Application of Material (3/3) ①Quad ②Material should be New Material

  44. Next Step...

  45. Application of Texture (1/8) ②Open Dropdown Menu ③Unlit ④Transparent ①See

    New Material
  46. Application of Texture (2/8) ①Click ▼ to open NewMateria ②Texture

    can be set here
  47. Application of Texture (3/8) ①Assets ②Import New Asset

  48. Application of Texture (4/8) ①arfukuoka.png ②Import

  49. Application of Texture (5/8) ①Quad ②Open New Materia

  50. Application of Texture (6/8) ①Assets ② see arfukuoka.png (Do NOT

  51. Application of Texture (7/8) Drag & Drop arfukuoka.png into None(Texture)

  52. Application of Texture (8/8) Texture is applied

  53. How to Save this Scene (1/4) File

  54. How to Save this Scene (2/4) SaveAs...

  55. How to Save this Scene (3/4) ①Name as Sample1 ②Save

  56. How to Save this Scene (4/4) Sample1 should be appear

    in Assets folder
  57. You can override the scene with current info by Ctrl

    / Command + S
  58. Letʼs use MRTK from now!

  59. What Youʼll Do with MRTK at First? Prepare moving around

    and using hand in UnityEditor
  60. Using Camera of MRTK (1/3) MixedRealityToolkit

  61. Using Camera of MRTK (2/3) Add to Scene and Configure...

  62. Using Camera of MRTK (3/3) MixedRealityToolKit and MixedRealityPlayspace should appear

  63. Play (1/4) Click Play

  64. Play (2/4) [A][D] :Move to Left and Right [W][S] :Zoom

    In/Out [Q][E] :Move to Up and Down RightClick + Drag :Rotation
  65. Play (3/4) ※Object canʼt be manipulated so far [Shift] :Show

    left hand [Space] :Show right hand While hand is appearing+ [Click] :Pinch [Scroll] :Forward/Back
  66. Play (4/4) Stop before going to next step

  67. Letʼs manipulate Quad

  68. Making Quad Manipulatable (1/3) ①Quad ②Add Component

  69. Making Quad Manipulatable (2/3) Search [manipulation] Select ManipulationHandler

  70. Making Quad Manipulatable (3/3) ①Quad ②ManipulationHandler is added

  71. Play (1/4) Playをクリック

  72. One Hand Manipulation

  73. Play (2/4) Shift or Space

  74. Play (3/4) Put the cursor on Quad

  75. Play (4/4) Drag and move Quad

  76. Two Hand Manipulation

  77. Play (1/5) Show left hand by pushing Shift

  78. Play (2/5) ①Click after putting cursor on Quad ② Hit

    [T] to fix left hand
  79. Play (3/5) Show right hand by pushing Space

  80. Play (4/5) Drag and move

  81. Play (5/5) You can Rotate/Scale Cube

  82. For Smartphone

  83. Play (1/2) Put the cursor shown in the center of

    view on Quad
  84. Play (2/2) Drag to move

  85. Letʼs Add Another Object

  86. Adding Obj File(1/6) Assets

  87. Adding Obj File(2/6) Sample Folder → Model

  88. Adding Obj File (3/6) Drag & Drop Model into Assets

  89. Adding Obj File (4/6) Model

  90. Adding Obj File (5/6) model.obj

  91. Adding Obj File (6/6) Drag & Drop model.obj into Hierarchy

  92. Adjusting Object (1/3) ①model Put chair as following Position 0.5

    0 1
  93. Adjusting Object (2/3) Click ▼ at the left of mode

    to open child hierarchy
  94. Adjusting Object (3/3) model under upper level of model Set

    trans form as bellow Position 0 -0.45 0 Rotation 0 180 0 It is possible to adjust the Transform using only the model in the upper level, but for the sake of efficiency, I divided the operation into two steps
  95. In order to enable chair to detect contact with hand,

    Collider should be added to model.obj
  96. Adding Collider (1/5) ①model (上の階層の⽅) ②Add Component

  97. Adding Collider (2/5) ②BoxCollider ①Search box

  98. Adding Collider (3/5) Bounding box will appear

  99. Adding Collider (4/5) ①model ②Edit Collider

  100. Adding Collider (5/5) Adjust position of each face by moving▪

  101. Make Chair Manipulatable (1/3) ①model ②Add Component

  102. Make Chair Manipulatable (2/3) ②ManipulationHandler ① Searchmanipulation

  103. Make Chair Manipulatable (3/3) ①model ② Manipulation Handler is added

  104. Allow Only Scaling for 2 Hand Operation ①model ②Set Scale

    for Two Hand Manipulation Type
  105. Play

  106. Play Stop

  107. Remove Performance Viewer Can be deleted

  108. Remove Performance Viewer ①MixedRealityToolKit ② Open dropdown menu under Mixed

    Reality Toolkit
  109. Remove Performance Viewer ①MixedRealityToolKit ②DefaultHololens2ConfigureProfile

  110. Remove Performance Viewer Clone

  111. Remove Performance Viewer ②Clone ①Change Profile Name to Sample1 MixedRealityToolkit...

  112. Remove Performance Viewer ①Diagnostics ②Turn Enable Diagnostics System into Off

  113. Play Performance Viewer become invisible

  114. Play Stop

  115. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

  116. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

  117. Build Setting for HoloLens2 File

  118. Build Setting for HoloLens2 Build Setting

  119. Build Setting for HoloLens2 ①Universal Windows Platform ②Switch Platform

  120. Build Setting for HoloLens2 Apply

  121. Build Setting for HoloLens2 Player Settings...

  122. Build Setting for HoloLens2 ①XR Settings ②Select 16-bit depth ad

    Depth Format
  123. Build for HoloLens2 Build

  124. Build for HoloLens2 New Folder

  125. Build for HoloLens2 Rename new folder to build

  126. Build for HoloLens2 ①build ②Select Folder

  127. Wait for finishing build

  128. Build for HoloLens2 Open build

  129. Build for HoloLens2 Open solution (*.sln) with VisualStudio2019

  130. Build for HoloLens2 Release ARM

  131. Build for HoloLens2 ▼

  132. Build for HoloLens2 Device

  133. Connect HoloLens2 and PC by using USB cable

  134. Build for HoloLens2 ①デバッグ ②Start without debug

  135. 動作の様⼦

  136. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

  137. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices

  138. Installation of ARFoundation Window

  139. Installation of ARFoundation Package Manager

  140. Installation of ARFoundation Advanced

  141. Installation of ARFoundation Turn ON Show preview packages

  142. Installation of ARFoundation ①AR Foundation ②Open version list

  143. Installation of ARFoundation ①All Version ②1.5.0-preview.6

  144. Installation of ARFoundation Install

  145. Setting procedure for iOS is from next page.

  146. Installation of ARKit (for iOS) ①ARKit XR Plugin ②2.1.2

  147. Installation of ARKit (for iOS) Install

  148. Setting procedure for Android is from next page.

  149. Installation of ARCore (for Android) ①ARCore XR Plugin ②2.1.2

  150. Installation of ARCore (for Android) Install

  151. Setting procedure for both ARCore/ARKi is from here.

  152. Build Setting for ARFoundation MixedRealityToolkit

  153. Build Setting for ARFoundation ①Camera ②Clone

  154. Build Setting for ARFoundation ②Clone ①Set ProfileName as Mobile MixedRealityToolkit...

  155. Build Setting for ARFoundation Camera Setting Providers

  156. Build Setting for ARFoundation Windows Mixed Reality Camera Settings

  157. Build Setting for ARFoundation See Type

  158. Build Setting for ARFoundation Open dropdown menu

  159. Build Setting for ARFoundation ①MicrosoftMixedReality.Toolkit.Experimental.UnityAR ②UnityARCameraSetting

  160. Remove Performance Viewer Turn Off Enable Diagnostics System

  161. Build for ARFoundation File

  162. Build for ARFoundation Build Settings...

  163. Build for ARKit(iOS) ①iOS ②Switch Platform

  164. Build for ARCore(Android) ①Android ②Switch Platform

  165. Build for ARFoundation Apply

  166. Build for ARFoundation Player Settings...

  167. Setting procedure for iOS is from next page.

  168. Build for ARKit(iOS) Other Settings

  169. Build for ARKit(iOS) Set Bundle Identifier ex) com.Yoshinaga.ARFukuoka

  170. Build for ARKit(iOS) Strip Engine CodeをOff

  171. Setting procedure for Android is from next page.

  172. Build for ARCore(Android) Set Package Name ex)com.yourname.arfukuoka

  173. Build and Run for ARCore/ARKit

  174. 動作の様⼦

  175. Ctrl / Command + S 現状を保存

  176. Transform with Bounding Box Next Step...

  177. Next Step…

  178. Applying Bounding Box ①model ②Add Component

  179. Applying Bounding Box ①Search [bounding] ②Click BoundingBox

  180. 動作確認

  181. Store Objects in the Dock Next Step...

  182. None
  183. Creating Dock Right Click Hierarchy

  184. Creating Dock Create Empty

  185. Creating Dock ①GameObject ②Rename as Dock1

  186. Creating Dock Add Component

  187. Creating Dock Search [sphere] SphereCollider

  188. Creating Dock SphereCollider is added

  189. Creating Dock Set radius of collider to 0.1m

  190. Creating Dock Add Component

  191. Creating Dock Search [dock] DockPosition

  192. Creating Dock DockPosition is added

  193. Creating Dock ①Right click Dock1 ②3D Object ③Quad

  194. Creating Dock Set translation Position: 0, -0.1, 0 Rotation: 90,

    0, 0 Scale: All 0.2
  195. Creating Dock ①MeshRenderer ②Open Materials

  196. Creating Dock Click at the right of Element0

  197. Creating Dock HolographicBackPlateCircular

  198. Creating Dock Like this

  199. Duplication of Dock Dock1

  200. Duplication of Dock Duplicate

  201. Duplication of Dock Dock1(1) Rename as Dock2

  202. Making Object Dockable ①Quad ②AddComponent

  203. Making Object Dockable dock Dockable

  204. Making Object Dockable Dockable

  205. Making Object Dockable ①model ②AddComponent

  206. Making Object Dockable dock Dockable

  207. Making Object Dockable Dockable

  208. Making Object Dockable ①Dock1 ①Dock Position

  209. Making Object Dockable ①See Quad ②Drag & Drop into Docked

  210. Making Object Dockable ①Dock2 ①Dock Position

  211. Making Object Dockable ①See model ②Drag & Drop into Docked

  212. Adjust Position of Docks ①Dock1 Position:-0.2,-0.2,0.5

  213. Adjust Position of Docks ①Dock2 Position: 0.2,-0.2,0.5

  214. None
  215. Fin.

  216. None