Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Getting Started with Non-Programming AR Development with MRTK v2.4.0

Getting Started with Non-Programming AR Development with MRTK v2.4.0

Slide deck of AR Fukuoka

TakashiYoshinaga

August 13, 2020
Tweet

More Decks by TakashiYoshinaga

Other Decks in Technology

Transcript

  1. Getting Started with Non-Programming
    AR Development with MRTK

    View full-size slide

  2. Speaker
    Name︓吉永崇(Takashi Yoshinaga)
    Affiliation︓Steampunk Digital
    ISIT
    Community: AR Fukuoka

    View full-size slide

  3. SNS
    https://www.linkedin.com/in/tks-yoshinaga/
    https://twitter.com/Taka_Yoshinaga
    LinkedIn
    Twitter

    View full-size slide

  4. #AR_Fukuoka
    Hashtag

    View full-size slide

  5. Download Sample Data
    http://arfukuoka.lolipop.jp/mrtk_np/Sample.zip

    View full-size slide

  6. Download MRTK v2.4.0 (1/3)
    https://github.com/Microsoft/MixedRealityToolkit-Unity
    Releases

    View full-size slide

  7. Download MRTK v2.4.0 (2/3)
    MRTK v2.4.0 is used for this tutorial
    Scroll down

    View full-size slide

  8. Download MRTK v2.4.0 (3/3)
    Download only Foundation.2.4.0.unitypackage
    Foundation.2.4.0.unitypackage

    View full-size slide

  9. Todayʼs Goal
    Show and Manipulate Virtual Object without Programming
    https://youtu.be/Yae3bOtidTk

    View full-size slide

  10. Creating Project (1/6)
    Launch Unity Hub and click Project
    プロジェクト

    View full-size slide

  11. Creating Project (2/6)
    Click ▽ aligned on the right side of NEW button.
    新規作成横の▽

    View full-size slide

  12. Creating Project (3/6)
    Select 2018.4.xxf1 (2018.4.23f1 is selected in this deck)
    2018.4.xxf1

    View full-size slide

  13. Creating Project (4/6)
    ①3D ②Set the project name
    (ex.ARFukuoka_MRTK)
    ③Select the directory
    to save the project

    View full-size slide

  14. Creating Project (5/6)
    Create

    View full-size slide

  15. Creating Project (6/6)
    Unity Editor will be shown

    View full-size slide

  16. Installation of MRTK (1/3)
    Double click the unitypackage.
    Click Import button of Import Unity Package dialog.
    Microsoft.MixedReality.
    Toolkit.Unity.Foundation
    .2.4.0.unitypackage
    Import

    View full-size slide

  17. Installation of MRTK (2/3)
    Apply
    Click Apply button to switch the setting for XR.

    View full-size slide

  18. Installation of MRTK (3/3)
    Close
    Close MRTK Project Configurator dialog.

    View full-size slide

  19. Adding 3D Object into the Space (1/3)
    Right click blank
    of Hierarchy

    View full-size slide

  20. Adding 3D Object into the Space (2/3)
    ①3D Object
    ②Quad

    View full-size slide

  21. Adding 3D Object into the Space (3/3)
    Quad is shown
    ※Backside is invisible

    View full-size slide

  22. Adjust Viewpoint of Scene
    [←] [→]: Move to Left/RIght
    [↑][↓]: Zoom In/Out
    [Alt]+Drag: Rotation
    +Drag: Move to ←→↑↓

    View full-size slide

  23. Adjust Viewpoint of Scene
    Object can be shown on the center of Scene tab.
    Double click Quad

    View full-size slide

  24. Run
    カメラから⾒た空間
    が表⽰される
    Click to play
    Click it again to
    stop preview

    View full-size slide

  25. Modification of Objectʼs Transform
    Click Quad

    View full-size slide

  26. Modification of Objectʼs Transform
    Move Rotate Scale

    View full-size slide

  27. Inputting Values to Transform Object
    Click Quad

    View full-size slide

  28. Inputting Values to Transform Object
    Inspector
    p Properties of each object can be added/edited in Inspector
    p Position/Rotation/Scale are editable in Transform.

    View full-size slide

  29. Inputting Values to Transform Object
    p Properties of each object can be added/edited in Inspector
    p Position/Rotation/Scale are editable in Transform.
    Set as following
    Positionを 0 0 1
    Rotation 0 0 0
    Scale 0.3 0.3 0.3

    View full-size slide

  30. Adjust Viewpoint of Scene
    Letʼs set the object at
    the center of scene tab

    View full-size slide

  31. Adjust Viewpoint of Scene
    Double Click Quad

    View full-size slide

  32. Next Step
    Not only Transform but also
    appearance and behavior and
    properties can be edited in Inspector

    View full-size slide

  33. Changing Color of Cube (1/4)
    ① Quad
    ② See Mesh Renderer
    ③ Open Materials

    View full-size slide

  34. Changing Color of Cube (2/4)
    Element0

    View full-size slide

  35. Changing Color of Cube (3/4)
    Appearance information
    is described in Material file

    View full-size slide

  36. Changing Color of Cube (4/4)
    Detail of Material is shown at the
    bottom of Inspector.
    But Default-Material is not editable.

    View full-size slide

  37. Itʼs necessary to create Material file
    to edit appearance of an object.

    View full-size slide

  38. Creating Material (1/3)
    Create Material to describe color of Quad.
    ①Assets
    ②Right Click

    View full-size slide

  39. Creating Material (2/3)
    ①Create
    ②Material
    Create Material to describe color of Quad.

    View full-size slide

  40. Creating Material (3/3)
    Create Material to describe color of Quad.
    New Material should appear

    View full-size slide

  41. Application of Material (1/3)
    ①Quad
    Open Materials of Quad in Inspector
    ② See Default-Material

    View full-size slide

  42. Application of Material (2/3)
    Replace Default-Material with New Material.
    Drag & Drop into
    Default-Material

    View full-size slide

  43. Application of Material (3/3)
    ①Quad
    ②Material should be
    New Material

    View full-size slide

  44. Next Step...

    View full-size slide

  45. Application of Texture (1/8)
    ②Open Dropdown
    Menu
    ③Unlit
    ④Transparent
    ①See New Material

    View full-size slide

  46. Application of Texture (2/8)
    ①Click ▼ to open
    NewMateria
    ②Texture can be set here

    View full-size slide

  47. Application of Texture (3/8)
    ①Assets
    ②Import New Asset

    View full-size slide

  48. Application of Texture (4/8)
    ①arfukuoka.png
    ②Import

    View full-size slide

  49. Application of Texture (5/8)
    ①Quad
    ②Open New Materia

    View full-size slide

  50. Application of Texture (6/8)
    ①Assets
    ② see arfukuoka.png
    (Do NOT click)

    View full-size slide

  51. Application of Texture (7/8)
    Drag & Drop arfukuoka.png
    into None(Texture)

    View full-size slide

  52. Application of Texture (8/8)
    Texture is applied

    View full-size slide

  53. How to Save this Scene (1/4)
    File

    View full-size slide

  54. How to Save this Scene (2/4)
    SaveAs...

    View full-size slide

  55. How to Save this Scene (3/4)
    ①Name as Sample1
    ②Save

    View full-size slide

  56. How to Save this Scene (4/4)
    Sample1 should be appear in Assets folder

    View full-size slide

  57. You can override the scene
    with current info by
    Ctrl / Command + S

    View full-size slide

  58. Letʼs use MRTK from now!

    View full-size slide

  59. What Youʼll Do with MRTK at First?
    Prepare moving around and using hand in UnityEditor

    View full-size slide

  60. Using Camera of MRTK (1/3)
    MixedRealityToolkit

    View full-size slide

  61. Using Camera of MRTK (2/3)
    Add to Scene and Configure...

    View full-size slide

  62. Using Camera of MRTK (3/3)
    MixedRealityToolKit and
    MixedRealityPlayspace
    should appear

    View full-size slide

  63. Play (1/4)
    Click Play

    View full-size slide

  64. Play (2/4)
    [A][D] :Move to Left and Right
    [W][S] :Zoom In/Out
    [Q][E] :Move to Up and Down
    RightClick + Drag :Rotation

    View full-size slide

  65. Play (3/4)
    ※Object canʼt be manipulated so far
    [Shift] :Show left hand
    [Space] :Show right hand
    While hand is appearing+
    [Click] :Pinch
    [Scroll] :Forward/Back

    View full-size slide

  66. Play (4/4)
    Stop before going to next step

    View full-size slide

  67. Letʼs manipulate Quad

    View full-size slide

  68. Making Quad Manipulatable (1/3)
    ①Quad
    ②Add Component

    View full-size slide

  69. Making Quad Manipulatable (2/3)
    Search [manipulation]
    Select ManipulationHandler

    View full-size slide

  70. Making Quad Manipulatable (3/3)
    ①Quad
    ②ManipulationHandler
    is added

    View full-size slide

  71. Play (1/4)
    Playをクリック

    View full-size slide

  72. One Hand Manipulation

    View full-size slide

  73. Play (2/4)
    Shift or Space

    View full-size slide

  74. Play (3/4)
    Put the cursor on Quad

    View full-size slide

  75. Play (4/4)
    Drag and move Quad

    View full-size slide

  76. Two Hand Manipulation

    View full-size slide

  77. Play (1/5)
    Show left hand by
    pushing Shift

    View full-size slide

  78. Play (2/5)
    ①Click after putting cursor on Quad
    ② Hit [T] to fix left hand

    View full-size slide

  79. Play (3/5)
    Show right hand by pushing Space

    View full-size slide

  80. Play (4/5)
    Drag and move

    View full-size slide

  81. Play (5/5)
    You can Rotate/Scale Cube

    View full-size slide

  82. For Smartphone

    View full-size slide

  83. Play (1/2)
    Put the cursor shown in the center of view on Quad

    View full-size slide

  84. Play (2/2)
    Drag to move

    View full-size slide

  85. Letʼs Add Another Object

    View full-size slide

  86. Adding Obj File(1/6)
    Assets

    View full-size slide

  87. Adding Obj File(2/6)
    Sample Folder → Model

    View full-size slide

  88. Adding Obj File (3/6)
    Drag & Drop
    Model into Assets

    View full-size slide

  89. Adding Obj File (4/6)
    Model

    View full-size slide

  90. Adding Obj File (5/6)
    model.obj

    View full-size slide

  91. Adding Obj File (6/6)
    Drag & Drop model.obj
    into Hierarchy

    View full-size slide

  92. Adjusting Object (1/3)
    ①model
    Put chair as following
    Position 0.5 0 1

    View full-size slide

  93. Adjusting Object (2/3)
    Click ▼ at the left of mode
    to open child hierarchy

    View full-size slide

  94. Adjusting Object (3/3)
    model under upper level of model
    Set trans form as bellow
    Position 0 -0.45 0
    Rotation 0 180 0
    It is possible to adjust the Transform using only the model in the upper level,
    but for the sake of efficiency, I divided the operation into two steps

    View full-size slide

  95. In order to enable chair to detect
    contact with hand, Collider should
    be added to model.obj

    View full-size slide

  96. Adding Collider (1/5)
    ①model
    (上の階層の⽅) ②Add Component

    View full-size slide

  97. Adding Collider (2/5)
    ②BoxCollider
    ①Search box

    View full-size slide

  98. Adding Collider (3/5)
    Bounding box will appear

    View full-size slide

  99. Adding Collider (4/5)
    ①model
    ②Edit Collider

    View full-size slide

  100. Adding Collider (5/5)
    Adjust position of each
    face by moving■

    View full-size slide

  101. Make Chair Manipulatable (1/3)
    ①model
    ②Add Component

    View full-size slide

  102. Make Chair Manipulatable (2/3)
    ②ManipulationHandler
    ① Searchmanipulation

    View full-size slide

  103. Make Chair Manipulatable (3/3)
    ①model
    ② Manipulation Handler is added

    View full-size slide

  104. Allow Only Scaling for 2 Hand Operation
    ①model
    ②Set Scale for
    Two Hand Manipulation Type

    View full-size slide

  105. Remove Performance Viewer
    Can be deleted

    View full-size slide

  106. Remove Performance Viewer
    ①MixedRealityToolKit
    ② Open dropdown menu
    under Mixed Reality Toolkit

    View full-size slide

  107. Remove Performance Viewer
    ①MixedRealityToolKit
    ②DefaultHololens2ConfigureProfile

    View full-size slide

  108. Remove Performance Viewer
    Clone

    View full-size slide

  109. Remove Performance Viewer
    ②Clone
    ①Change Profile Name to Sample1 MixedRealityToolkit...

    View full-size slide

  110. Remove Performance Viewer
    ①Diagnostics
    ②Turn Enable Diagnostics
    System into Off

    View full-size slide

  111. Play
    Performance Viewer become invisible

    View full-size slide

  112. p HoloLens2
    p ARFoundation(ARKit/ARCore)
    Running On AR Devices

    View full-size slide

  113. p HoloLens2
    p ARFoundation(ARKit/ARCore)
    Running On AR Devices

    View full-size slide

  114. Build Setting for HoloLens2
    File

    View full-size slide

  115. Build Setting for HoloLens2
    Build Setting

    View full-size slide

  116. Build Setting for HoloLens2
    ①Universal Windows Platform
    ②Switch Platform

    View full-size slide

  117. Build Setting for HoloLens2
    Apply

    View full-size slide

  118. Build Setting for HoloLens2
    Player Settings...

    View full-size slide

  119. Build Setting for HoloLens2
    ①XR Settings
    ②Select 16-bit depth ad Depth Format

    View full-size slide

  120. Build for HoloLens2
    Build

    View full-size slide

  121. Build for HoloLens2
    New Folder

    View full-size slide

  122. Build for HoloLens2
    Rename new folder to build

    View full-size slide

  123. Build for HoloLens2
    ①build
    ②Select Folder

    View full-size slide

  124. Wait for finishing build

    View full-size slide

  125. Build for HoloLens2
    Open build

    View full-size slide

  126. Build for HoloLens2
    Open solution (*.sln) with VisualStudio2019

    View full-size slide

  127. Build for HoloLens2
    Release ARM

    View full-size slide

  128. Build for HoloLens2

    View full-size slide

  129. Build for HoloLens2
    Device

    View full-size slide

  130. Connect HoloLens2 and PC
    by using USB cable

    View full-size slide

  131. Build for HoloLens2
    ①デバッグ
    ②Start without debug

    View full-size slide

  132. 動作の様⼦

    View full-size slide

  133. p HoloLens2
    p ARFoundation(ARKit/ARCore)
    Running On AR Devices

    View full-size slide

  134. p HoloLens2
    p ARFoundation(ARKit/ARCore)
    Running On AR Devices

    View full-size slide

  135. Installation of ARFoundation
    Window

    View full-size slide

  136. Installation of ARFoundation
    Package Manager

    View full-size slide

  137. Installation of ARFoundation
    Advanced

    View full-size slide

  138. Installation of ARFoundation
    Turn ON Show preview packages

    View full-size slide

  139. Installation of ARFoundation
    ①AR Foundation
    ②Open version list

    View full-size slide

  140. Installation of ARFoundation
    ①All Version
    ②1.5.0-preview.6

    View full-size slide

  141. Installation of ARFoundation
    Install

    View full-size slide

  142. Setting procedure for iOS
    is from next page.

    View full-size slide

  143. Installation of ARKit (for iOS)
    ①ARKit XR Plugin
    ②2.1.2

    View full-size slide

  144. Installation of ARKit (for iOS)
    Install

    View full-size slide

  145. Setting procedure for Android
    is from next page.

    View full-size slide

  146. Installation of ARCore (for Android)
    ①ARCore XR Plugin
    ②2.1.2

    View full-size slide

  147. Installation of ARCore (for Android)
    Install

    View full-size slide

  148. Setting procedure for both
    ARCore/ARKi is from here.

    View full-size slide

  149. Build Setting for ARFoundation
    MixedRealityToolkit

    View full-size slide

  150. Build Setting for ARFoundation
    ①Camera
    ②Clone

    View full-size slide

  151. Build Setting for ARFoundation
    ②Clone
    ①Set ProfileName as Mobile MixedRealityToolkit...

    View full-size slide

  152. Build Setting for ARFoundation
    Camera Setting Providers

    View full-size slide

  153. Build Setting for ARFoundation
    Windows Mixed Reality Camera Settings

    View full-size slide

  154. Build Setting for ARFoundation
    See Type

    View full-size slide

  155. Build Setting for ARFoundation
    Open dropdown menu

    View full-size slide

  156. Build Setting for ARFoundation
    ①MicrosoftMixedReality.Toolkit.Experimental.UnityAR
    ②UnityARCameraSetting

    View full-size slide

  157. Remove Performance Viewer
    Turn Off Enable Diagnostics System

    View full-size slide

  158. Build for ARFoundation
    File

    View full-size slide

  159. Build for ARFoundation
    Build Settings...

    View full-size slide

  160. Build for ARKit(iOS)
    ①iOS
    ②Switch Platform

    View full-size slide

  161. Build for ARCore(Android)
    ①Android
    ②Switch Platform

    View full-size slide

  162. Build for ARFoundation
    Apply

    View full-size slide

  163. Build for ARFoundation
    Player Settings...

    View full-size slide

  164. Setting procedure for iOS
    is from next page.

    View full-size slide

  165. Build for ARKit(iOS)
    Other Settings

    View full-size slide

  166. Build for ARKit(iOS)
    Set Bundle Identifier
    ex) com.Yoshinaga.ARFukuoka

    View full-size slide

  167. Build for ARKit(iOS)
    Strip Engine CodeをOff

    View full-size slide

  168. Setting procedure for Android
    is from next page.

    View full-size slide

  169. Build for ARCore(Android)
    Set Package Name
    ex)com.yourname.arfukuoka

    View full-size slide

  170. Build and Run
    for ARCore/ARKit

    View full-size slide

  171. 動作の様⼦

    View full-size slide

  172. Ctrl / Command + S
    現状を保存

    View full-size slide

  173. Transform with
    Bounding Box
    Next Step...

    View full-size slide

  174. Next Step…

    View full-size slide

  175. Applying Bounding Box
    ①model
    ②Add Component

    View full-size slide

  176. Applying Bounding Box
    ①Search [bounding]
    ②Click BoundingBox

    View full-size slide

  177. 動作確認

    View full-size slide

  178. Store Objects in the Dock
    Next Step...

    View full-size slide

  179. Creating Dock
    Right Click Hierarchy

    View full-size slide

  180. Creating Dock
    Create Empty

    View full-size slide

  181. Creating Dock
    ①GameObject
    ②Rename as Dock1

    View full-size slide

  182. Creating Dock
    Add Component

    View full-size slide

  183. Creating Dock
    Search [sphere]
    SphereCollider

    View full-size slide

  184. Creating Dock
    SphereCollider is added

    View full-size slide

  185. Creating Dock
    Set radius of collider to 0.1m

    View full-size slide

  186. Creating Dock
    Add Component

    View full-size slide

  187. Creating Dock
    Search [dock]
    DockPosition

    View full-size slide

  188. Creating Dock
    DockPosition is added

    View full-size slide

  189. Creating Dock
    ①Right click Dock1
    ②3D Object
    ③Quad

    View full-size slide

  190. Creating Dock
    Set translation
    Position: 0, -0.1, 0
    Rotation: 90, 0, 0
    Scale: All 0.2

    View full-size slide

  191. Creating Dock
    ①MeshRenderer
    ②Open Materials

    View full-size slide

  192. Creating Dock
    Click at the right of Element0

    View full-size slide

  193. Creating Dock
    HolographicBackPlateCircular

    View full-size slide

  194. Creating Dock
    Like this

    View full-size slide

  195. Duplication of Dock
    Dock1

    View full-size slide

  196. Duplication of Dock
    Duplicate

    View full-size slide

  197. Duplication of Dock
    Dock1(1)
    Rename as Dock2

    View full-size slide

  198. Making Object Dockable
    ①Quad
    ②AddComponent

    View full-size slide

  199. Making Object Dockable
    dock
    Dockable

    View full-size slide

  200. Making Object Dockable
    Dockable

    View full-size slide

  201. Making Object Dockable
    ①model
    ②AddComponent

    View full-size slide

  202. Making Object Dockable
    dock
    Dockable

    View full-size slide

  203. Making Object Dockable
    Dockable

    View full-size slide

  204. Making Object Dockable
    ①Dock1
    ①Dock Position

    View full-size slide

  205. Making Object Dockable
    ①See Quad
    ②Drag & Drop into Docked Object

    View full-size slide

  206. Making Object Dockable
    ①Dock2
    ①Dock Position

    View full-size slide

  207. Making Object Dockable
    ①See model
    ②Drag & Drop into Docked Object

    View full-size slide

  208. Adjust Position of Docks
    ①Dock1
    Position:-0.2,-0.2,0.5

    View full-size slide

  209. Adjust Position of Docks
    ①Dock2
    Position: 0.2,-0.2,0.5

    View full-size slide