Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Getting Started with Non-Programming AR Development with MRTK v2.4.0

Getting Started with Non-Programming AR Development with MRTK v2.4.0

Slide deck of AR Fukuoka

TakashiYoshinaga

August 13, 2020
Tweet

More Decks by TakashiYoshinaga

Other Decks in Technology

Transcript

  1. Getting Started with Non-Programming
    AR Development with MRTK

    View Slide

  2. Speaker
    Name︓吉永崇(Takashi Yoshinaga)
    Affiliation︓Steampunk Digital
    ISIT
    Community: AR Fukuoka

    View Slide

  3. SNS
    https://www.linkedin.com/in/tks-yoshinaga/
    https://twitter.com/Taka_Yoshinaga
    LinkedIn
    Twitter

    View Slide

  4. #AR_Fukuoka
    Hashtag

    View Slide

  5. Download Sample Data
    http://arfukuoka.lolipop.jp/mrtk_np/Sample.zip

    View Slide

  6. Download MRTK v2.4.0 (1/3)
    https://github.com/Microsoft/MixedRealityToolkit-Unity
    Releases

    View Slide

  7. Download MRTK v2.4.0 (2/3)
    MRTK v2.4.0 is used for this tutorial
    Scroll down

    View Slide

  8. Download MRTK v2.4.0 (3/3)
    Download only Foundation.2.4.0.unitypackage
    Foundation.2.4.0.unitypackage

    View Slide

  9. Todayʼs Goal
    Show and Manipulate Virtual Object without Programming
    https://youtu.be/Yae3bOtidTk

    View Slide

  10. Creating Project (1/6)
    Launch Unity Hub and click Project
    プロジェクト

    View Slide

  11. Creating Project (2/6)
    Click ▽ aligned on the right side of NEW button.
    新規作成横の▽

    View Slide

  12. Creating Project (3/6)
    Select 2018.4.xxf1 (2018.4.23f1 is selected in this deck)
    2018.4.xxf1

    View Slide

  13. Creating Project (4/6)
    ①3D ②Set the project name
    (ex.ARFukuoka_MRTK)
    ③Select the directory
    to save the project

    View Slide

  14. Creating Project (5/6)
    Create

    View Slide

  15. Creating Project (6/6)
    Unity Editor will be shown

    View Slide

  16. Installation of MRTK (1/3)
    Double click the unitypackage.
    Click Import button of Import Unity Package dialog.
    Microsoft.MixedReality.
    Toolkit.Unity.Foundation
    .2.4.0.unitypackage
    Import

    View Slide

  17. Installation of MRTK (2/3)
    Apply
    Click Apply button to switch the setting for XR.

    View Slide

  18. Installation of MRTK (3/3)
    Close
    Close MRTK Project Configurator dialog.

    View Slide

  19. Adding 3D Object into the Space (1/3)
    Right click blank
    of Hierarchy

    View Slide

  20. Adding 3D Object into the Space (2/3)
    ①3D Object
    ②Quad

    View Slide

  21. Adding 3D Object into the Space (3/3)
    Quad is shown
    ※Backside is invisible

    View Slide

  22. Adjust Viewpoint of Scene
    [←] [→]: Move to Left/RIght
    [↑][↓]: Zoom In/Out
    [Alt]+Drag: Rotation
    +Drag: Move to ←→↑↓

    View Slide

  23. Adjust Viewpoint of Scene
    Object can be shown on the center of Scene tab.
    Double click Quad

    View Slide

  24. Run
    カメラから⾒た空間
    が表⽰される
    Click to play
    Click it again to
    stop preview

    View Slide

  25. Modification of Objectʼs Transform
    Click Quad

    View Slide

  26. Modification of Objectʼs Transform
    Move Rotate Scale

    View Slide

  27. Inputting Values to Transform Object
    Click Quad

    View Slide

  28. Inputting Values to Transform Object
    Inspector
    p Properties of each object can be added/edited in Inspector
    p Position/Rotation/Scale are editable in Transform.

    View Slide

  29. Inputting Values to Transform Object
    p Properties of each object can be added/edited in Inspector
    p Position/Rotation/Scale are editable in Transform.
    Set as following
    Positionを 0 0 1
    Rotation 0 0 0
    Scale 0.3 0.3 0.3

    View Slide

  30. Adjust Viewpoint of Scene
    Letʼs set the object at
    the center of scene tab

    View Slide

  31. Adjust Viewpoint of Scene
    Double Click Quad

    View Slide

  32. Next Step
    Not only Transform but also
    appearance and behavior and
    properties can be edited in Inspector

    View Slide

  33. Changing Color of Cube (1/4)
    ① Quad
    ② See Mesh Renderer
    ③ Open Materials

    View Slide

  34. Changing Color of Cube (2/4)
    Element0

    View Slide

  35. Changing Color of Cube (3/4)
    Appearance information
    is described in Material file

    View Slide

  36. Changing Color of Cube (4/4)
    Detail of Material is shown at the
    bottom of Inspector.
    But Default-Material is not editable.

    View Slide

  37. Itʼs necessary to create Material file
    to edit appearance of an object.

    View Slide

  38. Creating Material (1/3)
    Create Material to describe color of Quad.
    ①Assets
    ②Right Click

    View Slide

  39. Creating Material (2/3)
    ①Create
    ②Material
    Create Material to describe color of Quad.

    View Slide

  40. Creating Material (3/3)
    Create Material to describe color of Quad.
    New Material should appear

    View Slide

  41. Application of Material (1/3)
    ①Quad
    Open Materials of Quad in Inspector
    ② See Default-Material

    View Slide

  42. Application of Material (2/3)
    Replace Default-Material with New Material.
    Drag & Drop into
    Default-Material

    View Slide

  43. Application of Material (3/3)
    ①Quad
    ②Material should be
    New Material

    View Slide

  44. Next Step...

    View Slide

  45. Application of Texture (1/8)
    ②Open Dropdown
    Menu
    ③Unlit
    ④Transparent
    ①See New Material

    View Slide

  46. Application of Texture (2/8)
    ①Click ▼ to open
    NewMateria
    ②Texture can be set here

    View Slide

  47. Application of Texture (3/8)
    ①Assets
    ②Import New Asset

    View Slide

  48. Application of Texture (4/8)
    ①arfukuoka.png
    ②Import

    View Slide

  49. Application of Texture (5/8)
    ①Quad
    ②Open New Materia

    View Slide

  50. Application of Texture (6/8)
    ①Assets
    ② see arfukuoka.png
    (Do NOT click)

    View Slide

  51. Application of Texture (7/8)
    Drag & Drop arfukuoka.png
    into None(Texture)

    View Slide

  52. Application of Texture (8/8)
    Texture is applied

    View Slide

  53. How to Save this Scene (1/4)
    File

    View Slide

  54. How to Save this Scene (2/4)
    SaveAs...

    View Slide

  55. How to Save this Scene (3/4)
    ①Name as Sample1
    ②Save

    View Slide

  56. How to Save this Scene (4/4)
    Sample1 should be appear in Assets folder

    View Slide

  57. You can override the scene
    with current info by
    Ctrl / Command + S

    View Slide

  58. Letʼs use MRTK from now!

    View Slide

  59. What Youʼll Do with MRTK at First?
    Prepare moving around and using hand in UnityEditor

    View Slide

  60. Using Camera of MRTK (1/3)
    MixedRealityToolkit

    View Slide

  61. Using Camera of MRTK (2/3)
    Add to Scene and Configure...

    View Slide

  62. Using Camera of MRTK (3/3)
    MixedRealityToolKit and
    MixedRealityPlayspace
    should appear

    View Slide

  63. Play (1/4)
    Click Play

    View Slide

  64. Play (2/4)
    [A][D] :Move to Left and Right
    [W][S] :Zoom In/Out
    [Q][E] :Move to Up and Down
    RightClick + Drag :Rotation

    View Slide

  65. Play (3/4)
    ※Object canʼt be manipulated so far
    [Shift] :Show left hand
    [Space] :Show right hand
    While hand is appearing+
    [Click] :Pinch
    [Scroll] :Forward/Back

    View Slide

  66. Play (4/4)
    Stop before going to next step

    View Slide

  67. Letʼs manipulate Quad

    View Slide

  68. Making Quad Manipulatable (1/3)
    ①Quad
    ②Add Component

    View Slide

  69. Making Quad Manipulatable (2/3)
    Search [manipulation]
    Select ManipulationHandler

    View Slide

  70. Making Quad Manipulatable (3/3)
    ①Quad
    ②ManipulationHandler
    is added

    View Slide

  71. Play (1/4)
    Playをクリック

    View Slide

  72. One Hand Manipulation

    View Slide

  73. Play (2/4)
    Shift or Space

    View Slide

  74. Play (3/4)
    Put the cursor on Quad

    View Slide

  75. Play (4/4)
    Drag and move Quad

    View Slide

  76. Two Hand Manipulation

    View Slide

  77. Play (1/5)
    Show left hand by
    pushing Shift

    View Slide

  78. Play (2/5)
    ①Click after putting cursor on Quad
    ② Hit [T] to fix left hand

    View Slide

  79. Play (3/5)
    Show right hand by pushing Space

    View Slide

  80. Play (4/5)
    Drag and move

    View Slide

  81. Play (5/5)
    You can Rotate/Scale Cube

    View Slide

  82. For Smartphone

    View Slide

  83. Play (1/2)
    Put the cursor shown in the center of view on Quad

    View Slide

  84. Play (2/2)
    Drag to move

    View Slide

  85. Letʼs Add Another Object

    View Slide

  86. Adding Obj File(1/6)
    Assets

    View Slide

  87. Adding Obj File(2/6)
    Sample Folder → Model

    View Slide

  88. Adding Obj File (3/6)
    Drag & Drop
    Model into Assets

    View Slide

  89. Adding Obj File (4/6)
    Model

    View Slide

  90. Adding Obj File (5/6)
    model.obj

    View Slide

  91. Adding Obj File (6/6)
    Drag & Drop model.obj
    into Hierarchy

    View Slide

  92. Adjusting Object (1/3)
    ①model
    Put chair as following
    Position 0.5 0 1

    View Slide

  93. Adjusting Object (2/3)
    Click ▼ at the left of mode
    to open child hierarchy

    View Slide

  94. Adjusting Object (3/3)
    model under upper level of model
    Set trans form as bellow
    Position 0 -0.45 0
    Rotation 0 180 0
    It is possible to adjust the Transform using only the model in the upper level,
    but for the sake of efficiency, I divided the operation into two steps

    View Slide

  95. In order to enable chair to detect
    contact with hand, Collider should
    be added to model.obj

    View Slide

  96. Adding Collider (1/5)
    ①model
    (上の階層の⽅) ②Add Component

    View Slide

  97. Adding Collider (2/5)
    ②BoxCollider
    ①Search box

    View Slide

  98. Adding Collider (3/5)
    Bounding box will appear

    View Slide

  99. Adding Collider (4/5)
    ①model
    ②Edit Collider

    View Slide

  100. Adding Collider (5/5)
    Adjust position of each
    face by moving■

    View Slide

  101. Make Chair Manipulatable (1/3)
    ①model
    ②Add Component

    View Slide

  102. Make Chair Manipulatable (2/3)
    ②ManipulationHandler
    ① Searchmanipulation

    View Slide

  103. Make Chair Manipulatable (3/3)
    ①model
    ② Manipulation Handler is added

    View Slide

  104. Allow Only Scaling for 2 Hand Operation
    ①model
    ②Set Scale for
    Two Hand Manipulation Type

    View Slide

  105. Play

    View Slide

  106. Play
    Stop

    View Slide

  107. Remove Performance Viewer
    Can be deleted

    View Slide

  108. Remove Performance Viewer
    ①MixedRealityToolKit
    ② Open dropdown menu
    under Mixed Reality Toolkit

    View Slide

  109. Remove Performance Viewer
    ①MixedRealityToolKit
    ②DefaultHololens2ConfigureProfile

    View Slide

  110. Remove Performance Viewer
    Clone

    View Slide

  111. Remove Performance Viewer
    ②Clone
    ①Change Profile Name to Sample1 MixedRealityToolkit...

    View Slide

  112. Remove Performance Viewer
    ①Diagnostics
    ②Turn Enable Diagnostics
    System into Off

    View Slide

  113. Play
    Performance Viewer become invisible

    View Slide

  114. Play
    Stop

    View Slide

  115. p HoloLens2
    p ARFoundation(ARKit/ARCore)
    Running On AR Devices

    View Slide

  116. p HoloLens2
    p ARFoundation(ARKit/ARCore)
    Running On AR Devices

    View Slide

  117. Build Setting for HoloLens2
    File

    View Slide

  118. Build Setting for HoloLens2
    Build Setting

    View Slide

  119. Build Setting for HoloLens2
    ①Universal Windows Platform
    ②Switch Platform

    View Slide

  120. Build Setting for HoloLens2
    Apply

    View Slide

  121. Build Setting for HoloLens2
    Player Settings...

    View Slide

  122. Build Setting for HoloLens2
    ①XR Settings
    ②Select 16-bit depth ad Depth Format

    View Slide

  123. Build for HoloLens2
    Build

    View Slide

  124. Build for HoloLens2
    New Folder

    View Slide

  125. Build for HoloLens2
    Rename new folder to build

    View Slide

  126. Build for HoloLens2
    ①build
    ②Select Folder

    View Slide

  127. Wait for finishing build

    View Slide

  128. Build for HoloLens2
    Open build

    View Slide

  129. Build for HoloLens2
    Open solution (*.sln) with VisualStudio2019

    View Slide

  130. Build for HoloLens2
    Release ARM

    View Slide

  131. Build for HoloLens2

    View Slide

  132. Build for HoloLens2
    Device

    View Slide

  133. Connect HoloLens2 and PC
    by using USB cable

    View Slide

  134. Build for HoloLens2
    ①デバッグ
    ②Start without debug

    View Slide

  135. 動作の様⼦

    View Slide

  136. p HoloLens2
    p ARFoundation(ARKit/ARCore)
    Running On AR Devices

    View Slide

  137. p HoloLens2
    p ARFoundation(ARKit/ARCore)
    Running On AR Devices

    View Slide

  138. Installation of ARFoundation
    Window

    View Slide

  139. Installation of ARFoundation
    Package Manager

    View Slide

  140. Installation of ARFoundation
    Advanced

    View Slide

  141. Installation of ARFoundation
    Turn ON Show preview packages

    View Slide

  142. Installation of ARFoundation
    ①AR Foundation
    ②Open version list

    View Slide

  143. Installation of ARFoundation
    ①All Version
    ②1.5.0-preview.6

    View Slide

  144. Installation of ARFoundation
    Install

    View Slide

  145. Setting procedure for iOS
    is from next page.

    View Slide

  146. Installation of ARKit (for iOS)
    ①ARKit XR Plugin
    ②2.1.2

    View Slide

  147. Installation of ARKit (for iOS)
    Install

    View Slide

  148. Setting procedure for Android
    is from next page.

    View Slide

  149. Installation of ARCore (for Android)
    ①ARCore XR Plugin
    ②2.1.2

    View Slide

  150. Installation of ARCore (for Android)
    Install

    View Slide

  151. Setting procedure for both
    ARCore/ARKi is from here.

    View Slide

  152. Build Setting for ARFoundation
    MixedRealityToolkit

    View Slide

  153. Build Setting for ARFoundation
    ①Camera
    ②Clone

    View Slide

  154. Build Setting for ARFoundation
    ②Clone
    ①Set ProfileName as Mobile MixedRealityToolkit...

    View Slide

  155. Build Setting for ARFoundation
    Camera Setting Providers

    View Slide

  156. Build Setting for ARFoundation
    Windows Mixed Reality Camera Settings

    View Slide

  157. Build Setting for ARFoundation
    See Type

    View Slide

  158. Build Setting for ARFoundation
    Open dropdown menu

    View Slide

  159. Build Setting for ARFoundation
    ①MicrosoftMixedReality.Toolkit.Experimental.UnityAR
    ②UnityARCameraSetting

    View Slide

  160. Remove Performance Viewer
    Turn Off Enable Diagnostics System

    View Slide

  161. Build for ARFoundation
    File

    View Slide

  162. Build for ARFoundation
    Build Settings...

    View Slide

  163. Build for ARKit(iOS)
    ①iOS
    ②Switch Platform

    View Slide

  164. Build for ARCore(Android)
    ①Android
    ②Switch Platform

    View Slide

  165. Build for ARFoundation
    Apply

    View Slide

  166. Build for ARFoundation
    Player Settings...

    View Slide

  167. Setting procedure for iOS
    is from next page.

    View Slide

  168. Build for ARKit(iOS)
    Other Settings

    View Slide

  169. Build for ARKit(iOS)
    Set Bundle Identifier
    ex) com.Yoshinaga.ARFukuoka

    View Slide

  170. Build for ARKit(iOS)
    Strip Engine CodeをOff

    View Slide

  171. Setting procedure for Android
    is from next page.

    View Slide

  172. Build for ARCore(Android)
    Set Package Name
    ex)com.yourname.arfukuoka

    View Slide

  173. Build and Run
    for ARCore/ARKit

    View Slide

  174. 動作の様⼦

    View Slide

  175. Ctrl / Command + S
    現状を保存

    View Slide

  176. Transform with
    Bounding Box
    Next Step...

    View Slide

  177. Next Step…

    View Slide

  178. Applying Bounding Box
    ①model
    ②Add Component

    View Slide

  179. Applying Bounding Box
    ①Search [bounding]
    ②Click BoundingBox

    View Slide

  180. 動作確認

    View Slide

  181. Store Objects in the Dock
    Next Step...

    View Slide

  182. View Slide

  183. Creating Dock
    Right Click Hierarchy

    View Slide

  184. Creating Dock
    Create Empty

    View Slide

  185. Creating Dock
    ①GameObject
    ②Rename as Dock1

    View Slide

  186. Creating Dock
    Add Component

    View Slide

  187. Creating Dock
    Search [sphere]
    SphereCollider

    View Slide

  188. Creating Dock
    SphereCollider is added

    View Slide

  189. Creating Dock
    Set radius of collider to 0.1m

    View Slide

  190. Creating Dock
    Add Component

    View Slide

  191. Creating Dock
    Search [dock]
    DockPosition

    View Slide

  192. Creating Dock
    DockPosition is added

    View Slide

  193. Creating Dock
    ①Right click Dock1
    ②3D Object
    ③Quad

    View Slide

  194. Creating Dock
    Set translation
    Position: 0, -0.1, 0
    Rotation: 90, 0, 0
    Scale: All 0.2

    View Slide

  195. Creating Dock
    ①MeshRenderer
    ②Open Materials

    View Slide

  196. Creating Dock
    Click at the right of Element0

    View Slide

  197. Creating Dock
    HolographicBackPlateCircular

    View Slide

  198. Creating Dock
    Like this

    View Slide

  199. Duplication of Dock
    Dock1

    View Slide

  200. Duplication of Dock
    Duplicate

    View Slide

  201. Duplication of Dock
    Dock1(1)
    Rename as Dock2

    View Slide

  202. Making Object Dockable
    ①Quad
    ②AddComponent

    View Slide

  203. Making Object Dockable
    dock
    Dockable

    View Slide

  204. Making Object Dockable
    Dockable

    View Slide

  205. Making Object Dockable
    ①model
    ②AddComponent

    View Slide

  206. Making Object Dockable
    dock
    Dockable

    View Slide

  207. Making Object Dockable
    Dockable

    View Slide

  208. Making Object Dockable
    ①Dock1
    ①Dock Position

    View Slide

  209. Making Object Dockable
    ①See Quad
    ②Drag & Drop into Docked Object

    View Slide

  210. Making Object Dockable
    ①Dock2
    ①Dock Position

    View Slide

  211. Making Object Dockable
    ①See model
    ②Drag & Drop into Docked Object

    View Slide

  212. Adjust Position of Docks
    ①Dock1
    Position:-0.2,-0.2,0.5

    View Slide

  213. Adjust Position of Docks
    ①Dock2
    Position: 0.2,-0.2,0.5

    View Slide

  214. View Slide

  215. Fin.

    View Slide

  216. View Slide