Getting Started with Non-Programming AR Development with MRTK v2.4.0

Slide deck of AR Fukuoka

Transcript

1. Getting Started with Non-Programming
   AR Development with MRTK
   

   View Slide

2. Speaker
   Name︓吉永崇(Takashi Yoshinaga)
   Affiliation︓Steampunk Digital
   ISIT
   Community: AR Fukuoka
   

   View Slide

3. SNS
   https://www.linkedin.com/in/tks-yoshinaga/
   https://twitter.com/Taka_Yoshinaga
   LinkedIn
   Twitter
   

   View Slide

4. #AR_Fukuoka
   Hashtag
   

   View Slide

5. Download Sample Data
   http://arfukuoka.lolipop.jp/mrtk_np/Sample.zip
   

   View Slide

6. Download MRTK v2.4.0 (1/3)
   https://github.com/Microsoft/MixedRealityToolkit-Unity
   Releases
   

   View Slide

7. Download MRTK v2.4.0 (2/3)
   MRTK v2.4.0 is used for this tutorial
   Scroll down
   

   View Slide

8. Download MRTK v2.4.0 (3/3)
   Download only Foundation.2.4.0.unitypackage
   Foundation.2.4.0.unitypackage
   

   View Slide

9. Todayʼs Goal
   Show and Manipulate Virtual Object without Programming
   https://youtu.be/Yae3bOtidTk
   

   View Slide

10. Creating Project (1/6)
    Launch Unity Hub and click Project
    プロジェクト
    

    View Slide

11. Creating Project (2/6)
    Click ▽ aligned on the right side of NEW button.
    新規作成横の▽
    

    View Slide

12. Creating Project (3/6)
    Select 2018.4.xxf1 (2018.4.23f1 is selected in this deck)
    2018.4.xxf1
    

    View Slide

13. Creating Project (4/6)
    ①3D ②Set the project name
    (ex.ARFukuoka_MRTK)
    ③Select the directory
    to save the project
    

    View Slide

14. Creating Project (5/6)
    Create
    

    View Slide

15. Creating Project (6/6)
    Unity Editor will be shown
    

    View Slide

16. Installation of MRTK (1/3)
    Double click the unitypackage.
    Click Import button of Import Unity Package dialog.
    Microsoft.MixedReality.
    Toolkit.Unity.Foundation
    .2.4.0.unitypackage
    Import
    

    View Slide

17. Installation of MRTK (2/3)
    Apply
    Click Apply button to switch the setting for XR.
    

    View Slide

18. Installation of MRTK (3/3)
    Close
    Close MRTK Project Configurator dialog.
    

    View Slide

19. Adding 3D Object into the Space (1/3)
    Right click blank
    of Hierarchy
    

    View Slide

20. Adding 3D Object into the Space (2/3)
    ①3D Object
    ②Quad
    

    View Slide

21. Adding 3D Object into the Space (3/3)
    Quad is shown
    ※Backside is invisible
    

    View Slide

22. Adjust Viewpoint of Scene
    [←] [→]: Move to Left/RIght
    [↑][↓]: Zoom In/Out
    [Alt]+Drag: Rotation
    +Drag: Move to ←→↑↓
    

    View Slide

23. Adjust Viewpoint of Scene
    Object can be shown on the center of Scene tab.
    Double click Quad
    

    View Slide

24. Run
    カメラから⾒た空間
    が表⽰される
    Click to play
    Click it again to
    stop preview
    

    View Slide

25. Modification of Objectʼs Transform
    Click Quad
    

    View Slide

26. Modification of Objectʼs Transform
    Move Rotate Scale
    

    View Slide

27. Inputting Values to Transform Object
    Click Quad
    

    View Slide

28. Inputting Values to Transform Object
    Inspector
    p Properties of each object can be added/edited in Inspector
    p Position/Rotation/Scale are editable in Transform.
    

    View Slide

29. Inputting Values to Transform Object
    p Properties of each object can be added/edited in Inspector
    p Position/Rotation/Scale are editable in Transform.
    Set as following
    Positionを 0 0 1
    Rotation 0 0 0
    Scale 0.3 0.3 0.3
    

    View Slide

30. Adjust Viewpoint of Scene
    Letʼs set the object at
    the center of scene tab
    

    View Slide

31. Adjust Viewpoint of Scene
    Double Click Quad
    

    View Slide

32. Next Step
    Not only Transform but also
    appearance and behavior and
    properties can be edited in Inspector
    

    View Slide

33. Changing Color of Cube (1/4)
    ① Quad
    ② See Mesh Renderer
    ③ Open Materials
    

    View Slide

34. Changing Color of Cube (2/4)
    Element0
    

    View Slide

35. Changing Color of Cube (3/4)
    Appearance information
    is described in Material file
    

    View Slide

36. Changing Color of Cube (4/4)
    Detail of Material is shown at the
    bottom of Inspector.
    But Default-Material is not editable.
    

    View Slide

37. Itʼs necessary to create Material file
    to edit appearance of an object.
    

    View Slide

38. Creating Material (1/3)
    Create Material to describe color of Quad.
    ①Assets
    ②Right Click
    

    View Slide

39. Creating Material (2/3)
    ①Create
    ②Material
    Create Material to describe color of Quad.
    

    View Slide

40. Creating Material (3/3)
    Create Material to describe color of Quad.
    New Material should appear
    

    View Slide

41. Application of Material (1/3)
    ①Quad
    Open Materials of Quad in Inspector
    ② See Default-Material
    

    View Slide

42. Application of Material (2/3)
    Replace Default-Material with New Material.
    Drag & Drop into
    Default-Material
    

    View Slide

43. Application of Material (3/3)
    ①Quad
    ②Material should be
    New Material
    

    View Slide

44. Next Step...
    

    View Slide

45. Application of Texture (1/8)
    ②Open Dropdown
    Menu
    ③Unlit
    ④Transparent
    ①See New Material
    

    View Slide

46. Application of Texture (2/8)
    ①Click ▼ to open
    NewMateria
    ②Texture can be set here
    

    View Slide

47. Application of Texture (3/8)
    ①Assets
    ②Import New Asset
    

    View Slide

48. Application of Texture (4/8)
    ①arfukuoka.png
    ②Import
    

    View Slide

49. Application of Texture (5/8)
    ①Quad
    ②Open New Materia
    

    View Slide

50. Application of Texture (6/8)
    ①Assets
    ② see arfukuoka.png
    (Do NOT click)
    

    View Slide

51. Application of Texture (7/8)
    Drag & Drop arfukuoka.png
    into None(Texture)
    

    View Slide

52. Application of Texture (8/8)
    Texture is applied
    

    View Slide

53. How to Save this Scene (1/4)
    File
    

    View Slide

54. How to Save this Scene (2/4)
    SaveAs...
    

    View Slide

55. How to Save this Scene (3/4)
    ①Name as Sample1
    ②Save
    

    View Slide

56. How to Save this Scene (4/4)
    Sample1 should be appear in Assets folder
    

    View Slide

57. You can override the scene
    with current info by
    Ctrl / Command + S
    

    View Slide

58. Letʼs use MRTK from now!
    

    View Slide

59. What Youʼll Do with MRTK at First?
    Prepare moving around and using hand in UnityEditor
    

    View Slide

60. Using Camera of MRTK (1/3)
    MixedRealityToolkit
    

    View Slide

61. Using Camera of MRTK (2/3)
    Add to Scene and Configure...
    

    View Slide

62. Using Camera of MRTK (3/3)
    MixedRealityToolKit and
    MixedRealityPlayspace
    should appear
    

    View Slide

63. Play (1/4)
    Click Play
    

    View Slide

64. Play (2/4)
    [A][D] :Move to Left and Right
    [W][S] :Zoom In/Out
    [Q][E] :Move to Up and Down
    RightClick + Drag :Rotation
    

    View Slide

65. Play (3/4)
    ※Object canʼt be manipulated so far
    [Shift] :Show left hand
    [Space] :Show right hand
    While hand is appearing+
    [Click] :Pinch
    [Scroll] :Forward/Back
    

    View Slide

66. Play (4/4)
    Stop before going to next step
    

    View Slide

67. Letʼs manipulate Quad
    

    View Slide

68. Making Quad Manipulatable (1/3)
    ①Quad
    ②Add Component
    

    View Slide

69. Making Quad Manipulatable (2/3)
    Search [manipulation]
    Select ManipulationHandler
    

    View Slide

70. Making Quad Manipulatable (3/3)
    ①Quad
    ②ManipulationHandler
    is added
    

    View Slide

71. Play (1/4)
    Playをクリック
    

    View Slide

72. One Hand Manipulation
    

    View Slide

73. Play (2/4)
    Shift or Space
    

    View Slide

74. Play (3/4)
    Put the cursor on Quad
    

    View Slide

75. Play (4/4)
    Drag and move Quad
    

    View Slide

76. Two Hand Manipulation
    

    View Slide

77. Play (1/5)
    Show left hand by
    pushing Shift
    

    View Slide

78. Play (2/5)
    ①Click after putting cursor on Quad
    ② Hit [T] to fix left hand
    

    View Slide

79. Play (3/5)
    Show right hand by pushing Space
    

    View Slide

80. Play (4/5)
    Drag and move
    

    View Slide

81. Play (5/5)
    You can Rotate/Scale Cube
    

    View Slide

82. For Smartphone
    

    View Slide

83. Play (1/2)
    Put the cursor shown in the center of view on Quad
    

    View Slide

84. Play (2/2)
    Drag to move
    

    View Slide

85. Letʼs Add Another Object
    

    View Slide

86. Adding Obj File(1/6)
    Assets
    

    View Slide

87. Adding Obj File(2/6)
    Sample Folder → Model
    

    View Slide

88. Adding Obj File (3/6)
    Drag & Drop
    Model into Assets
    

    View Slide

89. Adding Obj File (4/6)
    Model
    

    View Slide

90. Adding Obj File (5/6)
    model.obj
    

    View Slide

91. Adding Obj File (6/6)
    Drag & Drop model.obj
    into Hierarchy
    

    View Slide

92. Adjusting Object (1/3)
    ①model
    Put chair as following
    Position 0.5 0 1
    

    View Slide

93. Adjusting Object (2/3)
    Click ▼ at the left of mode
    to open child hierarchy
    

    View Slide

94. Adjusting Object (3/3)
    model under upper level of model
    Set trans form as bellow
    Position 0 -0.45 0
    Rotation 0 180 0
    It is possible to adjust the Transform using only the model in the upper level,
    but for the sake of efficiency, I divided the operation into two steps
    

    View Slide

95. In order to enable chair to detect
    contact with hand, Collider should
    be added to model.obj
    

    View Slide

96. Adding Collider (1/5)
    ①model
    (上の階層の⽅) ②Add Component
    

    View Slide

97. Adding Collider (2/5)
    ②BoxCollider
    ①Search box
    

    View Slide

98. Adding Collider (3/5)
    Bounding box will appear
    

    View Slide

99. Adding Collider (4/5)
    ①model
    ②Edit Collider
    

    View Slide

100. Adding Collider (5/5)
     Adjust position of each
     face by moving■
     

     View Slide

101. Make Chair Manipulatable (1/3)
     ①model
     ②Add Component
     

     View Slide

102. Make Chair Manipulatable (2/3)
     ②ManipulationHandler
     ① Searchmanipulation
     

     View Slide

103. Make Chair Manipulatable (3/3)
     ①model
     ② Manipulation Handler is added
     

     View Slide

104. Allow Only Scaling for 2 Hand Operation
     ①model
     ②Set Scale for
     Two Hand Manipulation Type
     

     View Slide

105. Play
     

     View Slide

106. Play
     Stop
     

     View Slide

107. Remove Performance Viewer
     Can be deleted
     

     View Slide

108. Remove Performance Viewer
     ①MixedRealityToolKit
     ② Open dropdown menu
     under Mixed Reality Toolkit
     

     View Slide

109. Remove Performance Viewer
     ①MixedRealityToolKit
     ②DefaultHololens2ConfigureProfile
     

     View Slide

110. Remove Performance Viewer
     Clone
     

     View Slide

111. Remove Performance Viewer
     ②Clone
     ①Change Profile Name to Sample1 MixedRealityToolkit...
     

     View Slide

112. Remove Performance Viewer
     ①Diagnostics
     ②Turn Enable Diagnostics
     System into Off
     

     View Slide

113. Play
     Performance Viewer become invisible
     

     View Slide

114. Play
     Stop
     

     View Slide

115. p HoloLens2
     p ARFoundation(ARKit/ARCore)
     Running On AR Devices
     

     View Slide

116. p HoloLens2
     p ARFoundation(ARKit/ARCore)
     Running On AR Devices
     

     View Slide

117. Build Setting for HoloLens2
     File
     

     View Slide

118. Build Setting for HoloLens2
     Build Setting
     

     View Slide

119. Build Setting for HoloLens2
     ①Universal Windows Platform
     ②Switch Platform
     

     View Slide

120. Build Setting for HoloLens2
     Apply
     

     View Slide

121. Build Setting for HoloLens2
     Player Settings...
     

     View Slide

122. Build Setting for HoloLens2
     ①XR Settings
     ②Select 16-bit depth ad Depth Format
     

     View Slide

123. Build for HoloLens2
     Build
     

     View Slide

124. Build for HoloLens2
     New Folder
     

     View Slide

125. Build for HoloLens2
     Rename new folder to build
     

     View Slide

126. Build for HoloLens2
     ①build
     ②Select Folder
     

     View Slide

127. Wait for finishing build
     

     View Slide

128. Build for HoloLens2
     Open build
     

     View Slide

129. Build for HoloLens2
     Open solution (*.sln) with VisualStudio2019
     

     View Slide

130. Build for HoloLens2
     Release ARM
     

     View Slide

131. Build for HoloLens2
     ▼
     

     View Slide

132. Build for HoloLens2
     Device
     

     View Slide

133. Connect HoloLens2 and PC
     by using USB cable
     

     View Slide

134. Build for HoloLens2
     ①デバッグ
     ②Start without debug
     

     View Slide

135. 動作の様⼦
     

     View Slide

136. p HoloLens2
     p ARFoundation(ARKit/ARCore)
     Running On AR Devices
     

     View Slide

137. p HoloLens2
     p ARFoundation(ARKit/ARCore)
     Running On AR Devices
     

     View Slide

138. Installation of ARFoundation
     Window
     

     View Slide

139. Installation of ARFoundation
     Package Manager
     

     View Slide

140. Installation of ARFoundation
     Advanced
     

     View Slide

141. Installation of ARFoundation
     Turn ON Show preview packages
     

     View Slide

142. Installation of ARFoundation
     ①AR Foundation
     ②Open version list
     

     View Slide

143. Installation of ARFoundation
     ①All Version
     ②1.5.0-preview.6
     

     View Slide

144. Installation of ARFoundation
     Install
     

     View Slide

145. Setting procedure for iOS
     is from next page.
     

     View Slide

146. Installation of ARKit (for iOS)
     ①ARKit XR Plugin
     ②2.1.2
     

     View Slide

147. Installation of ARKit (for iOS)
     Install
     

     View Slide

148. Setting procedure for Android
     is from next page.
     

     View Slide

149. Installation of ARCore (for Android)
     ①ARCore XR Plugin
     ②2.1.2
     

     View Slide

150. Installation of ARCore (for Android)
     Install
     

     View Slide

151. Setting procedure for both
     ARCore/ARKi is from here.
     

     View Slide

152. Build Setting for ARFoundation
     MixedRealityToolkit
     

     View Slide

153. Build Setting for ARFoundation
     ①Camera
     ②Clone
     

     View Slide

154. Build Setting for ARFoundation
     ②Clone
     ①Set ProfileName as Mobile MixedRealityToolkit...
     

     View Slide

155. Build Setting for ARFoundation
     Camera Setting Providers
     

     View Slide

156. Build Setting for ARFoundation
     Windows Mixed Reality Camera Settings
     

     View Slide

157. Build Setting for ARFoundation
     See Type
     

     View Slide

158. Build Setting for ARFoundation
     Open dropdown menu
     

     View Slide

159. Build Setting for ARFoundation
     ①MicrosoftMixedReality.Toolkit.Experimental.UnityAR
     ②UnityARCameraSetting
     

     View Slide

160. Remove Performance Viewer
     Turn Off Enable Diagnostics System
     

     View Slide

161. Build for ARFoundation
     File
     

     View Slide

162. Build for ARFoundation
     Build Settings...
     

     View Slide

163. Build for ARKit(iOS)
     ①iOS
     ②Switch Platform
     

     View Slide

164. Build for ARCore(Android)
     ①Android
     ②Switch Platform
     

     View Slide

165. Build for ARFoundation
     Apply
     

     View Slide

166. Build for ARFoundation
     Player Settings...
     

     View Slide

167. Setting procedure for iOS
     is from next page.
     

     View Slide

168. Build for ARKit(iOS)
     Other Settings
     

     View Slide

169. Build for ARKit(iOS)
     Set Bundle Identifier
     ex) com.Yoshinaga.ARFukuoka
     

     View Slide

170. Build for ARKit(iOS)
     Strip Engine CodeをOff
     

     View Slide

171. Setting procedure for Android
     is from next page.
     

     View Slide

172. Build for ARCore(Android)
     Set Package Name
     ex)com.yourname.arfukuoka
     

     View Slide

173. Build and Run
     for ARCore/ARKit
     

     View Slide

174. 動作の様⼦
     

     View Slide

175. Ctrl / Command + S
     現状を保存
     

     View Slide

176. Transform with
     Bounding Box
     Next Step...
     

     View Slide

177. Next Step…
     

     View Slide

178. Applying Bounding Box
     ①model
     ②Add Component
     

     View Slide

179. Applying Bounding Box
     ①Search [bounding]
     ②Click BoundingBox
     

     View Slide

180. 動作確認
     

     View Slide

181. Store Objects in the Dock
     Next Step...
     

     View Slide

182. View Slide

183. Creating Dock
     Right Click Hierarchy
     

     View Slide

184. Creating Dock
     Create Empty
     

     View Slide

185. Creating Dock
     ①GameObject
     ②Rename as Dock1
     

     View Slide

186. Creating Dock
     Add Component
     

     View Slide

187. Creating Dock
     Search [sphere]
     SphereCollider
     

     View Slide

188. Creating Dock
     SphereCollider is added
     

     View Slide

189. Creating Dock
     Set radius of collider to 0.1m
     

     View Slide

190. Creating Dock
     Add Component
     

     View Slide

191. Creating Dock
     Search [dock]
     DockPosition
     

     View Slide

192. Creating Dock
     DockPosition is added
     

     View Slide

193. Creating Dock
     ①Right click Dock1
     ②3D Object
     ③Quad
     

     View Slide

194. Creating Dock
     Set translation
     Position: 0, -0.1, 0
     Rotation: 90, 0, 0
     Scale: All 0.2
     

     View Slide

195. Creating Dock
     ①MeshRenderer
     ②Open Materials
     

     View Slide

196. Creating Dock
     Click at the right of Element0
     

     View Slide

197. Creating Dock
     HolographicBackPlateCircular
     

     View Slide

198. Creating Dock
     Like this
     

     View Slide

199. Duplication of Dock
     Dock1
     

     View Slide

200. Duplication of Dock
     Duplicate
     

     View Slide

201. Duplication of Dock
     Dock1(1)
     Rename as Dock2
     

     View Slide

202. Making Object Dockable
     ①Quad
     ②AddComponent
     

     View Slide

203. Making Object Dockable
     dock
     Dockable
     

     View Slide

204. Making Object Dockable
     Dockable
     

     View Slide

205. Making Object Dockable
     ①model
     ②AddComponent
     

     View Slide

206. Making Object Dockable
     dock
     Dockable
     

     View Slide

207. Making Object Dockable
     Dockable
     

     View Slide

208. Making Object Dockable
     ①Dock1
     ①Dock Position
     

     View Slide

209. Making Object Dockable
     ①See Quad
     ②Drag & Drop into Docked Object
     

     View Slide

210. Making Object Dockable
     ①Dock2
     ①Dock Position
     

     View Slide

211. Making Object Dockable
     ①See model
     ②Drag & Drop into Docked Object
     

     View Slide

212. Adjust Position of Docks
     ①Dock1
     Position:-0.2,-0.2,0.5
     

     View Slide

213. Adjust Position of Docks
     ①Dock2
     Position: 0.2,-0.2,0.5
     

     View Slide

214. View Slide

215. Fin.
     

     View Slide

216. View Slide