UX Design for Virtual Reality

A background around VR for designers and an overview of the various user interface and UX problems we faces as creators of digital products.
NOTE: Unfortunately this must be posted as PDF rather than PPT and so the GIFs inside are not animated.

1. UX Design for Virtual Reality O R I, FOR ONE, WELCOME OUR VIRTUAL OVERLOARDS! @torbochkina | @IxDAPGH | #uxvrpgh Slides = goo.gl/9lwAUR
2. ABOUT ME UX consultant + game designer Sci-fi + fantasy author (Lots of interests colliding in VR) @torbochkina | @IxDAPGH | #uxvrpgh Slides = goo.gl/9lwAUR
3. NOTE Lots of GIFs incoming Some with mild / non-graphic / cartoony violence @torbochkina | @IxDAPGH | #uxvrpgh Slides = goo.gl/9lwAUR
4. VR has long fascinated us.
6. Is it possible to simulate reality?
8. And if it is, what does that mean for us?
9. Memory Identity Consciousness Empathy
11. Largely dystopian vision of the future
12. Well, that future is here!
13. Well, that future is here! Yay.
14. Reality vs Fiction
15. What you can do in VR right now
16. @torbochkina | @IxDAPGH | #uxvrpgh
17. Learn how to use dangerous machinery @torbochkina | @IxDAPGH | #ux4vrpgh
18. Teach kids history and science @torbochkina | @IxDAPGH | #ux4vrpgh
19. Redo your kitchen! @torbochkina | @IxDAPGH | #ux4vrpgh
20. Hang out with friends long distance @torbochkina | @IxDAPGH | #ux4vrpgh
21. Meet new ones @torbochkina | @IxDAPGH | #ux4vrpgh
22. Meet new ones @torbochkina | @IxDAPGH | #ux4vrpgh
23. Get a workout @torbochkina | @IxDAPGH | #ux4vrpgh
24. Create art @torbochkina | @IxDAPGH | #ux4vrpgh
25. Travel to other places @torbochkina | @IxDAPGH | #ux4vrpgh
26. Visit Tuscany @torbochkina | @IxDAPGH | #ux4vrpgh
27. Impossible places @torbochkina | @IxDAPGH | #ux4vrpgh
28. Impossible places @torbochkina | @IxDAPGH | #ux4vrpgh
29. Watch movies @torbochkina | @IxDAPGH | #ux4vrpgh
33. Watch sports and esports @torbochkina | @IxDAPGH | #ux4vrpgh
34. Command virtual armies @torbochkina | @IxDAPGH | #ux4vrpgh
35. Evade and fight off robots @torbochkina | @IxDAPGH | #ux4vrpgh
36. Visualize large datasets @torbochkina | @IxDAPGH | #ux4vrpgh
37. Conquer other galactic nations @torbochkina | @IxDAPGH | #ux4vrpgh
38. Become a space pirate @torbochkina | @IxDAPGH | #ux4vrpgh
39. Slow time @torbochkina | @IxDAPGH | #ux4vrpgh
40. AND THROW BULLETS AT PEOPLE! @torbochkina | @IxDAPGH | #ux4vrpgh
41. Who knows what you can do @torbochkina | @IxDAPGH | #ux4vrpgh
42. And right now you’re like… @torbochkina | @IxDAPGH | #ux4vrpgh
43. Or maybe you’re like: @torbochkina | @IxDAPGH | #ux4vrpgh
44. Relieve pain better than morphine
45. Research by and copyright copyright Hunter Hoffman, University of Washington: https://www.hitl.washington. edu/projects/vrpain/ Images copyright NBC News: http://www.nbcnews.com/vi deo/rock-center/49849152 @torbochkina | @IxDAPGH | #ux4vrpgh
46. @torbochkina | @IxDAPGH | #ux4vrpgh
47. What even is Virtual Reality? @torbochkina | @IxDAPGH | #ux4vrpgh
48. An illusion @torbochkina | @IxDAPGH | #ux4vrpgh
49. A trick that your brain believes @torbochkina | @IxDAPGH | #ux4vrpgh
50. Imagine glasses to another world @torbochkina | @IxDAPGH | #ux4vrpgh
51. IT FEELS REALLY REAL! @torbochkina | @IxDAPGH | #ux4vrpgh
52. Main systems • Mobile | Google Cardboard, Samsung GearVR • Immersive | Oculus Rift, HTC Vive • Console | PlayStation VR, Xbox? @torbochkina | @IxDAPGH | #ux4vrpgh
53. Virtual Reality right now consists of: COMPUTER SPACE HEAD MOUNTED DISPLAY INPUT TRACKING HEAD, HANDS, FEET, SEAT, ETC. @torbochkina | @IxDAPGH | #ux4vrpgh
54. This is a medium @torbochkina | @IxDAPGH | #ux4vrpgh
55. Print | Radio | TV | Film | Internet @torbochkina | @IxDAPGH | #ux4vrpgh
56. Difference between AR + VR • Augmented = Overlaid on reality = Google Glass • Virtual = Different world • Similar UI patterns • Potentially very different usage environment • Also not talking about room or holodeck style VR @torbochkina | @IxDAPGH | #ux4vrpgh
57. Why now? @torbochkina | @IxDAPGH | #ux4vrpgh
58. 60s: Ivan Sutherland’s “Ultimate Display” @torbochkina | @IxDAPGH | #ux4vrpgh
59. 3D + Haptic + Audio + Interaction or “a room within which the computer can control the existence of matter” @torbochkina | @IxDAPGH | #ux4vrpgh
60. 60s: Flight simulators of Thomas Furness @torbochkina | @IxDAPGH | #ux4vrpgh
61. 1980s: Jaron Lanier’s visual programming lab @torbochkina | @IxDAPGH | #ux4vrpgh
62. 1990s: Nintendo + SEGA @torbochkina | @IxDAPGH | #ux4vrpgh
63. Palmer Luckey @torbochkina | @IxDAPGH | #ux4vrpgh
64. Palmer Luckey @torbochkina | @IxDAPGH | #ux4vrpgh
65. Revolution @torbochkina | @IxDAPGH | #ux4vrpgh
66. Why now? Cheap smart phone screens + Luck(ey) @torbochkina | @IxDAPGH | #ux4vrpgh
67. What does VR mean for us as designers? @torbochkina | @IxDAPGH | #ux4vrpgh
68. Well, there are a few problems. @torbochkina | @IxDAPGH | #ux4vrpgh
69. @torbochkina | @IxDAPGH | #ux4vrpgh
70. There’s a lot of work for designers to do! (Yay!) @torbochkina | @IxDAPGH | #ux4vrpgh
71. A lot of unexplored territory @torbochkina | @IxDAPGH | #ux4vrpgh
72. <3 Generalism @torbochkina | @IxDAPGH | #ux4vrpgh
73. Physical UX @torbochkina | @IxDAPGH | #ux4vrpgh
74. Industrial Design @torbochkina | @IxDAPGH | #ux4vrpgh
75. Design of Environments @torbochkina | @IxDAPGH | #ux4vrpgh
76. P R O B L E M : What experiences are best suited to VR? @torbochkina | @IxDAPGH | #ux4vrpgh
77. Too early to say but… • Immersive, Physical, Spatial • Long form, worthy of concentration and attention • Not in need of text input or extensive reading • Likely good: Some games, renovation, training, therapy… • Not yet proven: Reading, writing, work tasks, advertising-centric… • Most importantly: Worth putting on goggles for @torbochkina | @IxDAPGH | #ux4vrpgh
78. P R O B L E M : How do we architect information for VR? @torbochkina | @IxDAPGH | #ux4vrpgh
79. Site maps IA = Search Wireframes Labeling Content Models Controlled Vocab Hierarchy Taxonomy Topology Navigation @torbochkina | @IxDAPGH | #ux4vrpgh Organization Schemes
80. Site maps IA = Search Wireframes Labeling Content Models Controlled Vocab Hierarchy Taxonomy Topology Navigation @torbochkina | @IxDAPGH | #ux4vrpgh Organization Schemes
81. IA is built around the idea of a page @torbochkina | @IxDAPGH | #ux4vrpgh
82. Relationships between pages A favorite I still find myself referring to: Visual Vocab via Jesse James Garrett http://www.jjg.net/ia/visvocab/
84. Layout inside pages @torbochkina | @IxDAPGH | #ux4vrpgh
85. But in VR… @torbochkina | @IxDAPGH | #ux4vrpgh
86. There is no page. @torbochkina | @IxDAPGH | #ux4vrpgh
87. • http://www.jjg.net/ia/visvocab/
88. There’s not really any screen. @torbochkina | @IxDAPGH | #ux4vrpgh
89. • http://www.jjg.net/ia/visvocab/
90. There’s not even any corners. @torbochkina | @IxDAPGH | #ux4vrpgh
91. • http://www.jjg.net/ia/visvocab/
92. So how do we structure the information that the user needs? @torbochkina | @IxDAPGH | #ux4vrpgh
93. Where are we going to put all this stuff?!??! @torbochkina | @IxDAPGH | #ux4vrpgh
94. We break things into spaces and moments instead. (Still figuring this out really) @torbochkina | @IxDAPGH | #ux4vrpgh
95. P R O B L E M : Which inputs should we support? @torbochkina | @IxDAPGH | #ux4vrpgh
96. They are legion: • HTC Vive Controllers • Xbox Gamepad • Oculus Remote • Oculus Touch • Leap Motion • PlayStation Controllers @torbochkina | @IxDAPGH | #ux4vrpgh • One button Cardboard • Mini touchpad on GearVR • Specialized controllers like flight sticks?! • Weird haptic accessories? • 4D Platforms like the Omni
97. Wide range of context of use: • Seated • Standing • Small rooms • Large rooms • At home • On the go @torbochkina | @IxDAPGH | #ux4vrpgh
98. P R O B L E M : How do we best bring users into (and out of) our virtual worlds? @torbochkina | @IxDAPGH | #ux4vrpgh
99. IA of most early experiences + demos: A room or A world You’re in a VR ROOM! LOOK! WHOA!!!!! @torbochkina | @IxDAPGH | #ux4vrpgh
100. Tuscany @torbochkina | @IxDAPGH | #ux4vrpgh
101. A little more, but still very simple (no or 1 menu) Instructions Let’s get adjusted. How do we do this? OK, let’s go. Lobby Experience / Room @torbochkina | @IxDAPGH | #ux4vrpgh
102. People need time and space to adjust and prepare @torbochkina | @IxDAPGH | #ux4vrpgh
103. I Expect You to Die @torbochkina | @IxDAPGH | #ux4vrpgh
104. P R O B L E M : How do we represent the user? The controllers? @torbochkina | @IxDAPGH | #ux4vrpgh
105. @torbochkina | @IxDAPGH | #ux4vrpgh
106. @torbochkina | @IxDAPGH | #ux4vrpgh
107. No clear best practice • Not at all • With a new controller customized to your experience • Glowing orbs • With generic or transparent human hands and no body • With specific hands and/or body of a character you play @torbochkina | @IxDAPGH | #ux4vrpgh
108. P R O B L E M : What kinds of UIs are best suited to VR experiences? @torbochkina | @IxDAPGH | #ux4vrpgh
109. UI SOLUTION: 2D in 3D space
110. Oculus Store
111. A belt @torbochkina | @IxDAPGH | #ux4vrpgh
112. A belt @torbochkina | @IxDAPGH | #ux4vrpgh
113. Tiltbrush @torbochkina | @IxDAPGH | #ux4vrpgh
114. Menu, tied to Fingers @torbochkina | @IxDAPGH | #ux4vrpgh
115. Rapid prototyping
116. For flat UI, where do we anchor it? • Air? • Wall? • Ground? • In a dashboard? • On a physical object? @torbochkina | @IxDAPGH | #ux4vrpgh
117. UI SOLUTION: Gestural
118. Create Shape or New = Gesture @torbochkina | @IxDAPGH | #ux4vrpgh
119. @torbochkina | @IxDAPGH | #ux4vrpgh
120. Menu, tied to Arm @torbochkina | @IxDAPGH | #ux4vrpgh
121. Drawing @torbochkina | @IxDAPGH | #ux4vrpgh
122. Drawing? @torbochkina | @IxDAPGH | #ux4vrpgh
123. UI SOLUTION: Spaces
125. Air Mech
126. Air Mech
127. Air Mech
128. Air Mech
129. Air Mech
130. Air Mech
131. Air Mech
132. Spatial Work Areas
133. Side note: Personal space…
134. UI SOLUTION: Virtual Physical Objects
135. @torbochkina | @IxDAPGH | #ux4vrpgh
136. @torbochkina | @IxDAPGH | #ux4vrpgh
137. @torbochkina | @IxDAPGH | #ux4vrpgh
138. @torbochkina | @IxDAPGH | #ux4vrpgh
139. @torbochkina | @IxDAPGH | #ux4vrpgh
140. @torbochkina | @IxDAPGH | #ux4vrpgh
141. @torbochkina | @IxDAPGH | #ux4vrpgh
142. @torbochkina | @IxDAPGH | #ux4vrpgh
143. @torbochkina | @IxDAPGH | #ux4vrpgh
144. @torbochkina | @IxDAPGH | #ux4vrpgh
145. @torbochkina | @IxDAPGH | #ux4vrpgh
146. @torbochkina | @IxDAPGH | #ux4vrpgh
147. Toolbar > Cat @torbochkina | @IxDAPGH | #ux4vrpgh
148. Toolbar > Cat, Pet @torbochkina | @IxDAPGH | #ux4vrpgh
149. @torbochkina | @IxDAPGH | #ux4vrpgh
150. @torbochkina | @IxDAPGH | #ux4vrpgh
151. @torbochkina | @IxDAPGH | #ux4vrpgh
152. @torbochkina | @IxDAPGH | #ux4vrpgh
153. Exit > Burrito @torbochkina | @IxDAPGH | #ux4vrpgh
154. Exit > Burrito @torbochkina | @IxDAPGH | #ux4vrpgh
155. Question when real world metaphors surpass virtual ones
156. UI SOLUTION: All new combinations
157. Movement + WASD > Teleport http://www.gamasutra.com/view/news/266687/Do_the_locomotion_How_VR_devs_are_expanding_the_virtual_space.php @torbochkina | @IxDAPGH | #ux4vrpgh
158. The great beyond • A character that talks to you? • Voice recognition? • A door or place you move through? • Something else? • No “UI” at all
159. P R O B L E M : How do we keep experiences comfortable? @torbochkina | @IxDAPGH | #ux4vrpgh
160. Not making people sick • High latency and performance • Minimize moving cameras outside of user control • Noses + vehicle frames • Design for comfort • Seated experiences • Teleportation • Clear content warnings @torbochkina | @IxDAPGH | #ux4vrpgh
161. P R O B L E M : How do we effectively collaborate with developers? @torbochkina | @IxDAPGH | #ux4vrpgh
162. How do we collaborate? • In 3D – Unity, 3D environments, physical models • Establish vocabulary • Lots of iteration • Lots of user testing
163. That’s all. You got this.
164. So…
166. Will the dystopian vision of a dark VR future come to pass?
167. Does this cat herald the impending doom and inevitable demise of humanity and/or human consciousness as we know it???
168. Maybe? Maybe not?
169. Here is your chance.
170. The future is a thing that we make.
171. Thank you Twitter: @torbochkina Want more? Next PGH VR Meetup July 24! slides: https://goo.gl/9lwAUR game: astralshard.com books: rkthorne.com consulting: scarletinteractive.com @torbochkina | @IxDAPGH | #ux4vrpgh
172. Further reading: • UX Pointers for VR Design, by Timoni West • https://medium.com/@timoni/ux-pointers-for-vr-design-dd52b718e19#.btjlsgfqa • LeapMotion’s VR Design Best Practices • https://medium.com/@LeapMotion/vr-design-best-practices-bb889c2dc70#.92nmn180c • Design Insights for Virtual Reality UX, Punchcut • https://uxdesign.cc/design-insights-for-virtual-reality-ux-7ae41a0c5a1a#.1q6in8yv1 • Goodbye Metaphors, Hello World—Interface Design in Virtual Reality, by Austin McCasland • https://medium.com/yvr/goodbye-metaphors-hello-world-interface-design-in-virtual-reality-d2462e52a029#.mvqtimprw
173. References • Research on the positive effects of games: • http://www.ted.com/talks/jane_mcgonigal_the_game_that_can_give_you_10_extra_years_of_life • https://www.superbetter.com/ • https://janemcgonigal.com/ • Bullet Train: http://www.theverge.com/2015/9/24/9392659/bullet-train-oculus-touch-epic-games-virtual- reality • Gifs of Valve HTC Vive Demo / Launch videos: http://www.techinsider.io/what-virtual-reality-looks-like-photos- 2016-4 | Video itself: https://www.youtube.com/watch?v=qYfNzhLXYGc • Industrial training: http://www.virtalis.com/media/galleries/?tags=training • Ikea VR Experience: http://store.steampowered.com/app/447270/ • Oculus Toybox gif: http://www.huffingtonpost.com/sophia-dominguez/what-happened-at-oculus- c_b_8234326.html • Van Gogh world: The Night Café, by Borrowed Light VR • Eve Valkyrie space game gif: http://kotaku.com/one-wild-occasionally-nauseating-week-of-virtual-reali- 1767442615
174. References • Woman next to man using Leap Motion Gif https://medium.com/@downtohoerth/i-love-vr-but-hundreds-of- thousands-of-people-think-i-hate-it-5a3e803ea3e0#.kb3sj3v0r • Child falling using Vive: http://www.theverge.com/2016/3/10/11196388/virtual-reality-table-gif • Samsung Gear VR Users: via Austin Evans BIT.LY/1QJZTKG • VR History: http://www.vrs.org.uk/virtual-reality/history.html • VR History: http://www.theatlantic.com/technology/archive/2016/03/virtual-reality-is-just-television-for-the- computer-junkie/475632/ • VR History: http://www.theverge.com/a/virtual-reality/ • Images of Palmer Luckey: http://www.smithsonianmag.com/innovation/how-palmer-luckey-created-oculus-rift- 180953049/?all • Oculus prototype images: https://forums.oculus.com/community/discussion/7544/listing-of-all-oculus-rift- versions • Oculus protoype images: http://arstechnica.com/gaming/2012/09/virtual-realitys-time-to-shine-hands-on- with-the-oculus-rift/ • VR Icons via Noun Project, designers names on images themselves • Matrix images: http://www.nowverybad.com/the-matrix-1999/
175. References • Visual Vocab by Jesse James Garret http://www.jjg.net/ia/visvocab/ • World of Warcraft screenshot: BIT.LY/1KKJ6KT • GUIDEDMEDITATIONVR.COM • Airmech gameplay video: https://www.youtube.com/watch?v=8r6jI9Xp0fY • VRDC Fantastic Contraption ”Menus Suck” by Colin Northway: http://www.gdcvault.com/play/1023668/Menus • About Fantastic Contraption Trailers: http://vrscout.com/projects/fantastic-contraption-htc-vive-mixed- reality/ • Fantastic Contraption playthrough https://www.youtube.com/watch?v=FG7r2D02Qz0 • Teleporting Interactions: http://www.gamasutra.com/view/news/266687/Do_the_locomotion_How_VR_devs_are_expanding_the _virtual_space.php • eVRydayVR CV1 Oculus Home Walkthrough: https://www.youtube.com/watch?v=emO-eUL2YP4 • VR Interface Design Pre-Visualisation Methods https://www.youtube.com/watch?v=id86HeV-Vb8