Why XR Will Break All Rules

Extended Reality (the term used to collectively refer to Virtual, Augmented and Mixed Realities) is moving from an industry buzzword to a business opportunity, globally. Next year, it’s estimated over 1 billion people will be using AR. By 2021, worldwide XR spending will reach $140B. XR brings a new reality for humans. Designers and developers are at the forefront of shaping these profoundly new experiences. Designing for XR requires new considerations, new rules and integration with technologies such as AI, IoT and Blockchain. These design considerations extend beyond our collective experiences, challenging our conventional wisdom. It’s also an opportunity to be seen as icons of the XR generation.

Transcript

1. Why XR Will Break All Rules Nathan Lavertue Global Experience Design Director
2. As a designer, I naturally look at how I can positively impact the human experiences through technology and design. (tell story about how one of the first things I did with Dad was take apart of carburetor from a car)
3. And as an IBMer, I am constantly evaluating and exploring ways to drive business value and impact using design. As TJ Watson Jr so eloquently said in 1966, good design is good business.
4. For example… I was formally trained as an architectural designer — and this will become more relevant as I speak — and this is a quadrophonic headset prototype I developed over 20 years ago for my architectural thesis. The theory was if people could occupy a different sound scape, they would experience the physical scape in a new and different way. Through a sorted history which I’ll spare you, I’m now a business designer focused on digital experiences.
5. I like to think of myself as the White Rabbit. In Lewis Carroll’s book, Alice in Wonderland… the White Rabbit represents Alice’s curiosity and functions as her guide in Wonderland. Similarly, my purpose in everything I do is to create an experience which guides and enables humans conceived from my own curiosity.
6. XR is a technology I expect will greatly impact both the human experience and business.
7. Based on the vast monetary investments in XR from the likes of Microsoft….
8. …Bose….
9. …Google…
10. and Apple…
11. …to the numerous start-ups which seem to pop up on a weekly basis, there are many which also see a significant benefit and potential business opportunity.
12. $140,000,000,000 $9 $142 $0 $20 $40 $60 $80 $100 $120 $140 $160 2017 2021 One which, in fact, will spend $140B by 2021. That’s a 98.8% increase since 2017.
13. $1,000,000,000,000 Just a week ago I read a report from Alan Smithson who predicted by 2025 the market value will hit ONE TRILLION globally. That's a lot of zeros!
14. That’s pretty impressive, but what is XR doing for businesses now… ? Well, let’s take a look.
15. 34% increase in productivity for GE Renewable Energy employees wiring wind turbines GE has seen a 34% increase in productivity for their employees who are wiring wind turbines through the use of AR.
16. Boeing trainees outfitted with AR glasses completed work in 35% less time than those using traditional 2-D drawings Boeing’s trainees are learning and completing their work in 35% less time than it took using traditional drawings.
17. 32% of retailers expect to add virtual and augmented reality in the next three years In the next 3 years, 32% of retailers expect to add XR to their shopping experiences.
18. 41% of Millennial homebuyers made offers on properties they hadn't seen in person And, in this staggering number… 41% of millennials are making offers on homes before they even see them! Personally I wish this existed 2 years ago when I saw nearly 40-50 homes before I bought my home!
19. But, even with all this power and potential – XR is going to challenge what we think we know.
20. And today, I’m here to share my perspective on why I think XR will break all the rules in three specific areas.
21. Technological Harmony Moral Harmony Spatial Harmony Spatial Harmony – It’s going to challenge our understanding of interaction. Technological Harmony – It’s going to challenge our understanding of integration with data and technology. Moral Harmony – It’s going to challenge our understanding of being respectful of our users.
22. user interface In architectural design, we did not use the term user interface. But they exist.
23. A door knob is an interface.
24. A window is an interface.
25. The walls of a room define the extent of where the user can interface within.
26. In architecture, you typically utilize the full range of your senses to experience and engage. (to one extent or another). In the digital landscape, though, we are not yet as mature.
27. In 1968, the mouse was publicly released. This allowed a user to interface with a software-based cursor through a hardware-based screen. By clicking, we could engage in an action. We could interact. But this interaction was nothing like what we experienced in the physical word. It was foreign.
28. As technology evolved, we became portable. Released in 1994, through a partnership with IBM and BellSouth, we saw the first smartphone. We had the ability to carry screens with us. With these, there was a need to interact in a new way. One slightly closer to what we experienced in the physical world. We could touch. But we couldn’t feel.
29. And, we learned gestures weren’t just about letting someone know how you felt about them! (middle finger)
30. More recently, we’ve ditched the need to carry technology but keep it parked in the corner of the room for us to interact with. We can talk with our technology. And it talks back to us. (to the constant bemusement to my 7 year old niece!) While this is more natural, and slightly more human, it’s only one facet of how we as humans interact with the world.
31. The space where interaction occurs between humans and machines. XR presents us with the opportunity to reimagine interactivity. During my time studying architecture, I spend a great deal of time exploring space.
32. https://images.nasa.gov/ No, not deep space.
33. But the space we exist in. The space we experience. And, more importantly, how we experience it.
34. We use are senses to learn, adapt and interact with our physical world.
35. How will we use all of our senses? How will we see, touch, feel, hear, and (sometimes!) taste digital experiences.
36. redefine With XR, we need to evolve our thinking of what it means to interact or interface with technology through digital landscapes.
37. We need to see the body as an interface. With this, comes new learning and responsibilities.
38. proxemics We’ll start talking about proxemics.
39. https://en.wikipedia.org/wiki/Proxemics Proxemics is the study of personal space. And, as you can imagine, varies from culture to culture. Intimate. Personal. Social. And public.
40. What does it mean to be occupying these spaces in a virtual environment with a group or own your own? What does it mean to coexist in a social physical space and digital intimate space?
41. kinesthetics We’ll start talking about kinesthetic motion and the individual kinesphere.
42. Proxemics describes the physical space we occupy, while the kinesphere describes how we move through it. Through pathways and levels. Coincidently this concept was developed by an architectural student turned to choreography.
43. In some cases, these terms are already being used in XR field guides… this is a guide from 2017 which discusses VR and the kinesphere.
44. dimensionality We’ll talk more about dimensionality.
45. Rab − 1 2 Rgab = 8πG c4 Tab I’m not asking you to understand Einstein’s theory of relativity.
46. But, how does something exist across space, as well as time?
47. feedback response And, lastly, the feedback response.
48. No longer can we rely on only visual cues to guide humans. Or indicate a successful or failed task. If you need someone to turn left, do you vibrate the handheld, provide an audio cue or flash of light on the left? It’s also important to know audio inputs hits our brain before visual input does. 500-Channel “Wave Field Synthesis” Audio System from RPI Experimental Media and Performing Arts Center - 5cm apart
49. We will need to consider how to tap into ALL of these senses in XR environments. C2 Primal Experience from 2018. They provided candy to eat while you were in the story.
50. We’ve discussed rethinking the relationship between space and humans. Now we need to rethink the current, practical applications of nascent and established technologies with XR.
51. How can XR coexist within and transform our current digital ecosystem? To guarantee the success of XR, and to truly impact the human experience and business challenges, we need to understand the role of XR in the digital ecosystem. There are four key functional categories you can consider applicable areas for XR….
52. explain train create engage Explain. Train. Create. Engage.
53. Explain – content and experiences to support more efficient or effective execution of an activity.
54. Train – content and experiences meant for the purpose of learning and development.
55. Create – content and experiences to support the conceptualizing and developing of objects and environments.
56. Engage – content and experiences to attract and interact with customers.
57. Understanding how to leverage XR, we can take a closer look at how XR provides the means for technologies to enter the virtual environments.
58. Using another architectural analogy, think of XR as the wooden frame of a house. It’s the environment we build upon and within. It gives these technologies a home. A digital means to co-exist in a meaningful way.
59. Cloud would be the utilities. It provides the necessary power to function. It’s also where the environment will technically live.
60. IoT would be the WiFi. While it’s nice to stay at home. It’s also nice to connect to the outside world and feel a part of something greater. It allows for an interconnectivity among the ecosystems. Making them greater.
61. Cognitive is the family. It’s the people who create a sense of home. A trusted environment which recognizes me, knows me and aids me.
62. https://en.wikipedia.org/wiki/Blockchain Blockchain in the neighborhood. It’s the people you work with to create a trusted network of counterparts and structure.
63. Ok. That’s a lot. We’ve covered how XR will impact both spatial and technological harmony. It’s a lot to take in. But, there’s more.
64. As an empathetic designer, it’s my duty to encourage ALL of you to be experts in human experience design – regardless of your role, you have the power to determine how XR is seen by the public. We spoke about spatial harmony, and technological harmony. The last is moral harmony.
65. In an era when Facebook is having to go through its first business transformation since its inception, it’s putting security, transparency and privacy at the forefront of its digital transformation. It’s going back to the core of their original ethos. A trusted platform to safely interact with friends.
66. Beyond malware, bots, etc. the collective we will need to be responsible for designing experiences which are morally sound going forward… one could argue this is true beyond just designing for XR.
67. hyper-reality.co This demonstrates the potential negative experience we could craft. HYPER REALITY by Keiichi Matsuda — a dramatic dystopian look at how AR could be used.
68. We need to ensure safety for people when using XR. It’s not enough to create a set of safety guidelines, but truly create safe places to digital experience. How do you avoid putting people in physical danger? How will you you ensure people are not being surveilled during downtime?
69. How will you protect an individuals personal information? Or, any information gathered will be used for the power good and not evil? If you are capturing biometric data, how is it being used? Do you absolutely need to access this information? How will you prevent someone being impersonated? Or, mind-hacking — the theory of reprogramming how someone thinks, without their consent?
70. How will you be transparent to your users. It’s one thing to create a safe environment, which is judicially accessing their personal information… but, how will you communicate it to them? How do you communicate you are tracking their eye movement, heart rate or other biometric data?
71. Many people believe, and I do as well, XR has the ability to turn humans into super humans. But, what about being better humans?
72. Because, good design is also responsible. I want to show three real use cases of how XR is being used to truly transform humans.
73. Samsung Relumino — light sensitive glasses, a visual aid application for the people with low vision. Relúmĭno will be the life-changer for 240 million of the visually impaired people around the world.
74. Limbix — mental health platform The Limbix VR Kit is medical grade, wireless, and doesn't require internet. Simply turn it on, and you're in VR. Teleport patients around the world to the scene of an important life event or take them to tranquil nature environments where they can relax. Therapists can gradually expose patients to phobias or sources of distress to increase a patient's anxiety tolerance to distressing situations.
75. Clouds over Sidra. Follows a 12 yo Syrian refugee. Funded by the United Nations to help it’s philanthropic efforts. The Zaatari Refugee Camp is home to over 80,000 Syrians fleeing violence and war. Children make up half the camp's population. This is the story of Sidra, a 12-year old girl who has spent the last 18 months in Zaatari. A 2015 fundraising conference where Clouds Over Sidra was shown ended up raising $3.8 billion, over 70 percent more than projected.
76. But… you’re all super humans.
77. And you shouldn’t listen to me.
78. Ok, well, don’t listen to JUST me. I am one person on my own path and see massive potential for XR. And am anxious to see what you, the future icons of the XR generation, can do to dramatically change the way XR shapes our world.
79. Experiment. Tinker. Deconstruct. Prototype. Discuss.
80. Don’t be afraid to break the rules, because, frankly… there are none.