Space, the Final Frontier: Designing Interactions Beyond the Screen

How can ‘real’ space pull its interactive weight in the era of ubiquitous mobile computing?

Local Projects is an interactive design studio whose work has been bleeding off screens (and between disciplines) into objects and architecture during the past few years: houses that come to life with the story of their inhabitants, augmented-reality characters who guide you around museums, and physical DNA building blocks that children can use to create new lifeforms.

Eric and John, who lead technical and design teams at the studio, will share some challenges, triumphs, and insights into embedding interactions in physical systems:

— Examining the ‘skeuomorphic-cycle’: How physical affordances inspired digital paradigms which are now feeding back into physical interactions.
— Exploring new boundaries: How do we create seamless ‘interfaces’ between the digital and the physical?
— Identifying opportunities: What new experiences are possible as devices become increasingly spatially-aware, and sense a user’s presence, state, or emotion?

Logistically, creating these kinds of interactions — even in low quantities — is easier and cheaper than it has ever been, but the freedom this entails amplifies design challenges. This talk equips you with some approaches and paradigms as you step into this exciting new realm for interaction design.

- - - - - - -

Presented with Eric Mika at Interaction17, NYC: http://interaction17.ixda.org/session/space-final-frontier-designing-interactions-beyond-screen/

Transcript

1. Space, the Final Frontier Designing Interactions Beyond the Screen John Ryan & Eric Mika, Local Projects
2. How do we create seamless ‘interfaces’ between the digital and the physical?
3. LOCAL PROJECTS 3 A JOURNEY INTO SPACE
4. A JOURNEY INTO SPACE Local Projects 5 Chapters Takeaways The Future $% &'
5. $% &'
6. LOCAL PROJECTS 6
7. LOCAL PROJECTS 7
8. LOCAL PROJECTS 8
10. Technology in service of human stories and emotion.
11. Our tools are trapped inside tiny rectangles. — Brett Victor
18. Why space?
19. LOCAL PROJECTS 19
20. LOCAL PROJECTS 20
22. LOCAL PROJECTS 22
23. Why space? Custom interactions at scale
24. LOCAL PROJECTS 24
25. LOCAL PROJECTS 25
26. LOCAL PROJECTS 26
27. LOCAL PROJECTS 27
28. LOCAL PROJECTS 28
29. LOCAL PROJECTS 29
30. Why space? Custom interactions at scale Shared experiences
31. LOCAL PROJECTS 31 Dragan Ilic
32. The environment is part of our cognitive system
33. People lean slightly forward while thinking about the future. People lean slightly back
 when thinking about the past.
34. Why space? Custom interactions at scale Shared experiences Embodied cognition
35. A JOURNEY INTO SPACE
36. I. ADDING CONTEXT II. INTERACTION BEYOND SCREENS III. TANGIBLE INTERACTIONS IV. IMMERSIVE ENVIRONMENTS V. PROTOTYPE & POSSIBILITIES
37. CHAPTER I ADDING CONTEXT
41. LOCAL PROJECTS 41
43. LOCAL PROJECTS 43
44. Tech Museum: Body Metrics CHAPTER I — ADDING CONTEXT
46. Spatialize data flow across devices
47. CHAPTER II INTERACTION
 BEYOND SCREENS
50. INTERACTION SCREEN USER INTERFACE ✋
51. INTERACTION SCREEN USER INTERFACE ✋
52. Cleveland Museum of Art CHAPTER II — INTERACTION BEYOND SCREENS
56. Leverage how we already interact
57. ARoS Aarhus Art Museum CHAPTER II — INTERACTION BEYOND SCREENS
58. LOCAL PROJECTS 58
59. Reveal the perceptual process
60. One World Observatory CHAPTER II — INTERACTION BEYOND SCREENS
63. LOCAL PROJECTS 63
64. Align feedback with locus of interaction
65. CHAPTER III TANGIBLE INTERACTIONS
66. SCREEN USER INTERFACE ✋ INTERACTION
67. INTERACTION USER INTERFACE ✋ SCREEN
68. Cooper Hewitt National Design Museum CHAPTER III — TANGIBLE INTERACTIONS
75. Atoms are cheaper than ever
76. Tech Museum: Synthetic Biology CHAPTER III — TANGIBLE INTERACTIONS
77. LOCAL PROJECTS 77
78. LOCAL PROJECTS 78
80. LOCAL PROJECTS 80
82. LOCAL PROJECTS 82
83. Synchronicity and mirroring between tangible objects and digital world
84. Balance complexity with forgiveness
85. Skeuomorphic Cycle
86. ☎
87. LOCAL PROJECTS 87 Gabriele Meldaikyte
88. LOCAL PROJECTS 88
89. CHAPTER IV INTERACTIVE ENVIRONMENTS
90. INTERACTION USER INTERFACE ✋ SCREEN
91. ✋
92. Cooper Hewitt Immersion Room CHAPTER IV — INTERACTIVE ENVIRONMENTS
94. Leverage small gestures for big payoff
95. Target Open House CHAPTER IV — INTERACTIVE ENVIRONMENTS
97. Stories help us understand complex systems
98. REAL SPACE DIGITAL SPACE
99. CHAPTER V PROTOTYPES & POSSIBILITIES COLLAPSING REAL SPACE & DIGITAL SPACE
100. Technologies Spatial Tracking
101. LOCAL PROJECTS 101 Pozyx Indoor Positioning
102. LOCAL PROJECTS 102
103. FAITH & LIBERTY DISCOVERY CENTER | LOCAL PROJECTS 103 Change
104. • Reimplement & reinterpret physics • Spatially-relative interactions
105. Technologies Spatial Tracking Augmented Reality
106. LOCAL PROJECTS 106
107. LOCAL PROJECTS 107
108. • Gaze-reactive environments • Google Maps down to the millimeter • Personalized real-world layer
109. Technologies Spatial Tracking Augmented Reality Artificial Intelligence
110. LOCAL PROJECTS 110
111. Bodymetrics
112. EEG Headset Wide-area RFID Readers Brain Activity Location Muscle Activity Motion Images Sounds Heartrate Sensors Raw Data Semantic Extractions iPod Touch & RFID Tag Exhibit Interactive RFID Reader EXG Sensor Heart Rate Sensor Inferences Skin Resistance Interest Congeniality Buzz Energy Level Posture Mood Engagement Learning Focus Excitement Drowsiness Metabolism Gregariousness Popularity Attraction Anxiety Bustle Tonality Pace Restlessness Agitation Utterances Nearby Face Emotions Anger, Contempt, Disgust, Joy, Fear, Negative, Neutral, Positive, Sadness, Surprise, Confusion, Frustration Average Color Volume Level Step Counts Average Brightness Nearby Faces Seen Pitch Level Brainwave Levels "Attention", "Meditation", Alpha, theta, etc. Orientation Blinks Caloric Burn Latest interaction Location in Museum Tension Stress Exertion Distance from others Acceleration LOCAL PROJECTS 112
113. LOCAL PROJECTS 113
115. LOCAL PROJECTS 115
116. • Dynamically learned spaces • Metadata of the mundane • Intuitively blending inputs
119. I. ADDING CONTEXT II. INTERACTION BEYOND SCREENS III. TANGIBLE INTERACTIONS IV. IMMERSIVE ENVIRONMENTS V. PROTOTYPE & POSSIBILITIES
120. Boldly Go
121. Thank You John Ryan — @johndryan Eric Mika — @kitschpatrol www.localprojects.com
122. 00 QUESTIONS? COMMENTS?