Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Space, the Final Frontier: Designing Interactions Beyond the Screen

John Ryan
February 06, 2017

Space, the Final Frontier: Designing Interactions Beyond the Screen

How can ‘real’ space pull its interactive weight in the era of ubiquitous mobile computing?

Local Projects is an interactive design studio whose work has been bleeding off screens (and between disciplines) into objects and architecture during the past few years: houses that come to life with the story of their inhabitants, augmented-reality characters who guide you around museums, and physical DNA building blocks that children can use to create new lifeforms.

Eric and John, who lead technical and design teams at the studio, will share some challenges, triumphs, and insights into embedding interactions in physical systems:

— Examining the ‘skeuomorphic-cycle’: How physical affordances inspired digital paradigms which are now feeding back into physical interactions.
— Exploring new boundaries: How do we create seamless ‘interfaces’ between the digital and the physical?
— Identifying opportunities: What new experiences are possible as devices become increasingly spatially-aware, and sense a user’s presence, state, or emotion?

Logistically, creating these kinds of interactions — even in low quantities — is easier and cheaper than it has ever been, but the freedom this entails amplifies design challenges. This talk equips you with some approaches and paradigms as you step into this exciting new realm for interaction design.

- - - - - - -

Presented with Eric Mika at Interaction17, NYC: http://interaction17.ixda.org/session/space-final-frontier-designing-interactions-beyond-screen/

John Ryan

February 06, 2017
Tweet

More Decks by John Ryan

Other Decks in Design

Transcript

  1. Space, the Final Frontier
    Designing Interactions Beyond the Screen
    John Ryan & Eric Mika, Local Projects

    View Slide

  2. How do we create seamless
    ‘interfaces’ between the digital
    and the physical?

    View Slide

  3. LOCAL PROJECTS 3
    A JOURNEY INTO SPACE

    View Slide

  4. A JOURNEY INTO SPACE
    Local Projects


    5 Chapters Takeaways

    The Future
    $%
    &'

    View Slide

  5. $%
    &'

    View Slide

  6. LOCAL PROJECTS 6

    View Slide

  7. LOCAL PROJECTS 7

    View Slide

  8. LOCAL PROJECTS 8

    View Slide

  9. View Slide

  10. Technology in service
    of human stories and emotion.

    View Slide

  11. Our tools are trapped
    inside tiny rectangles.
    — Brett Victor

    View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. View Slide

  18. Why space?

    View Slide

  19. LOCAL PROJECTS 19

    View Slide

  20. LOCAL PROJECTS 20

    View Slide

  21. View Slide

  22. LOCAL PROJECTS 22

    View Slide

  23. Why space? Custom interactions at scale

    View Slide

  24. LOCAL PROJECTS 24

    View Slide

  25. LOCAL PROJECTS 25

    View Slide

  26. LOCAL PROJECTS 26

    View Slide

  27. LOCAL PROJECTS 27

    View Slide

  28. LOCAL PROJECTS 28

    View Slide

  29. LOCAL PROJECTS 29

    View Slide

  30. Why space? Custom interactions at scale
    Shared experiences

    View Slide

  31. LOCAL PROJECTS 31
    Dragan Ilic

    View Slide

  32. The environment is part
    of our cognitive system

    View Slide

  33. People lean slightly forward while
    thinking about the future.
    People lean slightly back

    when thinking about the past.

    View Slide

  34. Why space? Custom interactions at scale
    Shared experiences
    Embodied cognition

    View Slide

  35. A JOURNEY INTO SPACE

    View Slide

  36. I. ADDING CONTEXT
    II. INTERACTION BEYOND SCREENS
    III. TANGIBLE INTERACTIONS
    IV. IMMERSIVE ENVIRONMENTS
    V. PROTOTYPE & POSSIBILITIES

    View Slide

  37. CHAPTER I
    ADDING
    CONTEXT

    View Slide

  38. View Slide

  39. View Slide

  40. View Slide

  41. LOCAL PROJECTS 41

    View Slide

  42. View Slide

  43. LOCAL PROJECTS 43

    View Slide

  44. Tech Museum:
    Body Metrics
    CHAPTER I — ADDING CONTEXT

    View Slide

  45. View Slide


  46. Spatialize data flow across devices

    View Slide

  47. CHAPTER II
    INTERACTION

    BEYOND SCREENS

    View Slide

  48. View Slide

  49. View Slide




  50. INTERACTION
    SCREEN
    USER INTERFACE

    View Slide




  51. INTERACTION
    SCREEN
    USER INTERFACE

    View Slide

  52. Cleveland Museum of Art
    CHAPTER II — INTERACTION BEYOND SCREENS

    View Slide

  53. View Slide

  54. View Slide

  55. View Slide


  56. Leverage how we already interact

    View Slide

  57. ARoS Aarhus Art Museum
    CHAPTER II — INTERACTION BEYOND SCREENS

    View Slide

  58. LOCAL PROJECTS 58

    View Slide


  59. Reveal the perceptual process

    View Slide

  60. One World Observatory
    CHAPTER II — INTERACTION BEYOND SCREENS

    View Slide

  61. View Slide

  62. View Slide

  63. LOCAL PROJECTS 63

    View Slide


  64. Align feedback with locus of interaction

    View Slide

  65. CHAPTER III
    TANGIBLE
    INTERACTIONS

    View Slide




  66. SCREEN
    USER INTERFACE

    INTERACTION

    View Slide




  67. INTERACTION
    USER INTERFACE

    SCREEN

    View Slide

  68. Cooper Hewitt
    National Design Museum
    CHAPTER III — TANGIBLE INTERACTIONS

    View Slide

  69. View Slide

  70. View Slide

  71. View Slide

  72. View Slide

  73. View Slide

  74. View Slide


  75. Atoms are cheaper than ever

    View Slide

  76. Tech Museum:
    Synthetic Biology
    CHAPTER III — TANGIBLE INTERACTIONS

    View Slide

  77. LOCAL PROJECTS 77

    View Slide

  78. LOCAL PROJECTS 78

    View Slide

  79. View Slide

  80. LOCAL PROJECTS 80

    View Slide

  81. View Slide

  82. LOCAL PROJECTS 82

    View Slide


  83. Synchronicity and mirroring between
    tangible objects and digital world

    View Slide


  84. Balance complexity with forgiveness

    View Slide

  85. Skeuomorphic Cycle

    View Slide


  86. View Slide

  87. LOCAL PROJECTS 87
    Gabriele Meldaikyte

    View Slide

  88. LOCAL PROJECTS 88

    View Slide

  89. CHAPTER IV
    INTERACTIVE
    ENVIRONMENTS

    View Slide




  90. INTERACTION
    USER INTERFACE

    SCREEN

    View Slide







  91. View Slide

  92. Cooper Hewitt
    Immersion Room
    CHAPTER IV — INTERACTIVE ENVIRONMENTS

    View Slide

  93. View Slide


  94. Leverage small gestures
    for big payoff

    View Slide

  95. Target Open House
    CHAPTER IV — INTERACTIVE ENVIRONMENTS

    View Slide

  96. View Slide


  97. Stories help us understand
    complex systems

    View Slide

  98. REAL SPACE
    DIGITAL SPACE

    View Slide

  99. CHAPTER V
    PROTOTYPES &
    POSSIBILITIES
    COLLAPSING REAL SPACE & DIGITAL SPACE

    View Slide

  100. Technologies Spatial Tracking

    View Slide

  101. LOCAL PROJECTS 101
    Pozyx Indoor Positioning

    View Slide

  102. LOCAL PROJECTS 102

    View Slide

  103. FAITH & LIBERTY DISCOVERY CENTER | LOCAL PROJECTS
    103
    Change

    View Slide


  104. • Reimplement & reinterpret physics
    • Spatially-relative interactions

    View Slide

  105. Technologies Spatial Tracking
    Augmented Reality

    View Slide

  106. LOCAL PROJECTS 106

    View Slide

  107. LOCAL PROJECTS 107

    View Slide


  108. • Gaze-reactive environments
    • Google Maps down to the millimeter
    • Personalized real-world layer

    View Slide

  109. Technologies Spatial Tracking
    Augmented Reality
    Artificial Intelligence

    View Slide

  110. LOCAL PROJECTS 110

    View Slide

  111. Bodymetrics

    View Slide

  112. EEG Headset
    Wide-area RFID Readers
    Brain Activity
    Location
    Muscle Activity
    Motion
    Images
    Sounds
    Heartrate
    Sensors Raw Data Semantic Extractions
    iPod Touch & RFID Tag
    Exhibit Interactive
    RFID Reader
    EXG Sensor
    Heart Rate Sensor
    Inferences
    Skin Resistance
    Interest
    Congeniality
    Buzz
    Energy Level
    Posture
    Mood
    Engagement
    Learning
    Focus
    Excitement
    Drowsiness
    Metabolism
    Gregariousness
    Popularity
    Attraction
    Anxiety
    Bustle
    Tonality
    Pace
    Restlessness
    Agitation
    Utterances
    Nearby Face Emotions
    Anger, Contempt, Disgust, Joy, Fear,
    Negative, Neutral, Positive, Sadness,
    Surprise, Confusion, Frustration
    Average Color
    Volume Level
    Step Counts
    Average Brightness
    Nearby Faces Seen
    Pitch Level
    Brainwave Levels
    "Attention", "Meditation",
    Alpha, theta, etc.
    Orientation
    Blinks
    Caloric Burn
    Latest interaction
    Location in Museum
    Tension
    Stress
    Exertion
    Distance from others
    Acceleration
    LOCAL PROJECTS 112

    View Slide

  113. LOCAL PROJECTS 113

    View Slide

  114. View Slide

  115. LOCAL PROJECTS 115

    View Slide


  116. • Dynamically learned spaces
    • Metadata of the mundane
    • Intuitively blending inputs

    View Slide

  117. View Slide

  118. View Slide

  119. I. ADDING CONTEXT
    II. INTERACTION BEYOND SCREENS
    III. TANGIBLE INTERACTIONS
    IV. IMMERSIVE ENVIRONMENTS
    V. PROTOTYPE & POSSIBILITIES

    View Slide

  120. Boldly Go

    View Slide

  121. Thank You
    John Ryan — @johndryan
    Eric Mika — @kitschpatrol
    www.localprojects.com

    View Slide

  122. 00
    QUESTIONS?
    COMMENTS?

    View Slide