Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Space, the Final Frontier: Designing Interactions Beyond the Screen

John Ryan
February 06, 2017

Space, the Final Frontier: Designing Interactions Beyond the Screen

How can ‘real’ space pull its interactive weight in the era of ubiquitous mobile computing?

Local Projects is an interactive design studio whose work has been bleeding off screens (and between disciplines) into objects and architecture during the past few years: houses that come to life with the story of their inhabitants, augmented-reality characters who guide you around museums, and physical DNA building blocks that children can use to create new lifeforms.

Eric and John, who lead technical and design teams at the studio, will share some challenges, triumphs, and insights into embedding interactions in physical systems:

— Examining the ‘skeuomorphic-cycle’: How physical affordances inspired digital paradigms which are now feeding back into physical interactions.
— Exploring new boundaries: How do we create seamless ‘interfaces’ between the digital and the physical?
— Identifying opportunities: What new experiences are possible as devices become increasingly spatially-aware, and sense a user’s presence, state, or emotion?

Logistically, creating these kinds of interactions — even in low quantities — is easier and cheaper than it has ever been, but the freedom this entails amplifies design challenges. This talk equips you with some approaches and paradigms as you step into this exciting new realm for interaction design.

- - - - - - -

Presented with Eric Mika at Interaction17, NYC: http://interaction17.ixda.org/session/space-final-frontier-designing-interactions-beyond-screen/

John Ryan

February 06, 2017
Tweet

More Decks by John Ryan

Other Decks in Design

Transcript

  1. Space, the Final Frontier
    Designing Interactions Beyond the Screen
    John Ryan & Eric Mika, Local Projects

    View full-size slide

  2. How do we create seamless
    ‘interfaces’ between the digital
    and the physical?

    View full-size slide

  3. LOCAL PROJECTS 3
    A JOURNEY INTO SPACE

    View full-size slide

  4. A JOURNEY INTO SPACE
    Local Projects


    5 Chapters Takeaways

    The Future
    $%
    &'

    View full-size slide

  5. LOCAL PROJECTS 6

    View full-size slide

  6. LOCAL PROJECTS 7

    View full-size slide

  7. LOCAL PROJECTS 8

    View full-size slide

  8. Technology in service
    of human stories and emotion.

    View full-size slide

  9. Our tools are trapped
    inside tiny rectangles.
    — Brett Victor

    View full-size slide

  10. LOCAL PROJECTS 19

    View full-size slide

  11. LOCAL PROJECTS 20

    View full-size slide

  12. LOCAL PROJECTS 22

    View full-size slide

  13. Why space? Custom interactions at scale

    View full-size slide

  14. LOCAL PROJECTS 24

    View full-size slide

  15. LOCAL PROJECTS 25

    View full-size slide

  16. LOCAL PROJECTS 26

    View full-size slide

  17. LOCAL PROJECTS 27

    View full-size slide

  18. LOCAL PROJECTS 28

    View full-size slide

  19. LOCAL PROJECTS 29

    View full-size slide

  20. Why space? Custom interactions at scale
    Shared experiences

    View full-size slide

  21. LOCAL PROJECTS 31
    Dragan Ilic

    View full-size slide

  22. The environment is part
    of our cognitive system

    View full-size slide

  23. People lean slightly forward while
    thinking about the future.
    People lean slightly back

    when thinking about the past.

    View full-size slide

  24. Why space? Custom interactions at scale
    Shared experiences
    Embodied cognition

    View full-size slide

  25. A JOURNEY INTO SPACE

    View full-size slide

  26. I. ADDING CONTEXT
    II. INTERACTION BEYOND SCREENS
    III. TANGIBLE INTERACTIONS
    IV. IMMERSIVE ENVIRONMENTS
    V. PROTOTYPE & POSSIBILITIES

    View full-size slide

  27. CHAPTER I
    ADDING
    CONTEXT

    View full-size slide

  28. LOCAL PROJECTS 41

    View full-size slide

  29. LOCAL PROJECTS 43

    View full-size slide

  30. Tech Museum:
    Body Metrics
    CHAPTER I — ADDING CONTEXT

    View full-size slide


  31. Spatialize data flow across devices

    View full-size slide

  32. CHAPTER II
    INTERACTION

    BEYOND SCREENS

    View full-size slide




  33. INTERACTION
    SCREEN
    USER INTERFACE

    View full-size slide




  34. INTERACTION
    SCREEN
    USER INTERFACE

    View full-size slide

  35. Cleveland Museum of Art
    CHAPTER II — INTERACTION BEYOND SCREENS

    View full-size slide


  36. Leverage how we already interact

    View full-size slide

  37. ARoS Aarhus Art Museum
    CHAPTER II — INTERACTION BEYOND SCREENS

    View full-size slide

  38. LOCAL PROJECTS 58

    View full-size slide


  39. Reveal the perceptual process

    View full-size slide

  40. One World Observatory
    CHAPTER II — INTERACTION BEYOND SCREENS

    View full-size slide

  41. LOCAL PROJECTS 63

    View full-size slide


  42. Align feedback with locus of interaction

    View full-size slide

  43. CHAPTER III
    TANGIBLE
    INTERACTIONS

    View full-size slide




  44. SCREEN
    USER INTERFACE

    INTERACTION

    View full-size slide




  45. INTERACTION
    USER INTERFACE

    SCREEN

    View full-size slide

  46. Cooper Hewitt
    National Design Museum
    CHAPTER III — TANGIBLE INTERACTIONS

    View full-size slide


  47. Atoms are cheaper than ever

    View full-size slide

  48. Tech Museum:
    Synthetic Biology
    CHAPTER III — TANGIBLE INTERACTIONS

    View full-size slide

  49. LOCAL PROJECTS 77

    View full-size slide

  50. LOCAL PROJECTS 78

    View full-size slide

  51. LOCAL PROJECTS 80

    View full-size slide

  52. LOCAL PROJECTS 82

    View full-size slide


  53. Synchronicity and mirroring between
    tangible objects and digital world

    View full-size slide


  54. Balance complexity with forgiveness

    View full-size slide

  55. Skeuomorphic Cycle

    View full-size slide

  56. LOCAL PROJECTS 87
    Gabriele Meldaikyte

    View full-size slide

  57. LOCAL PROJECTS 88

    View full-size slide

  58. CHAPTER IV
    INTERACTIVE
    ENVIRONMENTS

    View full-size slide




  59. INTERACTION
    USER INTERFACE

    SCREEN

    View full-size slide

  60. Cooper Hewitt
    Immersion Room
    CHAPTER IV — INTERACTIVE ENVIRONMENTS

    View full-size slide


  61. Leverage small gestures
    for big payoff

    View full-size slide

  62. Target Open House
    CHAPTER IV — INTERACTIVE ENVIRONMENTS

    View full-size slide


  63. Stories help us understand
    complex systems

    View full-size slide

  64. REAL SPACE
    DIGITAL SPACE

    View full-size slide

  65. CHAPTER V
    PROTOTYPES &
    POSSIBILITIES
    COLLAPSING REAL SPACE & DIGITAL SPACE

    View full-size slide

  66. Technologies Spatial Tracking

    View full-size slide

  67. LOCAL PROJECTS 101
    Pozyx Indoor Positioning

    View full-size slide

  68. LOCAL PROJECTS 102

    View full-size slide

  69. FAITH & LIBERTY DISCOVERY CENTER | LOCAL PROJECTS
    103
    Change

    View full-size slide


  70. • Reimplement & reinterpret physics
    • Spatially-relative interactions

    View full-size slide

  71. Technologies Spatial Tracking
    Augmented Reality

    View full-size slide

  72. LOCAL PROJECTS 106

    View full-size slide

  73. LOCAL PROJECTS 107

    View full-size slide


  74. • Gaze-reactive environments
    • Google Maps down to the millimeter
    • Personalized real-world layer

    View full-size slide

  75. Technologies Spatial Tracking
    Augmented Reality
    Artificial Intelligence

    View full-size slide

  76. LOCAL PROJECTS 110

    View full-size slide

  77. EEG Headset
    Wide-area RFID Readers
    Brain Activity
    Location
    Muscle Activity
    Motion
    Images
    Sounds
    Heartrate
    Sensors Raw Data Semantic Extractions
    iPod Touch & RFID Tag
    Exhibit Interactive
    RFID Reader
    EXG Sensor
    Heart Rate Sensor
    Inferences
    Skin Resistance
    Interest
    Congeniality
    Buzz
    Energy Level
    Posture
    Mood
    Engagement
    Learning
    Focus
    Excitement
    Drowsiness
    Metabolism
    Gregariousness
    Popularity
    Attraction
    Anxiety
    Bustle
    Tonality
    Pace
    Restlessness
    Agitation
    Utterances
    Nearby Face Emotions
    Anger, Contempt, Disgust, Joy, Fear,
    Negative, Neutral, Positive, Sadness,
    Surprise, Confusion, Frustration
    Average Color
    Volume Level
    Step Counts
    Average Brightness
    Nearby Faces Seen
    Pitch Level
    Brainwave Levels
    "Attention", "Meditation",
    Alpha, theta, etc.
    Orientation
    Blinks
    Caloric Burn
    Latest interaction
    Location in Museum
    Tension
    Stress
    Exertion
    Distance from others
    Acceleration
    LOCAL PROJECTS 112

    View full-size slide

  78. LOCAL PROJECTS 113

    View full-size slide

  79. LOCAL PROJECTS 115

    View full-size slide


  80. • Dynamically learned spaces
    • Metadata of the mundane
    • Intuitively blending inputs

    View full-size slide

  81. I. ADDING CONTEXT
    II. INTERACTION BEYOND SCREENS
    III. TANGIBLE INTERACTIONS
    IV. IMMERSIVE ENVIRONMENTS
    V. PROTOTYPE & POSSIBILITIES

    View full-size slide

  82. Thank You
    John Ryan — @johndryan
    Eric Mika — @kitschpatrol
    www.localprojects.com

    View full-size slide

  83. 00
    QUESTIONS?
    COMMENTS?

    View full-size slide