Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Adaptive Input — UXLX

Adaptive Input — UXLX

Slides from the presentation at UX Lisbon

Jason Grigsby

June 06, 2014
Tweet

More Decks by Jason Grigsby

Other Decks in Technology

Transcript

  1. Adaptive Input
    Jason Grigsby • @grigs • cloudfour.com

    View Slide

  2. Follow along at @grigs_talks

    View Slide

  3. http://www.flickr.com/photos/cdm/51747860/

    View Slide

  4. http://www.flickr.com/photos/rheaney/4397863376
    It started with TVs.

    View Slide

  5. Designing for a 10-foot UI is very different.
    http://www.flickr.com/photos/chrisbartow/5835428673

    View Slide

  6. Larger text and fewer words.

    View Slide

  7. Make up, down, left, right directions clear.
    http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg

    View Slide

  8. How do we know what is a TV?

    View Slide

  9. This is HDTV.

    View Slide

  10. This is HDTV.
    1980 px
    1080 px

    View Slide

  11. Resolution does not
    define the optimal
    experience.

    View Slide

  12. Next came responsive web apps.
    https://twitter.com/freediverx/status/354698695041744896

    View Slide

  13. Responsive design for apps is inevitable.
    http://blog.cloudfour.com/responsive-design-for-apps-part-1/

    View Slide

  14. Lines between device classes are blurring
    Model Type Size
    Size Display Resolution
    Resolution Viewport
    Viewport
    W H W H W H
    Samsung Galaxy Note 2 Phone 3.17” 5.95” 5.5” 720 1280 360 640
    Motorola RAZR HD Phone 2.67” 5.19” 4.7” 720 1280 360 519
    Motorola Atrix HD Phone 2.75” 5.26” 4.5” 720 1280 540 812
    HTC Droid DNA Phone 2.78” 5.5” 5” 1080 1920 360 640
    Nexus 7 Tablet 4.72” 7.81” 7” 800 1280 600 793
    Kindle Fire Tablet 4.72” 7.44” 7” 600 1024 600 819
    Kindle Fire HD Tablet 5.4” 7.6” 7” 800 1280 533 731

    View Slide

  15. 640 px
    600 px
    519 px
    640 px
    622 px
    533 px
    812 px
    Which are phones and which are tablets?

    View Slide

  16. http://www.flickr.com/photos/geatchy/8489505999

    View Slide

  17. How do I make this responsive?
    How do I make this responsive?

    View Slide

  18. View Slide

  19. mobile
    desktop
    THE ART OF WEB DEVELOPMENT
    THE ART OF WEB DEVELOPMENT
    Web widgets
    THE ART OF WEB DEVELOPMENT
    THE ART OF WEB DEVELOPMENT
    Mobile widgets

    View Slide

  20. View Slide

  21. View Slide

  22. View Slide

  23. View Slide

  24. View Slide

  25. View Slide

  26. It’s not that we’re technically incapable, but adapting a phone UI to a tablet UI is
    not so dissimilar from trying to automatically adapt desktop UI to a phone. They
    are fundamentally different platforms with different usability
    considerations, and something that makes sense on phones may or may not
    belong on tablets.
    —Todd Anglin, Kendo UI
    http://www.kendoui.com/blogs/teamblog/posts/12-09-11/universal_mobile_apps_with_html5_and_kendo_ui.aspx

    View Slide

  27. Sometimes it’s hard to envision a responsive version.
    http://demos.kendoui.com/web/grid/editing.html

    View Slide

  28. http://www.flickr.com/photos/jesuspresley/384080245/
    We want people to be productive…

    View Slide

  29. and stay in the zone.
    http://www.flickr.com/photos/raccatography/8038855203

    View Slide

  30. http://www.flickr.com/photos/shantellmartin/4543010568
    Which seems very different from playing on an iPad.

    View Slide

  31. For both the TV…

    View Slide

  32. and the desktop web app…

    View Slide

  33. input matters much more than screen size.

    View Slide

  34. The grid is important to support d-pad interaction.
    http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg

    View Slide

  35. http://www.flickr.com/photos/royalsapien/2387707860

    View Slide

  36. And keyboard and mouse are what we envision work is.
    http://www.flickr.com/photos/royalsapien/2387707860

    View Slide

  37. http://www.flickr.com/photos/hellogeri/6154034099/
    A few years ago, Jeremy talked about how…

    View Slide

  38. http://www.flickr.com/photos/[email protected]/14301113/
    we told ourselves
    that the web was…

    View Slide

  39. http://www.flickr.com/photos/[email protected]/14301113/

    View Slide

  40. http://www.flickr.com/photos/[email protected]/14301113/
    640 px
    480 px

    View Slide

  41. 640 px
    480 px

    View Slide

  42. View Slide

  43. View Slide

  44. View Slide

  45. View Slide

  46. 1024 px
    768 px

    View Slide

  47. http://www.flickr.com/photos/adactio/6153481666/

    View Slide

  48. http://www.flickr.com/photos/adactio/6153481666/
    Then mobile came and made us realize…

    View Slide

  49. that it was a consensual hallucination all along.
    http://www.flickr.com/photos/garibaldi/303085857/

    View Slide

  50. The web never had a fixed canvas.
    http://www.flickr.com/photos/paulocarrillo/124755065/

    View Slide

  51. Even our tools perpetuate the lie.

    View Slide

  52. http://www.flickr.com/photos/[email protected]/7203485148/
    We’ve made tremendous progress.

    View Slide

  53. But there is another consensual hallucination.
    http://www.flickr.com/photos/garibaldi/
    303085857/

    View Slide

  54. View Slide

  55. =
    =

    View Slide

  56. View Slide

  57. Supports hover and pointer events.

    View Slide

  58. View Slide

  59. View Slide

  60. Keyboard and touch.

    View Slide

  61. View Slide

  62. View Slide

  63. Even the iPhone can have a keyboard.

    View Slide

  64. View Slide

  65. View Slide

  66. Are these laptops or tablets?

    View Slide

  67. View Slide

  68. View Slide

  69. Desktop computer with 23” touch screen

    View Slide

  70. View Slide

  71. Luke nailed it.
    http://static.lukew.com/unified_device_design.png

    View Slide

  72. View Slide

  73. We can no longer make assumptions about input
    based on screen size or form factor.
    And we probably never should have.

    View Slide

  74. http://www.flickr.com/photos/cblue98/7254221968

    View Slide

  75. Input represents a bigger challenge than screen size.
    http://www.flickr.com/photos/cblue98/7254221968

    View Slide

  76. http://www.flickr.com/photos/taedc/9278192929

    View Slide

  77. And it is changing more rapidly than ever before.
    http://www.flickr.com/photos/taedc/9278192929

    View Slide

  78. So let’s take a closer look…

    View Slide

  79. Let’s start with futuristic input.
    http://www.flickr.com/photos/jdhancock/3714748769/

    View Slide

  80. http://uncyclopedia.wikia.com/wiki/
    File:Man_yelling_at_computer.JPG
    VOICE

    View Slide

  81. http://uncyclopedia.wikia.com/wiki/
    File:Man_yelling_at_computer.JPG
    VOICE

    View Slide

  82. http://www.98ps.com/viewnews-15222.html

    View Slide

  83. Siri gets all of the hype…
    http://www.98ps.com/viewnews-15222.html

    View Slide

  84. but both Microsoft and Google have compelling
    voice input in their products.

    View Slide

  85. View Slide

  86. View Slide

  87. View Slide

  88. How should web pages change to support voice control?

    View Slide

  89. View Slide

  90. View Slide

  91. Google voice search

    View Slide

  92. You can use speech recognition too.
    http://www.google.com/intl/en/chrome/demos/speech.html
    http://www.moreawesomeweb.com/demos/speech_translate.html

    View Slide

  93. View Slide

  94. Gestures?

    View Slide

  95. http://leapmotion.com

    View Slide

  96. https://www.kickstarter.com/projects/1761670738/ring-shortcut-everything

    View Slide

  97. https://www.kickstarter.com/projects/1761670738/ring-shortcut-everything

    View Slide

  98. View Slide

  99. Amazing, but too new to know
    what, if anything, this technology
    will mean for the web.

    View Slide

  100. Let’s come back from the future and
    look at something much Dumber.
    Dumber

    View Slide

  101. Dumber

    View Slide

  102. Dumber

    View Slide

  103. -pad remote controls
    D

    View Slide

  104. -pad remote controls
    D

    View Slide

  105. -pad remote controls
    D

    View Slide

  106. http://www.flickr.com/photos/stewc/6669743035/

    View Slide

  107. http://www.flickr.com/photos/stewc/6669743035/
    TVs browsers that
    support d-pad, send
    arrow key events.

    View Slide

  108. http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/

    View Slide

  109. If then
    http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/

    View Slide

  110. View Slide

  111. is undetectable.
    This is a recurring theme for input.

    View Slide

  112. Sensors and camera
    http://www.flickr.com/photos/retrocactus/2170677056

    View Slide

  113. Sensors and camera
    Camera
    http://www.flickr.com/photos/retrocactus/2170677056

    View Slide

  114. GPS
    http://www.flickr.com/photos/3dking/149450434

    View Slide

  115. GPS
    GeoLocation
    http://www.flickr.com/photos/3dking/149450434

    View Slide

  116. View Slide

  117. Gyroscope & Accelerometer

    View Slide

  118. http://www.flickr.com/photos/chrisjagers/4694134078

    View Slide

  119. Back to today’s problems.
    http://www.flickr.com/photos/chrisjagers/4694134078

    View Slide

  120. View Slide

  121. Hover state No hover state

    View Slide

  122. Hover state
    Typing easier for many
    No hover state
    Typing often more difficult

    View Slide

  123. Higher precision with mouse means
    smaller targets possible
    Hover state
    Less precise than mouse and requires
    larger touch targets
    Typing easier for many
    No hover state
    Typing often more difficult

    View Slide

  124. Higher precision with mouse means
    smaller targets possible
    Hover state
    Less precise than mouse and requires
    larger touch targets
    Typing easier for many
    No hover state
    Typing often more difficult
    Right clicking and “power” tools Single and multi-touch gestures

    View Slide

  125. View Slide

  126. View Slide

  127. http://www.flickr.com/photos/[email protected]/5012309802

    View Slide

  128. I got this. Detect touch.
    http://www.flickr.com/photos/[email protected]/5012309802

    View Slide

  129. View Slide

  130. Whatever you may think, it currently isn't
    possible to reliably detect whether or not the
    current device has a touchscreen, from within the
    browser.
    —Stu Cox, You Can’t Reliably Detect a Touch Screen
    http://www.stucox.com/blog/you-cant-detect-a-touchscreen/

    View Slide

  131. Chrome has entertained idea of enabling touch by default.
    https://code.google.com/p/chromium/issues/detail?id=159527
    https://docs.google.com/a/cloudfour.com/presentation/d/1-n1qyzewpagREbzW2zm0wOalq33UhbtbSkWf9mEdly8/
    edit#slide=id.gc2d80e5b_171

    View Slide

  132. View Slide

  133. Detect a mouse? Not reliably.

    View Slide

  134. View Slide

  135. View Slide

  136. Surely we can detect a keyboard?

    View Slide

  137. Surely we can detect a keyboard?
    NOPE

    View Slide

  138. View Slide

  139. Input is dynamic.

    View Slide

  140. Input is dynamic.

    View Slide

  141. Boris Smus’s experiments responding to input.
    http://smus.com/touch-laptop-experiments/

    View Slide

  142. Boris Smus’s experiments responding to input.
    http://smus.com/touch-laptop-experiments/

    View Slide

  143. http://www.flickr.com/photos/lyza/7382235106
    Maybe we need to be more zen about input.

    View Slide

  144. View Slide

  145. After poking at this problem for a few weeks, my
    conclusion is: every desktop UI should be
    designed for touch now. When any desktop
    machine could have a touch interface, we have to
    proceed as if they all do.
    —Josh Clark
    http://globalmoxie.com/blog/desktop-touch-design.shtml

    View Slide

  146. View Slide

  147. What about those who won’t let
    go of their “power” interfaces?
    http://www.flickr.com/photos/ecos/4092571213/

    View Slide

  148. http://www.flickr.com/photos/scarygami/5689980135/
    One option: give them a choice.

    View Slide

  149. View Slide

  150. Gmail display density settings

    View Slide

  151. Th Dream Experience - …
    Uploaded 2 years ago
    More Of The Dream Exp…
    Uploaded 2 years ago
    The Dream Experience -…
    Uploaded 2 years ago
    The Dream Experience …
    Uploaded 2 years ago
    The Love King Breaks It…
    Uploaded 2 years ago
    War Paint for Trees
    From Lincoln Motor Company
    Join Log In Create Watch Upload Search s
    [ ]
    – VIDEOS
    Vimeo Couch Mode

    View Slide

  152. Couch Mode
    + See all
    Centric TV’s videos / Recently viewed / Related videos
    Th Dream Experience - …
    Uploaded 2 years ago
    More Of The Dream Exp…
    Uploaded 2 years ago
    The Dream Experience -…
    Uploaded 2 years ago
    The Dream Experience …
    Uploaded 2 years ago
    The Love King Breaks It…
    Uploaded 2 years ago
    PROMOTED
    War Paint for Trees
    From Lincoln Motor Company
    Join Log In Create Watch Upload Search s
    [ ]
    – VIDEOS
    Vimeo Couch Mode

    View Slide

  153. Couch Mode
    + See all
    Centric TV’s videos / Recently viewed / Related videos
    Th Dream Experience - …
    Uploaded 2 years ago
    More Of The Dream Exp…
    Uploaded 2 years ago
    The Dream Experience -…
    Uploaded 2 years ago
    The Dream Experience …
    Uploaded 2 years ago
    The Love King Breaks It…
    Uploaded 2 years ago
    PROMOTED
    War Paint for Trees
    From Lincoln Motor Company
    Join Log In Create Watch Upload Search s
    [ ]
    – VIDEOS
    Couch Mode
    + See all
    Centric TV’s videos / Recently viewed / Related videos
    Th Dream Experience - …
    Uploaded 2 years ago
    More Of The Dream Exp…
    Uploaded 2 years ago
    The Dream Experience -…
    Uploaded 2 years ago
    The Dream Experience …
    Uploaded 2 years ago
    The Love King Breaks It…
    Uploaded 2 years ago
    PROMOTED
    War Paint for Trees
    From Lincoln Motor Company
    Join Log In Create Watch Upload Search s
    [ ]
    – VIDEOS
    Vimeo Couch Mode

    View Slide

  154. Vimeo Couch Mode

    View Slide

  155. View Slide

  156. The key benefit of this approach:
    You’re designing for user need not
    for a specific form factor or input.

    View Slide

  157. http://www.flickr.com/photos/raver_mikey/504815463

    View Slide

  158. View Slide

  159. Progressive Input?

    View Slide

  160. Graph from Chapter 1 of Adaptive Web Design by
    Aaron Gustafson http://easy-readers.net/books/
    adaptive-web-design/

    View Slide

  161. Graph from Chapter 1 of Adaptive Web Design by
    Aaron Gustafson http://easy-readers.net/books/
    adaptive-web-design/
    Progressive enhancement contains a value judgment

    View Slide

  162. Who are we to judge what input is better?
    http://www.flickr.com/photos/fensterbme/4783366926

    View Slide

  163. We need to adapt
    and respond.
    http://www.flickr.com/photos/cdm/147947664/

    View Slide

  164. Thank You!
    Special thanks to Flickr users for generously sharing their
    photos under creative commons license.

    View Slide