Adaptive Input — UXLX

Slides from the presentation at UX Lisbon

Transcript

1. Adaptive Input
   Jason Grigsby • @grigs • cloudfour.com
   

   View Slide

2. Follow along at @grigs_talks
   

   View Slide

3. http://www.flickr.com/photos/cdm/51747860/
   

   View Slide

4. http://www.flickr.com/photos/rheaney/4397863376
   It started with TVs.
   

   View Slide

5. Designing for a 10-foot UI is very different.
   http://www.flickr.com/photos/chrisbartow/5835428673
   

   View Slide

6. Larger text and fewer words.
   

   View Slide

7. Make up, down, left, right directions clear.
   http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg
   

   View Slide

8. How do we know what is a TV?
   

   View Slide

9. This is HDTV.
   

   View Slide

10. This is HDTV.
    1980 px
    1080 px
    

    View Slide

11. Resolution does not
    define the optimal
    experience.
    

    View Slide

12. Next came responsive web apps.
    https://twitter.com/freediverx/status/354698695041744896
    

    View Slide

13. Responsive design for apps is inevitable.
    http://blog.cloudfour.com/responsive-design-for-apps-part-1/
    

    View Slide

14. Lines between device classes are blurring
    Model Type Size
    Size Display Resolution
    Resolution Viewport
    Viewport
    W H W H W H
    Samsung Galaxy Note 2 Phone 3.17” 5.95” 5.5” 720 1280 360 640
    Motorola RAZR HD Phone 2.67” 5.19” 4.7” 720 1280 360 519
    Motorola Atrix HD Phone 2.75” 5.26” 4.5” 720 1280 540 812
    HTC Droid DNA Phone 2.78” 5.5” 5” 1080 1920 360 640
    Nexus 7 Tablet 4.72” 7.81” 7” 800 1280 600 793
    Kindle Fire Tablet 4.72” 7.44” 7” 600 1024 600 819
    Kindle Fire HD Tablet 5.4” 7.6” 7” 800 1280 533 731
    

    View Slide

15. 640 px
    600 px
    519 px
    640 px
    622 px
    533 px
    812 px
    Which are phones and which are tablets?
    

    View Slide

16. http://www.flickr.com/photos/geatchy/8489505999
    

    View Slide

17. How do I make this responsive?
    How do I make this responsive?
    

    View Slide

18. View Slide

19. mobile
    desktop
    THE ART OF WEB DEVELOPMENT
    THE ART OF WEB DEVELOPMENT
    Web widgets
    THE ART OF WEB DEVELOPMENT
    THE ART OF WEB DEVELOPMENT
    Mobile widgets
    

    View Slide

20. View Slide

21. View Slide

22. View Slide

23. View Slide

24. View Slide

25. View Slide

26. It’s not that we’re technically incapable, but adapting a phone UI to a tablet UI is
    not so dissimilar from trying to automatically adapt desktop UI to a phone. They
    are fundamentally different platforms with different usability
    considerations, and something that makes sense on phones may or may not
    belong on tablets.
    —Todd Anglin, Kendo UI
    http://www.kendoui.com/blogs/teamblog/posts/12-09-11/universal_mobile_apps_with_html5_and_kendo_ui.aspx
    

    View Slide

27. Sometimes it’s hard to envision a responsive version.
    http://demos.kendoui.com/web/grid/editing.html
    

    View Slide

28. http://www.flickr.com/photos/jesuspresley/384080245/
    We want people to be productive…
    

    View Slide

29. and stay in the zone.
    http://www.flickr.com/photos/raccatography/8038855203
    

    View Slide

30. http://www.flickr.com/photos/shantellmartin/4543010568
    Which seems very different from playing on an iPad.
    

    View Slide

31. For both the TV…
    

    View Slide

32. and the desktop web app…
    

    View Slide

33. input matters much more than screen size.
    

    View Slide

34. The grid is important to support d-pad interaction.
    http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg
    

    View Slide

35. http://www.flickr.com/photos/royalsapien/2387707860
    

    View Slide

36. And keyboard and mouse are what we envision work is.
    http://www.flickr.com/photos/royalsapien/2387707860
    

    View Slide

37. http://www.flickr.com/photos/hellogeri/6154034099/
    A few years ago, Jeremy talked about how…
    

    View Slide

38. http://www.flickr.com/photos/60415054@N00/14301113/
    we told ourselves
    that the web was…
    

    View Slide

39. http://www.flickr.com/photos/60415054@N00/14301113/
    

    View Slide

40. http://www.flickr.com/photos/60415054@N00/14301113/
    640 px
    480 px
    

    View Slide

41. 640 px
    480 px
    

    View Slide

42. View Slide

43. View Slide

44. View Slide

45. View Slide

46. 1024 px
    768 px
    

    View Slide

47. http://www.flickr.com/photos/adactio/6153481666/
    

    View Slide

48. http://www.flickr.com/photos/adactio/6153481666/
    Then mobile came and made us realize…
    

    View Slide

49. that it was a consensual hallucination all along.
    http://www.flickr.com/photos/garibaldi/303085857/
    

    View Slide

50. The web never had a fixed canvas.
    http://www.flickr.com/photos/paulocarrillo/124755065/
    

    View Slide

51. Even our tools perpetuate the lie.
    

    View Slide

52. http://www.flickr.com/photos/69797234@N06/7203485148/
    We’ve made tremendous progress.
    

    View Slide

53. But there is another consensual hallucination.
    http://www.flickr.com/photos/garibaldi/
    303085857/
    

    View Slide

54. View Slide

55. =
    =
    

    View Slide

56. View Slide

57. Supports hover and pointer events.
    

    View Slide

58. View Slide

59. View Slide

60. Keyboard and touch.
    

    View Slide

61. View Slide

62. View Slide

63. Even the iPhone can have a keyboard.
    

    View Slide

64. View Slide

65. View Slide

66. Are these laptops or tablets?
    

    View Slide

67. View Slide

68. View Slide

69. Desktop computer with 23” touch screen
    

    View Slide

70. View Slide

71. Luke nailed it.
    http://static.lukew.com/unified_device_design.png
    

    View Slide

72. View Slide

73. We can no longer make assumptions about input
    based on screen size or form factor.
    And we probably never should have.
    

    View Slide

74. http://www.flickr.com/photos/cblue98/7254221968
    

    View Slide

75. Input represents a bigger challenge than screen size.
    http://www.flickr.com/photos/cblue98/7254221968
    

    View Slide

76. http://www.flickr.com/photos/taedc/9278192929
    

    View Slide

77. And it is changing more rapidly than ever before.
    http://www.flickr.com/photos/taedc/9278192929
    

    View Slide

78. So let’s take a closer look…
    

    View Slide

79. Let’s start with futuristic input.
    http://www.flickr.com/photos/jdhancock/3714748769/
    

    View Slide

80. http://uncyclopedia.wikia.com/wiki/
    File:Man_yelling_at_computer.JPG
    VOICE
    

    View Slide

81. http://uncyclopedia.wikia.com/wiki/
    File:Man_yelling_at_computer.JPG
    VOICE
    

    View Slide

82. http://www.98ps.com/viewnews-15222.html
    

    View Slide

83. Siri gets all of the hype…
    http://www.98ps.com/viewnews-15222.html
    

    View Slide

84. but both Microsoft and Google have compelling
    voice input in their products.
    

    View Slide

85. View Slide

86. View Slide

87. View Slide

88. How should web pages change to support voice control?
    

    View Slide

89. View Slide

90. View Slide

91. Google voice search
    

    View Slide

92. You can use speech recognition too.
    http://www.google.com/intl/en/chrome/demos/speech.html
    http://www.moreawesomeweb.com/demos/speech_translate.html
    

    View Slide

93. View Slide

94. Gestures?
    

    View Slide

95. http://leapmotion.com
    

    View Slide

96. https://www.kickstarter.com/projects/1761670738/ring-shortcut-everything
    

    View Slide

97. https://www.kickstarter.com/projects/1761670738/ring-shortcut-everything
    

    View Slide

98. View Slide

99. Amazing, but too new to know
    what, if anything, this technology
    will mean for the web.
    

    View Slide

100. Let’s come back from the future and
     look at something much Dumber.
     Dumber
     

     View Slide

101. Dumber
     

     View Slide

102. Dumber
     

     View Slide

103. -pad remote controls
     D
     

     View Slide

104. -pad remote controls
     D
     

     View Slide

105. -pad remote controls
     D
     

     View Slide

106. http://www.flickr.com/photos/stewc/6669743035/
     

     View Slide

107. http://www.flickr.com/photos/stewc/6669743035/
     TVs browsers that
     support d-pad, send
     arrow key events.
     

     View Slide

108. http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/
     

     View Slide

109. If then
     http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/
     

     View Slide

110. View Slide

111. is undetectable.
     This is a recurring theme for input.
     

     View Slide

112. Sensors and camera
     http://www.flickr.com/photos/retrocactus/2170677056
     

     View Slide

113. Sensors and camera
     Camera
     http://www.flickr.com/photos/retrocactus/2170677056
     

     View Slide

114. GPS
     http://www.flickr.com/photos/3dking/149450434
     

     View Slide

115. GPS
     GeoLocation
     http://www.flickr.com/photos/3dking/149450434
     

     View Slide

116. View Slide

117. Gyroscope & Accelerometer
     

     View Slide

118. http://www.flickr.com/photos/chrisjagers/4694134078
     

     View Slide

119. Back to today’s problems.
     http://www.flickr.com/photos/chrisjagers/4694134078
     

     View Slide

120. View Slide

121. Hover state No hover state
     

     View Slide

122. Hover state
     Typing easier for many
     No hover state
     Typing often more difficult
     

     View Slide

123. Higher precision with mouse means
     smaller targets possible
     Hover state
     Less precise than mouse and requires
     larger touch targets
     Typing easier for many
     No hover state
     Typing often more difficult
     

     View Slide

124. Higher precision with mouse means
     smaller targets possible
     Hover state
     Less precise than mouse and requires
     larger touch targets
     Typing easier for many
     No hover state
     Typing often more difficult
     Right clicking and “power” tools Single and multi-touch gestures
     

     View Slide

125. View Slide

126. View Slide

127. http://www.flickr.com/photos/28096801@N05/5012309802
     

     View Slide

128. I got this. Detect touch.
     http://www.flickr.com/photos/28096801@N05/5012309802
     

     View Slide

129. View Slide

130. Whatever you may think, it currently isn't
     possible to reliably detect whether or not the
     current device has a touchscreen, from within the
     browser.
     —Stu Cox, You Can’t Reliably Detect a Touch Screen
     http://www.stucox.com/blog/you-cant-detect-a-touchscreen/
     

     View Slide

131. Chrome has entertained idea of enabling touch by default.
     https://code.google.com/p/chromium/issues/detail?id=159527
     https://docs.google.com/a/cloudfour.com/presentation/d/1-n1qyzewpagREbzW2zm0wOalq33UhbtbSkWf9mEdly8/
     edit#slide=id.gc2d80e5b_171
     

     View Slide

132. View Slide

133. Detect a mouse? Not reliably.
     

     View Slide

134. View Slide

135. View Slide

136. Surely we can detect a keyboard?
     

     View Slide

137. Surely we can detect a keyboard?
     NOPE
     

     View Slide

138. View Slide

139. Input is dynamic.
     

     View Slide

140. Input is dynamic.
     

     View Slide

141. Boris Smus’s experiments responding to input.
     http://smus.com/touch-laptop-experiments/
     

     View Slide

142. Boris Smus’s experiments responding to input.
     http://smus.com/touch-laptop-experiments/
     

     View Slide

143. http://www.flickr.com/photos/lyza/7382235106
     Maybe we need to be more zen about input.
     

     View Slide

144. View Slide

145. After poking at this problem for a few weeks, my
     conclusion is: every desktop UI should be
     designed for touch now. When any desktop
     machine could have a touch interface, we have to
     proceed as if they all do.
     —Josh Clark
     http://globalmoxie.com/blog/desktop-touch-design.shtml
     

     View Slide

146. View Slide

147. What about those who won’t let
     go of their “power” interfaces?
     http://www.flickr.com/photos/ecos/4092571213/
     

     View Slide

148. http://www.flickr.com/photos/scarygami/5689980135/
     One option: give them a choice.
     

     View Slide

149. View Slide

150. Gmail display density settings
     

     View Slide

151. Th Dream Experience - …
     Uploaded 2 years ago
     More Of The Dream Exp…
     Uploaded 2 years ago
     The Dream Experience -…
     Uploaded 2 years ago
     The Dream Experience …
     Uploaded 2 years ago
     The Love King Breaks It…
     Uploaded 2 years ago
     War Paint for Trees
     From Lincoln Motor Company
     Join Log In Create Watch Upload Search s
     [ ]
     – VIDEOS
     Vimeo Couch Mode
     

     View Slide

152. Couch Mode
     + See all
     Centric TV’s videos / Recently viewed / Related videos
     Th Dream Experience - …
     Uploaded 2 years ago
     More Of The Dream Exp…
     Uploaded 2 years ago
     The Dream Experience -…
     Uploaded 2 years ago
     The Dream Experience …
     Uploaded 2 years ago
     The Love King Breaks It…
     Uploaded 2 years ago
     PROMOTED
     War Paint for Trees
     From Lincoln Motor Company
     Join Log In Create Watch Upload Search s
     [ ]
     – VIDEOS
     Vimeo Couch Mode
     

     View Slide

153. Couch Mode
     + See all
     Centric TV’s videos / Recently viewed / Related videos
     Th Dream Experience - …
     Uploaded 2 years ago
     More Of The Dream Exp…
     Uploaded 2 years ago
     The Dream Experience -…
     Uploaded 2 years ago
     The Dream Experience …
     Uploaded 2 years ago
     The Love King Breaks It…
     Uploaded 2 years ago
     PROMOTED
     War Paint for Trees
     From Lincoln Motor Company
     Join Log In Create Watch Upload Search s
     [ ]
     – VIDEOS
     Couch Mode
     + See all
     Centric TV’s videos / Recently viewed / Related videos
     Th Dream Experience - …
     Uploaded 2 years ago
     More Of The Dream Exp…
     Uploaded 2 years ago
     The Dream Experience -…
     Uploaded 2 years ago
     The Dream Experience …
     Uploaded 2 years ago
     The Love King Breaks It…
     Uploaded 2 years ago
     PROMOTED
     War Paint for Trees
     From Lincoln Motor Company
     Join Log In Create Watch Upload Search s
     [ ]
     – VIDEOS
     Vimeo Couch Mode
     

     View Slide

154. Vimeo Couch Mode
     

     View Slide

155. View Slide

156. The key benefit of this approach:
     You’re designing for user need not
     for a specific form factor or input.
     

     View Slide

157. http://www.flickr.com/photos/raver_mikey/504815463
     

     View Slide

158. View Slide

159. Progressive Input?
     

     View Slide

160. Graph from Chapter 1 of Adaptive Web Design by
     Aaron Gustafson http://easy-readers.net/books/
     adaptive-web-design/
     

     View Slide

161. Graph from Chapter 1 of Adaptive Web Design by
     Aaron Gustafson http://easy-readers.net/books/
     adaptive-web-design/
     Progressive enhancement contains a value judgment
     

     View Slide

162. Who are we to judge what input is better?
     http://www.flickr.com/photos/fensterbme/4783366926
     

     View Slide

163. We need to adapt
     and respond.
     http://www.flickr.com/photos/cdm/147947664/
     

     View Slide

164. Thank You!
     Special thanks to Flickr users for generously sharing their
     photos under creative commons license.
     

     View Slide