Adapting to Input

Responsive Web Design has forced us to accept that we don't know the size of our canvas, and we've learned to embrace the squishiness of the web. Input, it turns out, is every bit as challenging as screen size. We have tablets with keyboards, laptops that become tablets, laptops with touch screens, phones with physical keyboards, and even phones that become desktop computers. In this session, Jason will guide you through the input landscape, showing you new forms of input like sensors and voice control, as well as new lessons about old input standbys. You'll learn the design principles necessary to build web sites that respond and adapt to whatever input people use.

Transcript

1. Adapting to Input Jason Grigsby • @grigs • cloudfour.com
2. Web was created formless. https://www.flickr.com/photos/chiaralily/3068047592
3. http://www.wired.co.uk/magazine/archive/ 2014/03/web-at-25/tim-berners-lee
4. https://www.flickr.com/photos/inpivic/5206515374/
5. https://www.flickr.com/photos/tomokyo-blue/5812115264
6. http://www.flickr.com/photos/60415054@N00/14301113/ So we told ourselves that the web was…
7. http://www.flickr.com/photos/60415054@N00/14301113/ 640 px 480 px
8. 640 px 480 px
9. 800 px 600 px
10. 1024 px 768 px
11. http://www.flickr.com/photos/adactio/6153481666/ Then mobile came and made us realize…
12. that it was a consensual hallucination all along. http://www.flickr.com/photos/garibaldi/303085857/
13. The web never had a fixed canvas. http://www.flickr.com/photos/paulocarrillo/124755065/
15. https://www.flickr.com/photos/giuliatanini/7125944117/
16. http://www.flickr.com/photos/69797234@N06/7203485148/ We’ve made tremendous prog
17. But there is another consensual hallucination. http://www.flickr.com/photos/garibaldi/303085857/
18. = =
19. mobile desktop THE ART OF WEB DEVELOPMENT THE ART OF WEB DEVELOPMENT Web widgets THE ART OF WEB DEVELOPMENT THE ART OF WEB DEVELOPMENT Mobile widgets
23. It’s not that we’re technically incapable, but adapting a phone UI to a tablet UI is not so dissimilar from trying to automatically adapt desktop UI to a phone. They are fundamentally different platforms with different usability considerations… —Todd Anglin, Kendo UI http://www.kendoui.com/blogs/teamblog/posts/12-09-11/universal_mobile_apps_with_html5_and_kendo_ui.aspx
24. It’s fairly certain that the highest-value use will stay predominantly on desktop… Most complex tasks have vastly better user experience on the desktop and thus will be performed there. —Jacob Nielsen https://www.nngroup.com/articles/transmedia-design-for-3-screens/
25. http://www.flickr.com/photos/royalsapien/2387707860
26. We envision desktop users as different from mobile. http://demos.kendoui.com/web/grid/editing.html
27. http://www.flickr.com/photos/shantellmartin/4543010568
28. https://www.flickr.com/photos/keh-s/6020885164/ It’s time to break free of our assumptions about input.
29. Four truths about input
30. #1 Input is exploding
31. An incomplete history of computer input
32. Focus on broad adoption Using Apple as a lens
33. Keyboard 1874 First typewriters in 1860s Remington Typewriter 1874 Popularizes Qwerty keyboard Qwerty patent granted 1878
34. Mouse 1984 Mouse prototypes 1960s by Engelbart and Mallebrein. Xerox uses mouse in 70s, early 80s. Apple Macintosh popularizes mouse and GUI in 1984.
35. Many alternatives to the mouse are created—trackballs, trackpads, and joysticks to name a few—but all replace the mouse and control the cursor. No change in behavior.
36. Scroll Wheel 1996 First scrolling mouse in 1985. Scroll wheel on side. Microsoft Intellimouse popularizes scroll wheel in 1996. Scroll wheel controls windows, not cursor.
37. Camera 2005 / 2006 Cameras are now included in phones and computers. Cameraphones sold in 2005 alone greater than all digital cameras sold to date. All Mac computers include cameras starting 2006.
38. iPhone 2007 Multi-touch screen Camera Accelerometer Proximity Sensor Ambient Light Sensor
39. Macbook Air January 2008 Multi-touch trackpad Gestures in Mac OS Controls more than cursor
40. iPhone 3G 2008 GPS (Location services)
41. iPhone 3GS 2009 Voice control Magnetometer (Compass)
42. iPhone 4 2010 Gyroscope Front facing camera
43. Magic Trackpad 2010 Multi-touch trackpad Gestures in Mac OS No longer limited to laptops
44. iPhone 4S 2011 Siri Bluetooth LE
45. iPhone 5 2012 Nothing?
46. iPhone 5S 2013 Fingerprint Sensor
47. iPhone 6 2014 NFC (Apple Pay) Barometer
48. iPhone 6S 2015 3D Touch
49. Same story could be told using Android or Microsoft.
50. 1984 1874 142 years 1996 12 years
51. 1984 1874 1996
52. 1984 1996 2005 9 years 2 years 2007
53. 1984 1996 2005 2007
55. 2007 Touch Camera Accelerometer Proximity Sensor Ambient Light Sensor GPS Trackpad 2008 Voice control Magnetometer 2009 2010 Gyroscope Front-facing Camera Magic Trackpad 2011 Siri Bluetooth LE 2012 2013 Fingerprint Sensor 2014 NFC (Apple Pay) 2015 3D Touch 2016 ?
56. Touch 3D Touch Fingerprint sensor Camera (video and image) GPS (location) Compass Bluetooth LE Audio (Siri) Gyroscope Accelerometer Barometer Proximity sensor Ambient light sensor NFC (Apple Pay) iBeacon
57. Things aren’t slowing down. https://www.flickr.com/photos/andreastsonis/8675283555
58. Siri gets all of the hype… http://www.98ps.com/viewnews-15222.html
59. but both Microsoft and Google have compelling voice input in their products.
63. How should web pages change to support voice control?
66. Google voice search
67. https://www.youtube.com/watch?v=kwn9Lh0E_vU
68. Remember Leap Motion?
69. https://www.youtube.com/watch?v=LJPxyWM9Ujg
70. https://www.youtube.com/watch?v=fiZkEYLXctE
71. #2 Input is a continuum
72. = =
73. Supports hover and pointer events.
74. Keyboard and touch.
75. Even the iPhone can have a keyboard.
76. Are these laptops or tablets?
77. Desktop computer with 23” touch screen
79. Luke Wroblewski nailed it. http://static.lukew.com/unified_device_design.png
80. We can no longer make assumptions about input based on screen size or form factor.
81. 3. Input is undetectable
82. Higher precision with mouse means smaller targets possible Hover state Less precise than mouse and requires larger touch targets Typing easier for many No hover state Typing often more difficult Right clicking and “power” tools Single and multi-touch gestures
84. I got this. Detect touch. http://www.flickr.com/photos/28096801@N05/5012309802
85. Whatever you may think, it currently isn't possible to reliably detect whether or not the current device has a touchscreen, from within the browser. —Stu Cox http://www.stucox.com/blog/you-cant-detect-a-touchscreen/
86. Chrome has entertained idea of enabling touch by default. https://code.google.com/p/chromium/issues/detail?id=159527 https://docs.google.com/a/cloudfour.com/presentation/d/1-n1qyzewpagREbzW2zm0wOalq33UhbtbSkWf9mEdly8/edit#slide=id.gc2d80e5b_171
87. Detect a mouse? Not reliably.
88. Surely we can detect a keyboard? NOPE
89. https://www.flickr.com/photos/mediaflex/4628277817/
90. https://www.flickr.com/photos/ dentyone/2410917872/
91. Be thankful. This saves us from ourselves.
92. Interactive touch laptop experiments by Boris Smus
94. We can only detect input when it is used and that’s too late for our user interfaces.
95. 4. Input is transient
97. Knowing what input is used one moment tells you nothing about what will be used next.
98. Input is exploding continuum undetectable transient
99. Overwhelmed? https://www.flickr.com/photos/cheo70/2702682262/
100. https://www.flickr.com/photos/evilerin/3723714381/
102. We just need to rewire our brains again.
103. Adapting to input
104. #1 Design for the largest target by default
106. Fitts’s Law The time to acquire a target is a function of the distance to and size of the target.
107. After poking at this problem for a few weeks, my conclusion is: every desktop UI should be designed for touch now. When any desktop machine could have a touch interface, we have to proceed as if they all do. —Josh Clark http://globalmoxie.com/blog/desktop-touch-design.shtml
108. #2 Design for modes of interaction instead of inputs
109. Display density settings
110. Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] – VIDEOS Vimeo Couch Mode
111. Couch Mode + See all Centric TV’s videos / Recently viewed / Related videos Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago PROMOTED War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] – VIDEOS Couch Mode + See all Centric TV’s videos / Recently viewed / Related videos Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago PROMOTED War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] – VIDEOS Vimeo Couch Mode
112. Vimeo Couch Mode
113. You’re designing for user need—not for a specific form factor or input.
114. #3 Make things accessible
115. D-pad remote controls
116. http://www.flickr.com/photos/stewc/6669743035/ TVs browsers that support d-pad, send arrow key events.
117. If then http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/
118. is undetectable.
119. If then http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/
120. Support http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/
121. Building accessibly increases the likelihood of support for future, unknown inputs.
124. #4 Design for multiple concurrent inputs
125. = =
126. Interactive touch laptop experiments by Boris Smus
127. At minimum, don’t assume if one input is present that the person doesn’t have access to other types of input.
128. Look for opportunities to combine input to create more powerful experiences.
129. #5 Abstract baseline input
130. Mobile Tablet Desktop
131. Small Medium Large
132. TAP https://www.flickr.com/photos/openexhibits/5884451755
133. CLICK
134. POINT SELECT
135. Mouse events and touch events are fundamentally different beasts in browsers today, and that makes it hard to write cross-platform apps. These incompatibilities lead to applications having to listen to 2 sets of events, mouse on desktop and touch for mobile. https://github.com/jquery/PEP/
136. http://ie.microsoft.com/testdrive/ieblog/2011/Sep/20_TouchInputforIE10andMetrostyleApps_1.png http://www.w3.org/TR/pointerevents/ http://blog.webplatform.org/2013/02/pointing-toward-the-future/ New Pointer Events spec normalizes touch, mouse and stylus Pointer Events builds on the DOM event model to offer a new way to handle input on the web, enabling developers to build touch-first experiences that work with mouse, pen, and other pointing devices as well…They are also designed from the ground up to allow modern browsers to accelerate the touch-surface performance, leading to a smoother user experience.
137. Shipped Opposed In development
138. jQuery Pointer Events Polyfill https://github.com/jquery/PEP/
139. #6 Progressively enhance input
140. Opportunity is knocking!
143. We’re pretty good at using Geolocation.
144. Compass.js http://ai.github.io/compass.js/
145. Warby Parker Gyroscope https://www.warbyparker.com/eyeglasses/women/simone/tea-rose-fade
146. Or hover
148. https://www.youtube.com/watch?v=7Yp0BpiDzXE
151. https://www.youtube.com/watch?v=Ow8gA0qe9SI
152. <input type="email" autocomplete="email" name="email"> <input type="text" autocomplete="cc-name" name="card-name"> <input type="text" autocomplete="cc-number" name="card-num"> <input type="text" autocomplete="cc-exp-month" name="card-exp-month"> <input type="text" autocomplete="cc-exp-year" name="card-exp-year"> <input type="text" autocomplete="cc-csc" name="card-csc"> Remember to support browser auto-fill and new Payment Request API http://www.html5rocks.com/en/tutorials/forms/requestautocomplete/
155. You can use speech recognition too. http://www.google.com/intl/en/chrome/demos/speech.html http://www.moreawesomeweb.com/demos/speech_translate.html
156. Special thanks to Eric Bidelman http://moreawesomeweb.com Speech Recognition API Support
157. https://www.youtube.com/watch?v=_BUwOBdLjzQ Web Bluetooth
158. https://www.youtube.com/watch?v=FYxQ5TeVFdE Physical Web
159. https://www.youtube.com/watch?v=6z9ED4fmi1w Physical Web &
 Web Bluetooth
160. #7 Make input part of test plans
161. Need to add input to our device labs http://blog.adtile.me/2014/01/08/adtile-device-lab/
162. https://www.flickr.com/photos/jeepersmedia/14692542501
163. https://www.flickr.com/photos/yuckymuffintech/6988362001
165. 1. Design for the largest target by default 2. Design for modes of interaction instead of inputs 3. Make things accessible 4. Design for multiple concurrent inputs 5. Abstract baseline input 6. Progressively enhance input 7. Make input part of test plans
166. Input is exploding continuum undetectable transient
167. www.flickr.com/photos/brunauto/5062644167/ Learn from our mobile context mistakes.
168. Who are we to judge which input is better? http://www.flickr.com/photos/fensterbme/4783366926
169. We need to learn to adapt. http://www.flickr.com/photos/cdm/147947664/
170. Learn how to let go of illusions that comfort us. http://www.flickr.com/photos/garibaldi/303085857/
171. Thank You!