The Immobile Web

Presented at AEA DC, August 5, 2013

TVs are the last screens in our lives that haven’t been taken over by computers, but that is about to change. The last year has seen an explosion of SmartTVs many with surprisingly capable browsers. Microsoft recently added Internet Explorer to Xbox 360. Nintendo’s Wii U features a WebKit-based browser. And if the rumors that Apple will release a TV soon are true, web developers everywhere will start to tackle the glass screen hanging on our walls.

But even if you don’t have a TV project today, the lessons from TVs help inform the way we’re developing building web pages for the wide range of devices and inputs we face. TVs are a convenient and easy-to-understand framework to look at a series of challenges that all web developers are about to face in earnest.

Transcript

1. http://www.flickr.com/photos/rheaney/4397863376 Jason Grigsby • @grigs • cloudfour.com The Immobile Web
2. TVs? Really? http://www.flickr.com/photos/clovermountain/417511565
3. http://www.flickr.com/photos/alphachimpstudio/4990357031/
4. Photo by Alan Light http://www.flickr.com/photos/alan-light/216012860/
8. Image source: http://gigaom.com/video/99-dollar-logitech-revue-worth-buying/
9. Vimeo on Google TV
10. Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] – VIDEOS Woah… it’s a web page?
11. Couch Mode + See all Centric TV’s videos / Recently viewed / Related videos Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago PROMOTED War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] – VIDEOS
12. Couch Mode + See all Centric TV’s videos / Recently viewed / Related videos Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago PROMOTED War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] – VIDEOS Couch Mode + See all Centric TV’s videos / Recently viewed / Related videos Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago PROMOTED War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] – VIDEOS
13. If there is a coming zombie apocalypse of devices… http://www.flickr.com/photos/adactio/6301799843
14. then TVs are likely the next wave. http://www.flickr.com/photos/athena1970/3935208655/
15. consumes 33% of peak downstream traffic in U.S.
16. Big enough to get its own button.
17. http://blog.nielsen.com/nielsenwire/online_mobile/double-vision-global-trends-in-tablet-and-smartphone-use-while-watching-tv/
18. 88 % of tablet 86 % of smartphone owners use their device while watching TV. http://blog.nielsen.com/nielsenwire/online_mobile/double-vision-global-trends-in-tablet-and-smartphone-use-while-watching-tv/
20. Microsoft's Xbox 360 video game console is now used more for watching movies and TV shows and listening to music online than playing video games online. —LA Times, March 2012 http://latimesblogs.latimes.com/entertainmentnewsbuzz/2012/03/xbox-more-entertainment-gaming-hbo-go-comcast-xfinity-mlb.html
22. Xbox SmartGlass
23. “To make SmartGlass available to the devices that people already own (but best on Microsoft devices), they settled on a client cross-platform HTML5, JavaScript and JSON powered solution for Windows, Windows Phone, iOS and Android devices.” http://www.istartedsomething.com/20121104/build-2012-xbox-smartglass-powerful-developer-platform-held-back-by-limited-availability/
24. “By the summer of 2012, the majority of the televisions you see in stores will have Google TV embedded in it” Photo by JD Lasica/Socialmedia.biz http://www.flickr.com/photos/jdlasica/5181380514/
26. ?!?!
28. $35 Chromecast
30. =
31. http://www.unwiredview.com/2011/12/21/andy-rubin-%E2%80%9Candroid-daily-activations-top-700k-a-day%E2%80%9D-on-the-way-to-1-million-a-day-in-q2-2012/ Will Google TV follow Android’s path?
32. http://www.flickr.com/photos/acaben/541334636/ “I finally cracked it.”
34. The current Apple TV doesn’t have a browser…
36. Big opportunity is an App Store on Apple TV.
38. Many apps use a combo of web and native
39. Even Apple uses embedded webviews
40. We think the answer can be web standards. The world doesn’t need more proprietary platforms fragmenting the innovative efforts of developers everywhere. —Tom Conrad, Pandora CTO http://blog.pandora.com/2013/06/05/introducing-tv-pandora-com/
41. http://www.flickr.com/photos/34818713@N00/1314251273/
42. Apps = If that is true, don’t you think Apple will ship Safari on TV? http://www.flickr.com/photos/34818713@N00/1314251273/ Embedded Web Views = 3rd Party Browsers
43. Considering TVs helps inform how we build for the web. http://www.flickr.com/photos/revdancatt/3789612273/
44. And can help avoid short-sighted solutions http://www.flickr.com/photos/pss/4876189045/
45. So let’s dig in shall we?
46. http://www.flickr.com/photos/jonathan_moreau/4024305358/
47. Surprisingly good browsers in newer TVs.
48. Brand Model Score Bonus Points Samsung UN55F80BN3 406 12 Toshiba 58L7300 386 16 LG 60A7400 429 8 Panasonic TCP55VT60 317 6 Sharp LC70LE857 412 8 Microsoft Xbox 360 120 5 Sony NSZ-GT1 (Google TV) 330 8 iPhone 4S running iOS 6.0 386 9 Google Chrome on Mac (v27) 463 13 html5test.com of 2013 TVs Total possible points: 500
49. Brand Model Score Samsung UN55F80BN3 55% Toshiba 58L7300 65% LG 60A7400 62% Panasonic TCP55VT60 63% Sharp LC70LE857 56% Microsoft Xbox 360 32% Sony NSZ-GT1 (Google TV) 48% iPhone 4S running iOS 6.0 56% Google Chrome on Mac (v27) 65% css3test.com of 2013 TVs 218 features tested
50. Game console browsers Game console browsers Wii U Browser is a WebKit version of NetFront More details HTML5 Test 258/500 As of December 2012. Source: html5test.com CSS3 Test 48% As of December 2012. Source: css3test.com Acid 3 Test 100% As of December 2012. Source: acid3.acidtests.org Xbox 360 HTML5 Test 120/500 As of February 2013. Source: html5test.com CSS3 Test 33% As of February 2013. Source: css3test.com Acid3 Test 100% As of February 2013. Source: acid3.acidtests.org TV consoles Devices Devices Resources Resources Anna Debenham Anna Debenham http://console.maban.co.uk/
51. So what’s it like to use these TVs?
52. http://www.flickr.com/photos/nathaninsandiego/4829858186/
53. The biggest problem is input.
54. Welcome back T9. We’ve missed you.
58. Which is why you see remotes like this
61. Apple hasn’t solved this problem either
64. Apple’s iOS Remote App Makes Life Bearable
67. Nearly every manufacturer has their own apps now.
68. http://www.flickr.com/photos/marcof/5139160297/
69. It’s a multi-screen world. http://www.flickr.com/photos/marcof/5139160297/
71. How does the web play in this new world?
72. http://www.flickr.com/photos/chrisbartow/5835428673 Designing for a 10-foot UI
73. Larger text and fewer words.
74. Make up, down, left, right directions clear. http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg
76. -pad remote controls D
77. /* CSS */ #copyright { nav-down: #logo; } CSS3 Basic User Interface specification for directional focus navigation http://dev.opera.com/articles/view/tweaking-spatial-navigation-for-tv-browsing/
78. CSS3 Basic User Interface specification for directional focus navigation http://www.w3.org/TR/css3-ui/#nav-dir button#b1 { nav-index:1; nav-right:#b2; nav-left:#b4; nav-down:#b2; nav-up:#b4; } button#b2 { nav-index:2; nav-right:#b3; nav-left:#b1; nav-down:#b3; nav-up:#b1; } button#b3 { nav-index:3; nav-right:#b4; nav-left:#b2; nav-down:#b4; nav-up:#b2; } button#b4 { nav-index:4; nav-right:#b1; nav-left:#b3; nav-down:#b1; nav-up:#b3; }
79. Unfortunately, only TVs using Opera—two 2012 models—supported spatial navigation.
80. Unfortunately, only TVs using Opera—two 2012 models—supported spatial navigation. Late breaking news: Opera is going to add spatial navigation support to Blink.
81. Google TV jQuery UI Library http://code.google.com/p/gtv-resources/
82. While the 2013 TVs did much better with the Google TV code, it didn’t work on many capable TV browsers.
83. Component Button Link Container Grid Lightbox Tab Container Menu AJAX tv.ui.Link Browser link (<a> element) has serveral downsides: when it's focused by browser, layout can be damaged by page scrolling to the link, browser displays link destination on the bottom of the screen which distracts user, browsers by default renders its outline which might not match the UI of app. CSS class used for decoration: tv-link HTML structure: <div class="tv-link" href="http://link to external site">Link content</div> Click or press space or enter to go to url. Google TV features (goes to http://google.com/tv)
84. Component Button Link Container Grid Lightbox Tab Container Menu AJAX tv.ui.Link Browser link (<a> element) has serveral downsides: when it's focused by browser, layout can be damaged by page scrolling to the link, browser displays link destination on the bottom of the screen which distracts user, browsers by default renders its outline which might not match the UI of app. CSS class used for decoration: tv-link HTML structure: <div class="tv-link" href="http://link to external site">Link content</div> Click or press space or enter to go to url. Google TV features (goes to http://google.com/tv)
85. tv.ui.Link Browser link (<a> element) has serveral downsides: when it's focused by browser, layout can be damaged by page scrolling to the link, browser displays link destination on the bottom of the screen which distracts user, browsers by default renders its outline which might not match the UI of app. CSS class used for decoration: tv-link HTML structure: <div class="tv-link" href="http://link to external site">Link content</div> Click or press space or enter to go to url. Google TV features (goes to http://google.com/tv)
86. tv.ui.Link Browser link (<a> element) has serveral downsides: when it's focused by browser, layout can be damaged by page scrolling to the link, browser displays link destination on the bottom of the screen which distracts user, browsers by default renders its outline which might not match the UI of app. CSS class used for decoration: tv-link HTML structure: <div class="tv-link" href="http://link to external site">Link content</div> Click or press space or enter to go to url. Google TV features (goes to http://google.com/tv)
87. tv.ui.Link Browser link (<a> element) has serveral downsides: when it's focused by browser, layout can be damaged by page scrolling to the link, browser displays link destination on the bottom of the screen which distracts user, browsers by default renders its outline which might not match the UI of app. CSS class used for decoration: tv-link HTML structure: <div class="tv-link" href="http://link to external site">Link content</div> Click or press space or enter to go to url. Google TV features (goes to http://google.com/tv) tv.ui.Link Browser link (<a> element) has serveral downsides: when it's focused by browser, layout can be damaged by page scrolling to the link, browser displays link destination on the bottom of the screen which distracts user, browsers by default renders its outline which might not match the UI of app. CSS class used for decoration: tv-link HTML structure: <div class="tv-link" href="http://link to external site">Link content</div> Click or press space or enter to go to url. Google TV features (goes to http://google.com/tv) Wait… WTF?
88. Both solutions require adding a layer of CSS or JS specifically to support TV interaction.
89. http://www.flickr.com/photos/stewc/6669743035/
90. http://www.flickr.com/photos/stewc/6669743035/ TVs browsers that support d-pad, send arrow key events.
91. http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/
92. If then http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/
94. is undetectable. This is a recurring theme for input.
95. Browserscope report screen sizes TVs have inconsistent resolutions and overscan.
96. No problem. We’ll adapt to the screen resolution. http://bradfrostweb.com/blog/web/mobile-first-responsive-web-design/
97. No problem. We’ll adapt to the screen resolution. http://bradfrostweb.com/blog/web/mobile-first-responsive-web-design/
99. 1080p!
100. No worries. We’ll use media type. <link rel="stylesheet" media="screen" href="..."> <link rel="stylesheet" media="tv" href="...">
101. None of the TVs supported the TV media type.
102. Detect a TV with JavaScript?
103. NOPE! Detect a TV with JavaScript?
104. http://www.flickr.com/photos/77799978@N00/5351372848/ Alright fine. We’ll use device detection.
105. Mozilla/5.0 (Linux; U) AppleWebKit/ 537.4 (KHTML, like Gecko) Chrome/ 22.0.1229.79 Safari/537.4 User Agent String for a 2013 LG Smart TV
106. Mozilla/5.0 (Linux; U) AppleWebKit/ 537.4 (KHTML, like Gecko) Chrome/ 22.0.1229.79 Safari/537.4 User Agent String for a 2013 LG Smart TV Nothing that says TV in that string!
107. http://www.flickr.com/photos/plunkmasterknows/357836855/
108. What’s the solution?
109. What’s the solution? At the moment, device detection is best solution.
110. What’s the solution? At the moment, device detection is best solution. We need to put pressure on TV browser makers to interoperate.
111. What about testing? http://www.flickr.com/photos/tym/1255655367
112. “Testing on as many devices as possible is a great idea in theory, but in practice it is untenable. Even if we buy a few devices to try to cover more ground, they will be outdated in just a few months or a year at most. So are we supposed to buy multiple devices per year?” posted at 11:32 am on December 14, 2010 by klayon “If that’s the mobile landscape, I want no part of it.” posted at 07:22 am on December 15, 2010 by Polsonby http://www.alistapart.com/comments/smartphone-browser-landscape/
113. If you thought phones were bad, You ain’t seen nothing yet!
114. http://www.flickr.com/photos/judybaxter/4061019111
115. About About What is an "Open Device Lab"? What is an "Open Device Lab"? Contribute Contribute Help the ODL Movement Help the ODL Movement Sponsor Sponsor Donate to Open Device Labs Donate to Open Device Labs Find Find Locate an Open Device Lab Locate an Open Device Lab OpenDeviceLab.com Access a variety of devices. Worldwide. For free. 66 Open Device Labs 66 Open Device Labs across 22 countries across 22 countries 1059 devices accessible 1059 devices accessible Find the right Open Device Lab for your mission and location: washington, dc Filter by manufacturers Acer Advent Ainol Alcatel Aluratek Amazon Apple Archos Asus Barnes & Noble BlackBerry Bookeen Creative Dell EasyPix Ericsson Fujitsu Geeksphone Google HP HTC Huawei Intel Kobo LG Lenovo Microsoft Motorola Nextbook Device Lab DC Washington DC, Columbia United States 0 devices available 1 found this ODL open 1 say it is closed » More details Greyscale Colored
116. Most stores have no remotes and no wi-fi http://www.flickr.com/photos/elmada/1431918753/
117. Bring your phone for tethering to TVs http://www.flickr.com/photos/bendodson/3367856091/
118. Don’t be surprised if your data gets used
119. What are people actually expecting when they buy a smart tv? http://www.flickr.com/photos/presta/623444414/
120. Storage capacity not listed as a spec
121. Is there a Smart TV market?
122. http://www.flickr.com/photos/darrentunnicliff/4232232092
123. Google TV Spotlight in Chrome Fullscreen http://www.google.com/tv/spotlight-gallery.html
124. http://www.flickr.com/photos/playstationblog/9012525050
128. Are TVs the equivalent of phones before the iPhone? http://www.flickr.com/photos/rhodes/6297487639/
129. Are TVs the equivalent of phones before the iPhone? This gives me hope. http://www.flickr.com/photos/rhodes/6297487639/
130. What lessons can we take away from TVs?
131. It may be multiple screens interacting together. Need to think about multiple screens. http://www.apple.com/ipad/features/
132. It may be multiple screens interacting together. Need to think about multiple screens. http://www.apple.com/ipad/features/
133. 320 px Too much focus on the smaller screen.
134. This is HDTV. 1980 px 1080 px This is HDTV
135. Resolution does not define the optimal experience.
136. http://www.flickr.com/photos/cblue98/7254221968
137. Our big challenge now is adaptive input. http://www.flickr.com/photos/cblue98/7254221968
138. http://www.flickr.com/photos/hellogeri/6154034099/ Two years ago, Jeremy wrote about how…
139. http://www.flickr.com/photos/60415054@N00/14301113/ we told ourselves that the web was…
140. http://www.flickr.com/photos/60415054@N00/14301113/
141. http://www.flickr.com/photos/60415054@N00/14301113/ 640 px 480 px
142. 640 px 480 px
147. 1024 px 768 px
148. http://www.flickr.com/photos/adactio/6153481666/
149. http://www.flickr.com/photos/adactio/6153481666/ Then mobile came and made us realize…
150. that it was a consensual hallucination all along. http://www.flickr.com/photos/garibaldi/303085857/
151. The web never had a fixed canvas. http://www.flickr.com/photos/paulocarrillo/124755065/
152. Even our tools perpetuate the lie.
153. http://www.flickr.com/photos/69797234@N06/7203485148/ We’ve made tremendous progress.
154. But there is another consensual hallucination. http://www.flickr.com/photos/garibaldi/303085857/
156. = =
158. Supports hover and pointer events.
161. Keyboard and touch.
164. Even the iPhone can have a keyboard.
167. Are these laptops or tablets?
170. Desktop computer with 23” touch screen
172. Luke nailed it. http://static.lukew.com/unified_device_design.png
174. We can no longer make assumptions about input based on screen size or form factor. And we probably never should have.
176. mobile desktop THE ART OF WEB DEVELOPMENT THE ART OF WEB DEVELOPMENT Web widgets THE ART OF WEB DEVELOPMENT THE ART OF WEB DEVELOPMENT Mobile widgets
180. Hover state No hover state
181. Hover state Typing easier for many No hover state Typing often more difficult
182. Higher precision with mouse means smaller targets possible Hover state Less precise than mouse and requires larger touch targets Typing easier for many No hover state Typing often more difficult
183. Higher precision with mouse means smaller targets possible Hover state Less precise than mouse and requires larger touch targets Typing easier for many No hover state Typing often more difficult Right clicking and “power” tools Single and multi-touch gestures
186. http://www.flickr.com/photos/28096801@N05/5012309802
187. I got this. Detect touch. http://www.flickr.com/photos/28096801@N05/5012309802
189. Whatever you may think, it currently isn't possible to reliably detect whether or not the current device has a touchscreen, from within the browser. —Stu Cox, You Can’t Reliably Detect a Touch Screen http://www.stucox.com/blog/you-cant-detect-a-touchscreen/
190. Chrome would like to turn on touch events by default. https://code.google.com/p/chromium/issues/detail?id=159527 https://docs.google.com/a/cloudfour.com/presentation/d/1-n1qyzewpagREbzW2zm0wOalq33UhbtbSkWf9mEdly8/edit#slide=id.gc2d80e5b_171
192. Detect a mouse? Not reliably.
195. Surely we can detect a keyboard?
196. Surely we can detect a keyboard? NOPE
198. Input is dynamic.
199. Input is dynamic.
201. is undetectable. Just like every other form of input.
202. Boris Smus’s experiments responding to input.
203. Boris Smus’s experiments responding to input.
204. http://www.flickr.com/photos/lyza/7382235106 Maybe we need to be more zen about input.
206. After poking at this problem for a few weeks, my conclusion is: every desktop UI should be designed for touch now. When any desktop machine could have a touch interface, we have to proceed as if they all do. —Josh Clark http://globalmoxie.com/blog/desktop-touch-design.shtml
208. What about those who won’t let go of their “power” interfaces? http://www.flickr.com/photos/ecos/4092571213/
209. http://www.flickr.com/photos/scarygami/5689980135/ One option: give them a choice.
211. Gmail display density settings
212. Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] – VIDEOS Vimeo Couch Mode
213. Couch Mode + See all Centric TV’s videos / Recently viewed / Related videos Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago PROMOTED War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] – VIDEOS Vimeo Couch Mode
214. Couch Mode + See all Centric TV’s videos / Recently viewed / Related videos Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago PROMOTED War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] – VIDEOS Couch Mode + See all Centric TV’s videos / Recently viewed / Related videos Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago PROMOTED War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] – VIDEOS Vimeo Couch Mode
215. Vimeo Couch Mode
217. The key benefit of this approach: You’re designing for user need not for a specific form factor or input.
218. http://www.flickr.com/photos/raver_mikey/504815463
220. Progressive Input?
221. Graph from Chapter 1 of Adaptive Web Design by Aaron Gustafson http://easy-readers.net/books/adaptive-web-design/
222. Graph from Chapter 1 of Adaptive Web Design by Aaron Gustafson http://easy-readers.net/books/adaptive-web-design/ Progressive enhancement contains a value judgment
223. Who are we to judge what input is better? http://www.flickr.com/photos/fensterbme/4783366926
224. We need to learn to adapt. http://www.flickr.com/photos/cdm/147947664/
225. http://www.flickr.com/photos/nathaninsandiego/4829858186/ Yes, TVs suck right now.
226. But it would be dangerous to dismiss them.
229. Fundamental Truth: We can’t predict future behavior from a current experience that sucks.
230. So even if we don’t have to design for TVs today…
231. It behooves us to start thinking about and planning for what it will be like to do so.
232. TVs help us understand the challenges we already face.
233. Flickr photo by Drift Words: http://www.flickr.com/photos/44124413076@N01/11846265/ And helps us prepare for the uncertainty of what is to come.
234. Thank You! Special thanks to Luke Wroblewski, Patrick H. Lauke, Chewy Chong, Nishant Kothary, Paul Irish, Flickr users sharing under creative commons and the kind folks at Beaverton Video Only.