Upgrade to Pro — share decks privately, control downloads, hide ads and more …

UX Immersion 2013 — Immobile Web

UX Immersion 2013 — Immobile Web

TVs are the last screens in our lives that haven’t been taken over by computers, but that’s about to change. During the past year, SmartTVs exploded (many with surprisingly capable browsers), Microsoft added IE to Xbox 360, and Nintendo built a WebKit-based browser into the Wii U.

Plus, if the rumors are true that Apple will release a TV, web developers will jump into tackling this new-yet-familiar UI.

But even if you’re not yet designing for TV, you should definitely hear Jason Grisgby talk about:

Today’s TVs, their capabilities, and their design
Best Practices
What TVs can teach us about designing for mobile devices—and optimizing inputs, too
Why methods like responsive design aren’t well-suited to TVs
The challenges of building web pages for TVs—and today’s options for overcoming them
You’ll leave this talk with an exciting vision of how our web will change once we have various screens interacting with each other. It’s an entirely different topic from his full-day workshop.

And no joke, Jason once brought donuts to the sales people who let him spend hours testing the web browsers on their TVs. His fascination with this soon-to-be reality is an understandable extension of his career-long fascination with multi-device experiences. And you couldn’t be learning from a better, more passionate teacher.


Jason Grigsby

April 23, 2013

More Decks by Jason Grigsby

Other Decks in Technology


  1. e Immobile Web Jason Grigsby • @grigs • cloudfour.com Slides:

  2. Tweets: @grigs_talks Slides: bit.ly/ux-immobile-web http://www.flickr.com/photos/stevegarfield/4247757731/

  3. TVs? Really? http://www.flickr.com/photos/kmakice/3906072190/

  4. http://www.flickr.com/photos/alphachimpstudio/4990357031/

  5. Photo by Alan Light http://www.flickr.com/photos/alan-light/216012860/

  6. None
  7. None
  8. Image source: http://gigaom.com/video/99-dollar-logitech-revue-worth-buying/

  9. Vimeo on Google TV

  10. Woah… it’s just a web page?

  11. View in Couch Mode

  12. If there is a coming zombie apocalypse of devices… http://www.flickr.com/photos/adactio/6301799843

  13. then TVs are likely the next wave. http://www.flickr.com/photos/athena1970/3935208655/

  14. consumes 32.7% of peak downstream traffic in U.S.

  15. Big enough to get its own button.

  16. 88% tablet 86% smartphone People often use mobile while watching

    tv. owners say they use their device while watching TV at least once a month. http://blog.nielsen.com/nielsenwire/online_mobile/double-vision-global-trends-in-tablet- and-smartphone-use-while-watching-tv/
  17. None
  18. http://www.flickr.com/photos/acaben/541334636/ “I nally cracked it.”

  19. None
  20. None
  21. None
  22. None
  23. http://www.flickr.com/photos/yamagatacamille/4267887034/

  24. http://allthingsd.com/20091111/nokia-apple/

  25. e current Apple TV doesn’t have a browser…

  26. http://www.flickr.com/photos/acaben/541334636/

  27. [People] “don’t want a computer on their TV,” Apple CEO

    Steve Jobs said today. “ ey have computers. ey go to their wide-screen TVs for entertainment. Not to have another computer. is is a hard one for people in the computer industry to understand, but it's really easy for consumers to understand. ey get it.” http://www.flickr.com/photos/acaben/541334636/
  28. And so, it turns out people want keyboards. I mean,

    when I started in this business one of the biggest challenges was that people couldn’t type.… And if you do email of any volume, you gotta have a keyboard. So we look at the tablet and we think it’s gonna fail. —Steve Jobs, 2003 http://www.flickr.com/photos/acaben/541334636/
  29. Big opportunity is an App Store on Apple TV.

  30. None
  31. Major Brands Using Combo of Web and Native

  32. Even Apple uses embedded webviews

  33. None
  34. None
  35. Apps = http://www.flickr.com/photos/34818713@N00/1314251273/

  36. Apps = http://www.flickr.com/photos/34818713@N00/1314251273/ Embedded Web Views =

  37. Apps = http://www.flickr.com/photos/34818713@N00/1314251273/ Embedded Web Views =3rd Party Browsers

  38. Apps = If that is true, don’t you think Apple

    will ship Safari? http://www.flickr.com/photos/34818713@N00/1314251273/ Embedded Web Views =3rd Party Browsers
  39. “By the summer of 2012, the majority of the televisions

    you see in stores will have Google TV embedded in it” Photo by JD Lasica/Socialmedia.biz http://www.flickr.com/photos/jdlasica/5181380514/
  40. ?!?!

  41. None
  42. Samsung 23.6% Vizio 15.4% LG Electronics 12.4% Sony 8.0% Toshiba

    7.8% Others 32.8% Q4 2011 US LCD TV Market Share
  43. Samsung 23.6% Vizio 15.4% LG Electronics 12.4% Sony 8.0% Toshiba

    7.8% Others 32.8% Q4 2011 US LCD TV Market Share
  44. Samsung 23.6% Vizio 15.4% LG Electronics 12.4% Sony 8.0% Toshiba

    7.8% Others 32.8% Q4 2011 US LCD TV Market Share
  45. Samsung 23.6% Vizio 15.4% LG Electronics 12.4% Sony 8.0% Toshiba

    7.8% Others 32.8% Q4 2011 US LCD TV Market Share ?
  46. None
  47. =

  48. =

  49. http://www.unwiredview.com/2011/12/21/andy-rubin-%E2%80%9Candroid-daily-activations-top-700k-a-day%E2%80%9D-on-the-way-to-1-million-a-day-in-q2-2012/ Will Google TV follow Android’s path?

  50. Considering TVs helps inform how we build for mobile. http://www.flickr.com/photos/revdancatt/3789612273/

  51. And can help us avoid short-sighted solutions http://www.flickr.com/photos/pss/4876189045/

  52. So let’s dig in shall we?

  53. First, the good news.

  54. Surprisingly good browsers in newer TVs.

  55. Year Brand Model Score Bonus Points Notes 2011 LG 60PZ950

    221 6 2011 Sony KDL55HX729 222 6 2011 Sony NSZ-GT1 343 8 Google TV v3.2 2012 LG 47LS5700 302 8 2012 LG 55LM62BND1 302 8 2012 Sharp LC80LE844 281 0 2012 Samsung UN55ES6100 229 2 2012 Samsung UN55ES8000 244 2 2012 Sony KDL55HX750 267 6 2012 iPhone 4S 305 9 Running iOS 5.1 2012 Google Chrome 378 13 Chrome 18 on Mac html5test.com Total possible points: 475
  56. • backgroundsize • borderimage • borderradius • boxshadow • canvas

    • canvastext • csstransforms • csstransitions • fontface • generatedcontent • hashchange • hsla • input:autofocus • input:max • input:min • input:pattern • input:required • input:step • multiplebgs • opacity • postmessage • rgba • sessionstorage • smil • svg • svgclippaths • textshadow 100% of Smart TVs tested support
  57. • localstorage • applicationcache • csscolumns • cssgradients • history

    • input:multiple • input:placeholder • inputtypes:email • inputtypes:number • inputtypes:search • inputtypes:tel • inputtypes:url • webworkers • audio:m4a • audio:mp3 • cssanimations • cssre ections • draganddrop • exbox-legacy • inputtypes:range • video:h264 • websqldatabase 70% or more support
  58. • websockets • inlinesvg • inputtypes:date • inputtypes:datetime • inputtypes:datetime-

    local • inputtypes:month • inputtypes:time • inputtypes:week • input:autocomplete • input:list • geolocation • indexeddb • inputtypes:color • audio:wav • webgl • audio:ogg • csstransforms3d • exbox • touch • video:ogg • video:webm • audio* • video* Poor support for the following * is has to be a mistake.
  59. More importantly, the 2012 models are much faster.

  60. More importantly, the 2012 models are much faster. (How fast

    remains a secret. No one publishes CPU speeds.)
  61. Testing Websites in Game Console Browsers by#ANNA DEBENHAM#∙#September#11,#2012 Published#in#Browsers,#Project#Management,#Workflow#&#Tools#∙#22#Comments More%than%one%in%eight%internet%users%in%the%UK,%US,%and%France—and%nearly%one%in%four

    American%teens—uses%a%game%console%to%get%online,%according%to%studies%from%2010%and 2011. This%isn’t%new.%Internet%capabilities%were%introduced%in consoles%back%in%1997%with%the%Game.com,%which%allowed users%to%upload%scores%by%plugging%in%a%dialIup%modem and%even%to%check%email%and%browse%websites%if%they%had an%internet%cartridge. Today’s%three%main%consoles%(Microsoft’s%Xbox%360, Sony’s%PlayStation%3,%and%Nintendo’s%Wii)%all%have 1 361 Issue#№ HOME ARTICLES COLUMNS BLOG TOPICS
  62. Game console browsers Game console browsers Wii U Browser is

    a WebKit version of NetFront More details HTML5 Test 258/500 As of December 2012. Source: html5test.com CSS3 Test 48% As of December 2012. Source: css3test.com Acid 3 Test 100% As of December 2012. Source: acid3.acidtests.org Xbox 360 HTML5 Test 120/500 As of February 2013. Source: html5test.com CSS3 Test 33% As of February 2013. Source: css3test.com Acid3 Test 100% As of February 2013. Source: acid3.acidtests.org TV consoles Devices Devices Resources Resources Anna Debenham Anna Debenham
  63. None
  64. None
  65. None
  66. So what’s it like to use these TVs?

  67. http://www.flickr.com/photos/nathaninsandiego/4829858186/

  68. e biggest problem is input.

  69. Welcome back T9. We’ve missed you.

  70. None
  71. None
  72. Which is why you see remotes like this

  73. Apple hasn’t solved this problem either

  74. Apple’s iOS Remote App Makes Life Bearable

  75. Nearly every manufacturer has their own apps now.

  76. Nearly every manufacturer has their own apps now.

  77. Nearly every manufacturer has their own apps now.

  78. One of the core lessons: Multi-screen world. It’s a multi-screen

    world. http://www.flickr.com/photos/marcof/5139160297/
  79. How does the web play in this new world?

  80. Two main methods of input on these TVs. (Other than

    linked apps)
  81. D-pads

  82. Pointers http://thecoolgadgets.com/lg-magic-motion-remote-control-updating-with-3d-button-gesture-and-voice-recognition/

  83. When done properly, D-Pad works very well. (Free Opera TV

    SDK Emulator!)
  84. When done properly, D-Pad works very well. (Free Opera TV

    SDK Emulator!)
  85. But when D-pads control on-screen pointers… Yuck!

  86. Pointers work well with on screen cursors, but are harder

    to use precisely. Easy to miss your target. Well implemented D-pad is faster.
  87. Plenty of experiments.

  88. None
  89. Touchpads integrated into remotes

  90. Voice control?

  91. Gesture Controls?

  92. Eventually something will stick, right?

  93. Is Samsung on the right track?

  94. None
  95. e TV Context http://www.flickr.com/photos/imnohero/2330548144

  96. e TV Context Yes, I said “Context.” http://www.flickr.com/photos/imnohero/2330548144

  97. http://www.flickr.com/photos/brunauto/5062644167/ Our vision of mobile context is often wrong.

  98. 80% during misc downtime 76% while waiting in lines 62%

    while watching TV 69% for point of sale research http://www.flickr.com/photos/missmeng/5327470961
  99. TMI: 39% use phone on the toilet.

  100. Yes, we can’t know the mobile context. http://www.flickr.com/photos/timcaynes/158599960/

  101. But when it comes to TVs, context seems to matter

    again. http://www.flickr.com/photos/imnohero/2330548144
  102. Two great sources for information on designing for TVs.

  103. http://dev.opera.com/articles/view/creating-web-content-for-tv/

  104. https://developers.google.com/tv/web/docs/design_for_tv

  105. http://www.flickr.com/photos/chrisbartow/5835428673 Designing for a 10-foot UI

  106. • Limit paragraphs to 90 words • Break into small

    chunks • Line length: 5-7 words • Body text around 21pt on 720p and 28pt on 1080p • Add more leading • Minimum font size of 22px • Line length: 10 words or less • Generous leading Making text easy to read Google Opera
  107. None
  108. “A good rule of thumb is to increase the size

    of an element (such as an image or font) 1.5x for 720p and 2.0x for 1080p relative to the size of that element in a normal PC browser experience.” —Google TV Guide
  109. • When designing a web page for TV, the viewable

    area should display less information overall, and what's there should focus on a con ned set of tasks (even consider performing their desired task automatically or select by default). • Primary activity often revolves around quick information look- up (for instance, cast and crew details for a particular movie, weather reports, TV listings) and quick access to services. Web content for TV should therefore be optimised — in terms of overall presentation, navigation and functionality — and task- focused, giving quick and clear access to all relevant features and information. Optimize for tasks Google Opera
  110. None
  111. Hover is back!

  112. “ e main interface of Google TV encourages the use

    of the D-pad on the remote to make selections on a screen -- it's likely that users will keep this habit even on the web.”
  113. /* CSS */ #copyright { nav-down: #logo; } CSS3 Basic

    User Interface speci cation for directional focus navigation http://dev.opera.com/articles/view/tweaking-spatial-navigation-for-tv-browsing/
  114. button#b1 { nav-index:1; nav-right:#b2; nav-left:#b4; nav-down:#b2; nav-up:#b4; } button#b2 {

    nav-index:2; nav-right:#b3; nav-left:#b1; nav-down:#b3; nav-up:#b1; } W3C’s example of four buttons button#b3 { nav-index:3; nav-right:#b4; nav-left:#b2; nav-down:#b4; nav-up:#b2; } button#b4 { nav-index:4; nav-right:#b1; nav-left:#b3; nav-down:#b1; nav-up:#b3; } http://www.w3.org/TR/css3-ui/#nav-dir
  115. Unfortunately, only 2 of the 10 TVs I tested supported

    spatial navigation.
  116. Google TV jQuery UI Library http://code.google.com/p/gtv-resources/

  117. Unfortunately, I had trouble getting the Google TV jQuery UI

    library to work on non-Google TVs. Needs more testing.
  118. Both solutions require adding a layer of CSS or JS

    speci cally to support TV interaction.
  119. • Google TV may not be able to render a

    page as quickly as your workstation. Performance Challenges Google Opera • Modest hardware. Somewhere between high-end smart phones and low end laptops. • Avoid overly heavy and complex JavaScript. • Avoid layering and opacity. • Low limit on caching. Cannot assume assets cached. Cannot rely on cookies for subsequent session.
  120. http://thereisnofold.com/ Uh oh… scrolling sucks on many TVs.

  121. Horizontal paging is preferred http://gtv-resources.googlecode.com/svn/trunk/gtv-jquery-demo/index.html

  122. • Only HDTVs. • 720p and 1080i/p • e exact

    pixel dimensions of the display varies by TV manufacturer. • Provides an auto-zoom feature which you need to design for or around. Supporting different screen resolutions Google Opera • Most modern web-enabled TVs support 1280×720 as a minimum resolution. • 720p content is usually upscaled • Virtual resolutions — as an example, the Nintendo Wii has a virtual width of 800 pixels. Height varies based on the type of TV (4:3 or 16:9 aspect ratio) and user settings.
  123. Browserscope report screen sizes Actual resolutions differ greatly.

  124. None
  125. No problem. We’ll adapt to the screen resolution. http://bradfrostweb.com/blog/web/mobile-first-responsive-web-design/

  126. No problem. We’ll adapt to the screen resolution. http://bradfrostweb.com/blog/web/mobile-first-responsive-web-design/

  127. None
  128. HDTV!

  129. No worries. We’ll use media type. <link rel="stylesheet" media="screen" href="...">

    <link rel="stylesheet" media="tv" href="...">
  130. None of the TVs supported the TV media type.

  131. https://twitter.com/#!/patrick_h_lauke/status/190078528568569856 https://twitter.com/#!/patrick_h_lauke/status/190078688287653889 Media types are useless except for screen and

  132. Web developers are litter bugs. http://www.flickr.com/photos/jpdaigle/3393858438/

  133. None
  134. Sympathy for the Browser Makers

  135. We need to be careful of the impact we have

    on the world. http://www.flickr.com/photos/samout3/3411361042/
  136. Detect a TV with JavaScript?

  137. Detect a TV with JavaScript? Nope.

  138. http://www.flickr.com/photos/77799978@N00/5351372848/ Alright ne. We’ll use device detection.

  139. Mozilla/5.0 (DirectFB; Linux; ko-KR) AppleWebKit/534.26+ (KHTML, like Gecko) Version/5.0 Safari/534.26+

    User Agent String for a 2012 LG Smart TV
  140. Mozilla/5.0 (DirectFB; Linux; ko-KR) AppleWebKit/534.26+ (KHTML, like Gecko) Version/5.0 Safari/534.26+

    User Agent String for a 2012 LG Smart TV Nothing we can use in that string!
  141. at sucks. http://www.flickr.com/photos/plunkmasterknows/357836855/

  142. What’s the solution? I don’t see many people actively working

    on it.
  143. Are Smart TVs the equivalent of phones before the iPhone

    was released? http://www.flickr.com/photos/rhodes/6297487639/
  144. Are Smart TVs the equivalent of phones before the iPhone

    was released? is gives me hope. http://www.flickr.com/photos/rhodes/6297487639/
  145. Responsive web design still makes sense.

  146. Choosing responsiveness, as a characteristic shouldn’t necessarily de ne the

    wider implementation approach. Device Experiences (i.e. standalone sites, aimed at a group of devices) can also be responsive, providing the exibility to support a much wider range of devices. —Stephanie Rieger

    M B E R 14, 2010 Smartphone Browser Landscape by P E T E R - P A U L K O C H Published in: User Interface Design, Mobile, Mobile Design, Mobile Development Discuss this article » | Share this article » Users expect websites to work on their mobile phones. In two to three years, mobile support will become standard for any site. Web developers must add mobile web development to their skill set or risk losing clients. How do you make websites mobile compatible? The answer is obvious: By testing them on all mobile phones, and by solving the problems you encounter. But, that’s a useless answer. It’s impossible to test your designs on every mobile phone out there. Within the mobile phone landscape, there are at least ten operating systems (OSs) and fifteen browsers that require consideration. Mobile devices are expensive, and not every web developer can afford to buy five to ten of them. Testing “on all mobile phones” is impossible for most web developers. In this article, I’ll give you an overview of the mobile web market, as well as phone platforms and their browsers, so that you can decide which mobile devices to test on. Then, we’ll look at how to set up a mobile test bed. Search ALA include discussions Topics Code Content Culture Design Mobile Process User Science Snapshot Most web designers and developers (not to mention the entire blogosphere) fall squarely in the high-end market. A cultural bias exists against OSs aimed at any other market. As a result, most people focus on the struggle between iOS and Android, and ignore the rest. This has to change. Stay in better touch with customers with No. 320
  148. “Testing on as many devices as possible is a great

    idea in theory, but in practice it is untenable. Even if we buy a few devices to try to cover more ground, they will be outdated in just a few months or a year at most. So are we supposed to buy multiple devices per year?” posted at 11:32 am on December 14, 2010 by klayon “If that’s the mobile landscape, I want no part of it.” posted at 07:22 am on December 15, 2010 by Polsonby http://www.alistapart.com/comments/smartphone-browser-landscape/
  149. If you thought phones were bad, You ain’t seen nothing

  150. Most stores have no remotes and no wi- http://www.flickr.com/photos/elmada/1431918753/

  151. Bring your phone for tethering to TVs http://www.flickr.com/photos/bendodson/3367856091/

  152. Don’t be surprised if your data gets used…

  153. What are people actually expecting when they buy a smart

    tv? http://www.flickr.com/photos/presta/623444414/
  154. Storage capacity not listed as a spec.

  155. Is there a Smart TV market?

  156. Opera TV Emulator http://www.opera.com/business/tv/emulator/

  157. None
  158. Google TV Emulator https://developers.google.com/tv/android/docs/gtv_emulator

  159. Google TV Spotlight in Chrome Fullscreen http://www.google.com/tv/spotlight-gallery.html

  160. Google TV Spotlight in Chrome Fullscreen http://www.google.com/tv/spotlight-gallery.html

  161. What lessons can we take away from TVs?

  162. 320 px Too much focus on the smaller screen.

  163. Why do we need to look at our phones to

    get directions? One vibration for left. Two for right. http://www.flickr.com/photos/williamhook/4225307113
  164. TV resolution reinforces this idea 1980 px 1080 px This

    is HDTV
  165. It may be multiple screens interacting together. Need to think

    about multiple screens. http://www.apple.com/ipad/features/
  166. It may be multiple screens interacting together. Need to think

    about multiple screens. http://www.apple.com/ipad/features/
  167. What does TV tell us about context?

  168. No context Context? What does TV tell us about context?

  169. Or is TV context different simply because we’re still in

    the feature phone era of Smart TVs? http://www.flickr.com/photos/rhodes/6297487639/
  170. Net ix: posture de ning different experiences http://www. ickr.com/photos/ezu/66815736/

  171. “Some people at Net ix have been arguing for a

    single experience across all devices. is has never born out in any kind of testing. Instead, Net ix has a variety of experiences on different devices and even regions.” http://www.lukew.com/ff/entry.asp?1339
  172. • User posture: Stationary, Lean back, on-the-go, shared • Input

    capabilities: pointer/keyboard, LRUD/OSK, Gesture/OSK • Navigation style: controls & windows, panes • Display capabilities: Hi-Res, near, far away, small, medium, large • ese constraints are really powerful. You need to embrace them to get to appropriate designs. http://www.lukew.com/ff/entry.asp?1339
  173. When we need more control to craft an experience for

    a given device, how can we do so in a sustainable manner?
  174. http://www.flickr.com/photos/arselectronica/8570840931/

  175. https://developers.google.com/glass/ui-guidelines

  176. http://www.flickr.com/photos/nathaninsandiego/4829858186/ Yes, Smart TVs suck right now.

  177. But it would be dangerous to dismiss them.

  178. None
  179. None
  180. Fundamental Truth: We can’t predict future behavior from a current

    experience that sucks.
  181. So even if we don’t have to design for TVs

  182. It behooves us to start thinking about and planning for

    what it will be like to do so…
  183. Flickr photo by Drift Words: http://www.flickr.com/photos/44124413076@N01/11846265/ So we won’t build

    solutions for today’s problems and then nd ourselves surprised by what comes next.
  184. None
  185. Jason Grigsby @grigs • cloudfour.com Slides: bit.ly/ux-immobile-web ank You! Special

    thanks to Patrick H. Lauke, the Google TV team, Flickr users sharing under creative commons & the kind folks at Beaverton Video Only. http://www.flickr.com/photos/sualk61/4083223760/