WebKit In Your Living Room

From 2011, describes development challenges and approaches faced while building high quality user interfaces on embedded devices.

Video: https://vimeo.com/85376597

Transcript

1. WebKit in you r living room Matt Seeley, Net fl ix UI Engineer
2. ‘Yes’ I do watch movies all day long* * Actually, ‘no.’ Just after a big lunch. @innerhtml
3. Native + Web
4. 450+ devices in 40+ countries * As of Q4 2011
5. CPU GPU Main memory Video memory Graphics driver CPU architecture Memory bus speed Network stack Devices capabilities vary
6. CPU GPU Main memory Video memory Graphics driver CPU architecture Memory bus speed Network stack Devices capabilities vary
7. • Device performance tip s • WebKit rendering basics • Accelerated compositing Topics:
8. Respond to user input
10. Gallery
11. Bob
12. Video Input comes at various speeds
13. Video Sometimes input can come too fast
14. • Recycling of previous ro w • Load data or retrieve from cach e • XMLHttpReques t • JSON.parse( ) • DOM access for each bo x • Paint the row Activating a row could b e expensive, defer it
15. Video Defer & stagger activation
16. var activateTimeout; // @private function activateSync (/* id1 [, ..., idN]*/) { // perform greedy operation } // @public function activate (ids) { if (cfg.delay) { clearTimeout(activateTimeout); activateTimeout = setTimeout(function () { activateSync.apply(null, ids.splice(0, cfg.size)); if (ids.length) { activate(ids); } }, cfg.delay); // increase delay to stagger } else { activateSync.apply(null, ids); } } Work deferral & chunking
17. Fast Slow JSON.pars e XMLHttpReques t document. * element.* Cross this divide with care mostEverythingElse
18. Minding our memory
19. Total memory - Host processes - Netflix SDK - WebKit - Network buffer - Video buffer UI budget Understand the memory requirements of your components
20. Video memory contain s uncompressed images 1920 x 1080 8.3 MB 1280 x 720 3.7 MB
21. Memory saving techniques • Avoid unbounded growt h • Minimize number of throwaway object s • display:none; may clear video memor y • Pool elements and objects
22. Video Reuse instead of allocate & destroy
23. All that glitters...
24. Don’t do i n software what can b e handled by hardware.
25. -webkit-gradien t -webkit-box-shado w background-positio n background-repea t border Static images render faste r than CSS effects
27. Use CSS effects for all transitions • Only use transitions on capable device s • Declarative, easy, supports multiple transform s • Rendering engine may optimiz e • Animations which don’t require repaints are hardware accelerated, ie: 
 -webkit-transition-property: 
 opacity, -webkit-transform;
28. Open sourced, hackable, port-able
29. Browser or Toolkit Networking, Graphics, Threading , Memory management, Disk access, etc... WebCore JavascriptCore
30. WebCore DOM tree construction Painting of render tree Parsing Parsing Render tree construction Layout o f render tree Render fl ow (abbreviated)
31. Toolkit or Browser port RenderLayer Layout & Paint GraphicsContext RenderLayer, one or more RenderObjects RenderObject sub-class
32. Accelerated compositing 1
33. Compositing laye r RenderLayer(s) fl attened into a single backing surface (bitmap); sometimes mapped to a GPU texture
34. Re d A compositing layer; number tracks number of recalculations . Yello w A container layer; no backing surface - children are layers . Cya n Overflow box; no backing surface - debugging tool . Gree n Tiled layer; created when layer width or height is > 1024px. Safari’s visual hints
35. defaults write com.apple.Safari \ IncludeInternalDebugMenu 1 Enable Safari’s Debug menu
36. Enable compositing border s in Chrome
37. • opacity • overflow • -webkit-transform • z-index • positio n • Element’s box overlaps a compositing layer Layer creation triggers
38. virtual bool requiresLayer() const { return isRoot() || isPositioned() || isRelPositioned() || isTransparent() || hasOverflowClip() || hasTransform() || hasMask() || hasReflection() || style()->specifiesColumns() ; } WebCore/rendering/RenderBoxModelObject.h Explicit layer creation
39. WebCore/rendering/RenderLayerCompositor.cpp Implicit layer creation bool RenderLayerCompositor::needsToBeComposited(const RenderLayer* layer) const { // edited.. . return requiresCompositingLayer(layer) || layer->mustOverlapCompositedLayers() || (inCompositingMode() && layer->isRootLayer()) ; } bool RenderLayerCompositor::requiresCompositingLayer(const RenderLayer* layer) const { RenderObject* renderer = layer->renderer() ; // edited... return requiresCompositingForTransform(renderer) || requiresCompositingForTransformingForVideo(renderer) || requiresCompositingForCanvas(renderer) || requiresCompositingForPlugin(renderer) || requiresCompositingForFrame(renderer) || (canRender3DTransforms() && /* edited */) || clipsCompositingDescendants(layer) || requiresCompositingForAnimation(renderer) || requiresCompositingForFullScreen(renderer) ; }
40. A (short) tale of two layers Animated gif with translateZ(0); Static gif wit h keyframes animation
41. translateZ(0) == zoom:1
42. • Change opacity • Change -webkit-transform • Hide layer offscreen • Change conten t • Change CSS box propertie s • Hide text content using text-indent • Hide layer with display* Safe Caution * May release video memory
43. <div class="listRow"> <h2 class="listTitle"></h2> <div class="listItems"> <div class="ct ct-seq0"></div > ... </div> </div> Up & down layer composition
44. 1 container laye r 2 compositing layers Up & down layer composition
45. 2 container layer s 7+ compositing layers Left & right layer composition
46. .listRow { -webkit-transform: translateZ(0); } .listTitle { -webkit-transform: translateZ(0); } .listItems { -webkit-transform: translate(0, 30px); } .ct-seq0 { -webkit-transform: translateX(60px); } .listRow-animate-lr .ct-seq0 { /* Use Z to punch .ct-seq# into layers */ -webkit-transform: translateX(60px) translateZ(0); }
47. • Reduce the quantity of layer s • Keep layers as small as possibl e • Update layers infrequentl y • Tailor layer composition to purpos e • Trial and error; testing is important Accelerated compositing tips
48. mseeley@net fl ix @innerhtml