Fluid User Interface with Hardware Acceleration

Fluid User Interface with Hardware Acceleration

Talk at W3Conf 2013, San Francisco.

0284b8950e0f4a57bcc092d4dbb98d97?s=128

Ariya Hidayat

February 22, 2013
Tweet

Transcript

  1. Fluid User Interface with Hardware Acceleration Ariya Hidayat San Francisco

    Feb 22, 2013 @ariyahidayat 1
  2. 2

  3. Magical Advice Use translate3d for hardware acceleration 3

  4. whoami 4

  5. CSS3 5

  6. Web Browser from 10,000 ft 6

  7. Browser: High Level User Interface Browser Engine Graphics Stack Data

    Persistence Render Engine JavaScript Engine Networking I/O 7
  8. From Contents to Pixels HTML Parser DOM CSS Parser HTML

    Style Sheets DOM Tree Style Rules Render Tree Render Layout Paint http://www.html5rocks.com/en/tutorials/internals/howbrowserswork 8
  9. DOM Tree <html><body><p>Hello</p></body></html> HTMLDocument HTMLBodyElement HTMLParagraphElement attributes, children, contents 9

  10. DOM Tree vs Render Tree HTMLDocument HTMLBodyElement HTMLParagraphElement RenderRoot RenderBody

    RenderText tree structure/navigation metrics (box model) hit testing RenderStyle computed style many:1 10
  11. Style Recalc vs Layout document.getElementById('box').style.top = '100px'; Aggregated “dirty” area

    document.getElementById('box').style.backgroundColor = 'red'; No layout necessary, metrics are not flagged as “changed” 11
  12. Minimizing Layout http://gent.ilcore.com/2011/03/how-not-to-trigger-layout-in-webkit.html http://code.google.com/p/chromium/source/search?q=%22-%3EupdateLayoutIgnorePendingStylesheets()%22 clientHeight clientLeft clientTop clientWidth offsetHeight offsetLeft

    offsetParent offsetTop offsetWidth scrollLeft scrollTop scrollWidth innerText outerText Element focus() getBoundingClientRect() getClientRects() scrollByLines() scrollByPages() scrollHeight scrollIntoView() scrollIntoViewIfNeeded() 12
  13. CSS Painting: 10 Stages http://www.w3.org/TR/CSS2/zindex.html Appendix E. Elaborate description of

    Stacking Contexts • Background • Floats • Foreground • Selection • Outline Done by the render objects (in the render tree) 13
  14. Graphics Stack: Crash Course 14

  15. Path is Everything 15

  16. Stroke = Outline Solid Dashed Dotted Textured 16

  17. Brush = Fill Solid None Gradient Textured 17

  18. Path Approximation Curves Polygon 18

  19. Antialiasing 19

  20. Transformation Scaling Rotation Perspective 20

  21. Graphics Processing Unit “Fixed” geometry Transformation Textured triangles Parallelism 21

  22. Optimized for Games http://epicgames.com/technology/epic-citadel http://www.rage.com/ 22

  23. GPU Workflow Vertices Rendered Textured Matrix 23

  24. Web Page Rendering 24

  25. Game vs Web Page Predictable contents (assets) ✔ Mostly text

    ✔ Mostly images ✔ immediate Initial response 25
  26. GPU Physical Limitations • Memory: Can’t store too much stuff

    • Speed: Quad-core CPU can do certain operations better • Bandwidth: Data transfer can be the bottleneck • Power: Electrons always need energy 26
  27. Fluid Animation At the beginning, push as many resources as

    possible to the GPU During the animation, minimize the interaction between CPU-GPU 1 2 27
  28. Immediate vs Retained draw the shape at (x, y) x

    = x + 10 blit the buffer at (x, y) x = x + 10 For every animation tick... At the beginning... draw the shape onto a buffer Off main thread 28
  29. Animation Mechanics Initialization Animation step “Hey, this is good stuff.

    Cache it as texture #42.” “Apply [operation] to texture #42.” Animation loop 29
  30. Element = Layer http://www.webkit.org/blog-files/leaves/ 30

  31. Compositing: “Logical” 3-D Compositor Tweening Actual operation 31

  32. By Force transform: translateZ(0) Not needed for CSS animation! .someid

    { animation-name: somekeyframeanimation; animation-duration: 7s; transform: translateZ(0); } Don’t do that 32
  33. Tools & Examples 33

  34. Debugging in Safari defaults write com.apple.Safari IncludeInternalDebugMenu 1 https://developer.apple.com/videos/wwdc/2012/?id=601 34

  35. Compositing Indicator Composited layer Container layer No associated texture Overflow

    Texture (number = upload) 35
  36. Texture Upload No reupload Upload Opacity Position Size Filter Everything

    else! “Hardware accelerated CSS” 36
  37. Constant Upload = Bad http://codepen.io/ariya/full/xuwgy translate3d ??? 37

  38. Transformation http://ariya.github.com/css/spinningcube/ http://dev.sencha.com/animator/demos/cogs/ 38

  39. Fade In/Out http://dev.sencha.com/animator/demos/ions/ http://codepen.io/ariya/full/spzad 39

  40. three.js Periodic Table http://mrdoob.github.com/three.js/examples/css3d_periodictable.html 40

  41. Montage MovieShow http://montagejs.github.com/montage/samples/popcorn/ 41

  42. Photon CSS Lighting http://photon.attasi.com 42

  43. FPS View http://www.keithclark.co.uk/labs/css3-fps/ 43

  44. Filter http://html.adobe.com/webstandards/csscustomfilters/cssfilterlab/ 44

  45. Hacks & Tricks 45

  46. Traffic Congestion The speed of the car vs the traffic

    condition 46
  47. Avoid Overcapacity .... .... Texture upload Think of the GPU

    memory like a cache. 47
  48. Prepare and Reuse Hide the poster (=layer) offscreen Viewport 48

  49. Color Transition @keyframes box { 0% { transform: background-color: blue;

    } 100% { transform: background-color: green; } } Need a new texture uploaded to the GPU for every in-between color 49
  50. Color Transition Trick Blended with http://codepen.io/ariya/full/ofDIh 50

  51. Timer Latency Depending on user reaction Animation end triggers the

    JavaScript callback JavaScript code kicks the next animation Prepare both animation and hide the “wrong” one 51
  52. More Stuff Jank Busters: http://www.youtube.com/watch?v=hAzhayTnhEI Continuous painting: http://updates.html5rocks.com/2013/02/Profiling-Long-Paint-Times-with- DevTools-Continuous-Painting-Mode Chrome

    GPU rendering benchmark (Telemetry): http://www.chromium.org/developers/design-documents/rendering-benchmarks Skia debugger: http://wesleyhales.com/blog/2013/02/18/Adventures-With-the-Skia-Debugger/ 52
  53. Final Words 53

  54. http://www.trollquotes.org/619-super-spider-bat-troll-quote/ 54

  55. Use CSS3 animation & transition Be advised of texture uploads

    Never assume, always measure 55
  56. There is No Silver Bullet Traditional graphics programming has been

    always full of tricks. It will always be. 56
  57. Thank You ariya.hidayat@gmail.com @AriyaHidayat ariya.ofilabs.com Background artwork from http://kde-look.org/content/show.php/Sin+Flow?content=139751 CC

    BY-NC-SA speakerdeck.com/ariya 57