Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fluid User Interface with Hardware Acceleration

Fluid User Interface with Hardware Acceleration

Talk at W3Conf 2013, San Francisco.

Ariya Hidayat

February 22, 2013
Tweet

More Decks by Ariya Hidayat

Other Decks in Programming

Transcript

  1. Fluid User Interface with
    Hardware Acceleration
    Ariya Hidayat
    San Francisco
    Feb 22, 2013
    @ariyahidayat
    1

    View full-size slide

  2. Magical Advice
    Use translate3d for
    hardware acceleration
    3

    View full-size slide

  3. Web Browser from 10,000 ft
    6

    View full-size slide

  4. Browser: High Level
    User Interface
    Browser Engine
    Graphics
    Stack
    Data Persistence
    Render Engine
    JavaScript
    Engine
    Networking
    I/O
    7

    View full-size slide

  5. From Contents to Pixels
    HTML
    Parser
    DOM
    CSS
    Parser
    HTML
    Style
    Sheets
    DOM
    Tree
    Style
    Rules
    Render
    Tree
    Render
    Layout
    Paint
    http://www.html5rocks.com/en/tutorials/internals/howbrowserswork
    8

    View full-size slide

  6. DOM Tree
    Hello
    HTMLDocument
    HTMLBodyElement
    HTMLParagraphElement attributes, children, contents
    9

    View full-size slide

  7. DOM Tree vs Render Tree
    HTMLDocument
    HTMLBodyElement
    HTMLParagraphElement
    RenderRoot
    RenderBody
    RenderText
    tree structure/navigation
    metrics (box model)
    hit testing
    RenderStyle
    computed style
    many:1
    10

    View full-size slide

  8. Style Recalc vs Layout
    document.getElementById('box').style.top = '100px';
    Aggregated “dirty” area
    document.getElementById('box').style.backgroundColor = 'red';
    No layout necessary,
    metrics are not flagged as “changed”
    11

    View full-size slide

  9. Minimizing Layout
    http://gent.ilcore.com/2011/03/how-not-to-trigger-layout-in-webkit.html
    http://code.google.com/p/chromium/source/search?q=%22-%3EupdateLayoutIgnorePendingStylesheets()%22
    clientHeight
    clientLeft
    clientTop
    clientWidth
    offsetHeight
    offsetLeft
    offsetParent
    offsetTop
    offsetWidth
    scrollLeft
    scrollTop
    scrollWidth
    innerText
    outerText
    Element
    focus()
    getBoundingClientRect()
    getClientRects()
    scrollByLines()
    scrollByPages()
    scrollHeight
    scrollIntoView()
    scrollIntoViewIfNeeded()
    12

    View full-size slide

  10. CSS Painting: 10 Stages
    http://www.w3.org/TR/CSS2/zindex.html
    Appendix E. Elaborate description of Stacking Contexts

    Background

    Floats

    Foreground

    Selection

    Outline
    Done by the render objects
    (in the render tree)
    13

    View full-size slide

  11. Graphics Stack: Crash Course
    14

    View full-size slide

  12. Path is Everything
    15

    View full-size slide

  13. Stroke = Outline
    Solid Dashed Dotted Textured
    16

    View full-size slide

  14. Brush = Fill
    Solid
    None Gradient Textured
    17

    View full-size slide

  15. Path Approximation
    Curves
    Polygon
    18

    View full-size slide

  16. Antialiasing
    19

    View full-size slide

  17. Transformation
    Scaling
    Rotation
    Perspective
    20

    View full-size slide

  18. Graphics Processing Unit
    “Fixed” geometry
    Transformation
    Textured triangles
    Parallelism
    21

    View full-size slide

  19. Optimized for Games
    http://epicgames.com/technology/epic-citadel http://www.rage.com/
    22

    View full-size slide

  20. GPU Workflow
    Vertices Rendered Textured
    Matrix
    23

    View full-size slide

  21. Web Page Rendering
    24

    View full-size slide

  22. Game vs Web Page
    Predictable contents (assets) ✔
    Mostly text

    Mostly images

    immediate
    Initial response
    25

    View full-size slide

  23. GPU Physical Limitations
    • Memory: Can’t store too much stuff
    • Speed: Quad-core CPU can do certain operations better
    • Bandwidth: Data transfer can be the bottleneck
    • Power: Electrons always need energy
    26

    View full-size slide

  24. Fluid Animation
    At the beginning, push as many resources
    as possible to the GPU
    During the animation, minimize
    the interaction between CPU-GPU
    1
    2
    27

    View full-size slide

  25. Immediate vs Retained
    draw the shape at (x, y)
    x = x + 10
    blit the buffer at (x, y)
    x = x + 10
    For every animation tick...
    At the beginning...
    draw the shape onto
    a buffer
    Off main thread
    28

    View full-size slide

  26. Animation Mechanics
    Initialization
    Animation step
    “Hey, this is good stuff. Cache it as texture #42.”
    “Apply [operation] to texture #42.”
    Animation loop
    29

    View full-size slide

  27. Element = Layer
    http://www.webkit.org/blog-files/leaves/
    30

    View full-size slide

  28. Compositing: “Logical” 3-D
    Compositor
    Tweening
    Actual operation
    31

    View full-size slide

  29. By Force
    transform: translateZ(0)
    Not needed for CSS animation!
    .someid {
    animation-name: somekeyframeanimation;
    animation-duration: 7s;
    transform: translateZ(0);
    }
    Don’t do that
    32

    View full-size slide

  30. Tools & Examples
    33

    View full-size slide

  31. Debugging in Safari
    defaults write com.apple.Safari IncludeInternalDebugMenu 1
    https://developer.apple.com/videos/wwdc/2012/?id=601
    34

    View full-size slide

  32. Compositing Indicator
    Composited layer
    Container layer
    No associated texture
    Overflow
    Texture (number = upload)
    35

    View full-size slide

  33. Texture Upload
    No reupload Upload
    Opacity
    Position
    Size
    Filter
    Everything else!
    “Hardware accelerated CSS”
    36

    View full-size slide

  34. Constant Upload = Bad
    http://codepen.io/ariya/full/xuwgy
    translate3d ???
    37

    View full-size slide

  35. Transformation
    http://ariya.github.com/css/spinningcube/ http://dev.sencha.com/animator/demos/cogs/
    38

    View full-size slide

  36. Fade In/Out
    http://dev.sencha.com/animator/demos/ions/
    http://codepen.io/ariya/full/spzad
    39

    View full-size slide

  37. three.js Periodic Table
    http://mrdoob.github.com/three.js/examples/css3d_periodictable.html
    40

    View full-size slide

  38. Montage MovieShow
    http://montagejs.github.com/montage/samples/popcorn/
    41

    View full-size slide

  39. Photon CSS Lighting
    http://photon.attasi.com
    42

    View full-size slide

  40. FPS View
    http://www.keithclark.co.uk/labs/css3-fps/
    43

    View full-size slide

  41. Filter
    http://html.adobe.com/webstandards/csscustomfilters/cssfilterlab/
    44

    View full-size slide

  42. Hacks & Tricks
    45

    View full-size slide

  43. Traffic Congestion
    The speed of the car
    vs the traffic condition
    46

    View full-size slide

  44. Avoid Overcapacity
    ....
    ....
    Texture upload
    Think of the GPU memory like a cache.
    47

    View full-size slide

  45. Prepare and Reuse
    Hide the poster (=layer) offscreen
    Viewport
    48

    View full-size slide

  46. Color Transition
    @keyframes box {
    0% { transform: background-color: blue; }
    100% { transform: background-color: green; }
    }
    Need a new texture uploaded to
    the GPU for every in-between color
    49

    View full-size slide

  47. Color Transition Trick
    Blended with
    http://codepen.io/ariya/full/ofDIh
    50

    View full-size slide

  48. Timer Latency
    Depending on
    user reaction
    Animation end triggers the JavaScript callback
    JavaScript code kicks the next animation
    Prepare both
    animation and hide the
    “wrong” one
    51

    View full-size slide

  49. More Stuff
    Jank Busters: http://www.youtube.com/watch?v=hAzhayTnhEI
    Continuous painting:
    http://updates.html5rocks.com/2013/02/Profiling-Long-Paint-Times-with-
    DevTools-Continuous-Painting-Mode
    Chrome GPU rendering benchmark (Telemetry):
    http://www.chromium.org/developers/design-documents/rendering-benchmarks
    Skia debugger:
    http://wesleyhales.com/blog/2013/02/18/Adventures-With-the-Skia-Debugger/
    52

    View full-size slide

  50. Final Words
    53

    View full-size slide

  51. http://www.trollquotes.org/619-super-spider-bat-troll-quote/
    54

    View full-size slide

  52. Use CSS3 animation & transition
    Be advised of texture uploads
    Never assume, always measure
    55

    View full-size slide

  53. There is No Silver Bullet
    Traditional graphics
    programming has been
    always full of tricks.
    It will always be.
    56

    View full-size slide

  54. Thank You
    [email protected]
    @AriyaHidayat
    ariya.ofilabs.com
    Background artwork from http://kde-look.org/content/show.php/Sin+Flow?content=139751 CC BY-NC-SA
    speakerdeck.com/ariya
    57

    View full-size slide