Upgrade to Pro — share decks privately, control downloads, hide ads and more …

WebKit Rendering: Overview for HTML5 Developer

WebKit Rendering: Overview for HTML5 Developer

WebKit is an open-source rendering engine for many popular desktop browsers, application run-times, and mobile devices. Do you ever wonder how WebKit pulls the bits off the server and converts them into visible pixels on the screen? Do you want to know how WebKit takes advantage of modern graphics system for hardware-accelerated animation and other effects? This session will show you a different take on WebKit and give you an insight on how to leverage WebKit to assist your HTML5 application development and performance workflow.

Ariya Hidayat

April 02, 2013
Tweet

More Decks by Ariya Hidayat

Other Decks in Programming

Transcript

  1. WebKit Rendering
    Overview for HTML5 Developer
    @ariyahidayat
    San Francisco, April 2, 2013
    1

    View full-size slide

  2. WebKit is Everywhere
    4

    View full-size slide

  3. KHTML, Apple, and Community
    0
    50000
    100000
    150000
    0 3 6 9 12
    Revisions
    Years
    ~3000 commits/month
    http://ariya.ofilabs.com/2011/11/one-hundred-thousand-and-counting.html
    5

    View full-size slide

  4. Heavy on TDD
    Source Code
    Tests
    Repository Checkout
    6

    View full-size slide

  5. Involvement: Social Layers
    Contributor
    Committer
    Reviewer
    accept or reject patches
    checks in reviewed patches
    after 10-20 patches
    after 80 patches
    7

    View full-size slide

  6. Big Players
    http://blog.bitergia.com/2013/02/06/report-on-the-activity-of-companies-in-the-webkit-project/
    http://blog.bitergia.com/2013/03/01/reviewers-and-companies-in-webkit-project/
    8

    View full-size slide

  7. Browser at a High Level
    User Interface
    Browser Engine
    Graphics
    Stack
    Data Persistence
    Render Engine
    JavaScript
    Engine
    Networking
    I/O
    http://www.html5rocks.com/en/tutorials/internals/howbrowserswork
    9

    View full-size slide

  8. WebKit Components
    Render Engine
    WebCore
    JavaScript Engine (JSC/V8)
    Client Interface
    HTML
    DOM
    CSS
    SVG
    Canvas
    10

    View full-size slide

  9. Platform Abstraction
    Client Interface
    Networking
    I/O
    Graphics
    Theme
    Events
    Clipboard
    Thread Geolocation Timer
    API Calls
    Events
    Port
    (Chrome, Safari, etc.)
    http://ariya.ofilabs.com/2011/06/your-webkit-port-is-special-just-like-every-other-port.html
    11

    View full-size slide

  10. Loader & Parser
    12

    View full-size slide

  11. From Contents to Pixels
    HTML
    Parser
    DOM
    CSS
    Parser
    HTML
    Style
    Sheets
    DOM
    Tree
    Style
    Rules
    Render
    Tree
    Render
    Layout
    Paint
    http://www.html5rocks.com/en/tutorials/internals/howbrowserswork
    13

    View full-size slide

  12. Loader: Resource, Document, Frame, Page
    https://www.webkit.org/blog/1188/how-webkit-loads-a-web-page/
    14

    View full-size slide

  13. Browser Caches
    http://gent.ilcore.com/2011/02/chromes-10-caches.html
    HTTP Disk Cache
    HTTP Memory Cache
    DNS Host Cache
    Preconnect Domain Cache
    V8 Compilation Cache
    SSL Session Cache
    TCP Connections
    Cookies
    HTML5 Caches
    ....
    15

    View full-size slide

  14. HTML Tokenizer
    http://www.whatwg.org/specs/web-apps/current-work/multipage/tokenization.html#tokenization
    ~70 states
    Hello
    tag
    text
    16

    View full-size slide

  15. HTML Parser
    https://www.webkit.org/blog/1273/the-html5-parsing-algorithm/
    ~10,000 lines of code
    Work in progress: Threaded HTML Parser
    Tokenizer
    Tree Builder
    DOM Tree
    Hello
    tag
    text
    17

    View full-size slide

  16. Preloader Scanner
    Schedule concurrent loading of resources
    CSS @import
    HTML , <img><br/>//ajax.googleapis.com/ajax/libs/ext-core/3/ext-core.js<br/>https://use.typekit.com/bqo8qzg.js<br/>/img/sencha-large.png<br/>/css/carousel.css<br/>/js/carousel.js<br/>http://cdn.sencha.io/img/home/promos/promos-20130328-bundle-webinar.png<br/>http://cdn.sencha.io/img/home/20130219-promo-senchacon.png<br/>http://cdn.sencha.io/img/home/december-2012/icon-news.png<br/>http://cdn.sencha.io/img/home/december-2012/icon-blog.png<br/>http://cdn.sencha.io/img/20130317-mz-pivot-grid.png<br/>http://cdn.sencha.io/img/20120607-whats-new-io.png<br/>http://cdn.sencha.io/img/20130317-development-enterprise.png<br/>http://cdn.sencha.io/img/home/december-2012/icon-twitter.png<br/>http://cdn.sencha.io/img/home/december-2012/customer-logos.png<br/>//platform.twitter.com/widgets.js<br/>/forum/clientscript/vbulletin_md5.js?v=403<br/>/min/?g=main-js&1362779803&debug=1<br/>sencha.com<br/>18<br/>

    View full-size slide

  17. DOM Tree
    Hello
    HTMLDocument
    HTMLBodyElement
    HTMLParagraphElement attributes, children, contents
    20

    View full-size slide

  18. DOM Tree vs Render Tree
    There is no Render* for display:none
    HTMLDocument
    HTMLBodyElement
    HTMLParagraphElement
    RenderRoot
    RenderBody
    RenderText
    tree structure/navigation
    metrics (box model)
    hit testing
    RenderStyle
    computed style
    many:1
    21

    View full-size slide

  19. “Attach” Process
    Historical: Node is invisible, attach it to the main view to make it visible
    42
    View
    RenderObject RenderStyle
    tree structure/navigation
    metrics (box model)
    hit testing
    computed style
    Node/Element
    1:1 many:1
    another tree structure
    22

    View full-size slide

  20. It’s a forest!
    23

    View full-size slide

  21. http://www.webkit.org/coding/dom-element-attach.html
    24

    View full-size slide

  22. Style Recalc vs Layout
    • RecalcStyle refers to style
    resolution, i.e. solving style
    properties for each node.
    • Layout needs style information,
    e.g. the computed dimension
    defines the box geometry.
    • Layout will be deferred if there
    is painting in progress (not
    completed yet).
    • Layout is on-demand: triggered
    by style change.
    More complication when loading
    data
    data
    data
    data
    RecalcStyle
    RecalcStyle
    Layout
    Paint
    RecalcStyle
    Layout
    Paint
    preorder traversal
    might be
    incremental
    RenderObject
    specific
    25

    View full-size slide

  23. Effects of Style Recalc and Layout
    document.getElementById('box').style.top = '100px';
    document.getElementById('box').style.backgroundColor = 'red';
    Aggregated “dirty” area
    No layout necessary,
    metrics are not flagged as “changed”
    26

    View full-size slide

  24. Getting the Right Style
    HTMLDocument
    HTMLBodyElement
    HTMLParagraphElement
    DOM Tree
    p { color: blue }
    Stylesheet List
    RenderStyle
    CSS Style Selector: id, class, tags, ...
    27

    View full-size slide

  25. Minimizing Layout
    http://gent.ilcore.com/2011/03/how-not-to-trigger-layout-in-webkit.html
    http://code.google.com/p/chromium/source/search?q=%22-%3EupdateLayoutIgnorePendingStylesheets()%22
    clientHeight
    clientLeft
    clientTop
    clientWidth
    offsetHeight
    offsetLeft
    offsetParent
    offsetTop
    offsetWidth
    scrollLeft
    scrollTop
    scrollWidth
    innerText
    outerText
    Element
    focus()
    getBoundingClientRect()
    getClientRects()
    scrollByLines()
    scrollByPages()
    scrollHeight
    scrollIntoView()
    scrollIntoViewIfNeeded()
    28

    View full-size slide

  26. Goals of Painting
    • “Visualize” all RenderObjects (again, tree traversal)
    • All painting commands go to GraphicsContext abstraction
    • Important
    • Clipped to the visible viewport
    • Back-to-front
    Optimizations
    Transparency &
    clipping
    30

    View full-size slide

  27. Graphics Abstraction
    Mac & iOS
    Chromium &
    Android Qt
    CoreGraphics
    Skia
    QPainter
    graphics stack
    GraphicsContext
    http://ariya.ofilabs.com/2011/06/your-webkit-port-is-special-just-like-every-other-port.html
    Gtk+
    Cairo
    31

    View full-size slide

  28. CSS Painting: 10 Stages
    http://www.w3.org/TR/CSS2/zindex.html
    Appendix E. Elaborate description of Stacking Contexts

    Background

    Floats

    Foreground

    Selection

    Outline
    Done by the render objects
    (in the render tree)
    32

    View full-size slide

  29. Painting Flow: Browser vs WebKit
    WebKit
    Browser
    Request painting, GraphicsContext
    User scrolls
    the viewport
    Style change
    Mark “dirty” area
    Sometimes to the
    backing store
    33

    View full-size slide

  30. Path is Everything
    34

    View full-size slide

  31. Stroke and Brush
    35

    View full-size slide

  32. Painting Complexity
    Polygon
    Multiple levels of transparency
    36

    View full-size slide

  33. Transformation
    Scaling
    Rotation
    Perspective
    37

    View full-size slide

  34. Hardware Acceleration
    38

    View full-size slide

  35. translate3d
    39

    View full-size slide

  36. SoC = System-on-a-Chip
    CPU GPU
    40

    View full-size slide

  37. Graphics Processing Unit
    “Fixed” geometry
    Transformation
    Textured triangles
    Parallelism
    41

    View full-size slide

  38. GPU Workflow
    Vertices Rendered Textured
    Matrix
    42

    View full-size slide

  39. Efficient Uses of GPU
    Drawing
    primitives
    Backing
    stores
    Layer &
    compositing
    43

    View full-size slide

  40. Responsive Interface
    Processor
    Rendering
    User interaction
    decoupled
    44

    View full-size slide

  41. Rendering vs Interaction
    Web Page
    Backing Store
    Screen
    Rendering
    User interaction
    46

    View full-size slide

  42. Checkerboard Pattern
    47

    View full-size slide

  43. Pinch to Zoom
    when you pinch...
    48

    View full-size slide

  44. Progressive Rendering
    Crisp but slow
    Fast but blurry
    49

    View full-size slide

  45. Tile Transformation
    Panning = Translation Zooming = Scaling
    50

    View full-size slide

  46. Perceived Responsiveness
    User pinches
    Smooth scaled
    Blocky (but fast!)
    51

    View full-size slide

  47. Tiling System
    CPU GPU
    ....
    ....
    Texture upload
    52

    View full-size slide

  48. Per Element Backing Store = Layer
    http://techblog.netflix.com/2012/01/webkit-in-your-living-room.html
    http://aerotwist.com/blog/on-translate3d-and-layer-creation-hacks/
    3-D transform/perspective
    , , plugins
    CSS animation, filters
    ....
    53

    View full-size slide

  49. Mechanics of Animation
    Initialization
    Animation step
    “Hey, this is good stuff. Cache it as texture #42.”
    “Apply [operation] to texture #42.”
    54

    View full-size slide

  50. Element and “Logical” 3-D
    http://www.webkit.org/blog-files/leaves/
    Compositor
    Tweening
    Can be in a separate thread
    55

    View full-size slide

  51. Debugging in Safari
    defaults write com.apple.Safari IncludeInternalDebugMenu 1
    56

    View full-size slide

  52. Compositing Indicators
    Composited layer
    Container layer
    No associated texture
    Overflow
    Texture (number = upload)
    57

    View full-size slide

  53. Frame Rate HUD
    http://ariya.ofilabs.com/2013/03/frame-rate-hud-on-chrome-for-android.html
    Frame rate chart
    GPU memory usage
    58

    View full-size slide

  54. Avoid Texture Reupload
    No reupload Upload
    Opacity
    Position
    Size
    Filter
    Everything else!
    “Hardware accelerated CSS”
    59

    View full-size slide

  55. Constant Upload = Bad
    http://codepen.io/ariya/full/xuwgy
    translate3d ???
    60

    View full-size slide

  56. Transformation
    http://ariya.github.com/css/spinningcube/ http://dev.sencha.com/animator/demos/cogs/
    61

    View full-size slide

  57. Fade In/Out
    http://dev.sencha.com/animator/demos/ions/
    http://codepen.io/ariya/full/spzad
    62

    View full-size slide

  58. three.js Periodic Table
    http://mrdoob.github.com/three.js/examples/css3d_periodictable.html
    63

    View full-size slide

  59. Montage MovieShow
    http://montagejs.github.com/montage/samples/popcorn/
    64

    View full-size slide

  60. Photon CSS Lighting
    http://photon.attasi.com
    65

    View full-size slide

  61. FPS View
    http://www.keithclark.co.uk/labs/css3-fps/
    66

    View full-size slide

  62. Filter
    http://html.adobe.com/webstandards/csscustomfilters/cssfilterlab/
    67

    View full-size slide

  63. Timer Latency
    Depending on
    user reaction
    Animation end triggers the JavaScript callback
    JavaScript code kicks the next animation
    Prepare both
    animation and hide the
    “wrong” one
    68

    View full-size slide

  64. Prepare and Reuse
    Hide the poster (=layer) offscreen
    Viewport
    69

    View full-size slide

  65. Avoid Overcapacity
    ....
    ....
    Texture upload
    Think of the GPU memory like a cache.
    70

    View full-size slide

  66. Color Transition
    @keyframes box {
    0% { transform: background-color: blue; }
    100% { transform: background-color: green; }
    }
    Need a new texture uploaded to
    the GPU for every in-between color
    71

    View full-size slide

  67. Color Transition Trick
    Blended with
    http://codepen.io/ariya/full/ofDIh
    72

    View full-size slide

  68. Traffic Congestion
    The speed of the car vs the traffic condition
    73

    View full-size slide

  69. Conclusion
    74

    View full-size slide

  70. Rendering Aspects
    75

    View full-size slide

  71. Run a thorough feature-cost analysis
    Don’t fight the rendering engine
    Never assume, always measure
    76

    View full-size slide

  72. More Stuff
    Jank Busters: http://www.youtube.com/watch?v=hAzhayTnhEI
    Continuous painting:
    http://updates.html5rocks.com/2013/02/Profiling-Long-Paint-Times-with-
    DevTools-Continuous-Painting-Mode
    Chrome GPU rendering benchmark (Telemetry):
    http://www.chromium.org/developers/design-documents/rendering-benchmarks
    Skia debugger:
    http://wesleyhales.com/blog/2013/02/18/Adventures-With-the-Skia-Debugger/
    77

    View full-size slide

  73. Thank You
    [email protected]
    @AriyaHidayat
    ariya.ofilabs.com
    speakerdeck.com/ariya
    Credits: Some artworks are from http://openclipart.org/user-cliparts/nicubunu
    78

    View full-size slide