Upgrade to Pro — share decks privately, control downloads, hide ads and more …

WebKit Rendering: Overview for HTML5 Developer

WebKit Rendering: Overview for HTML5 Developer

WebKit is an open-source rendering engine for many popular desktop browsers, application run-times, and mobile devices. Do you ever wonder how WebKit pulls the bits off the server and converts them into visible pixels on the screen? Do you want to know how WebKit takes advantage of modern graphics system for hardware-accelerated animation and other effects? This session will show you a different take on WebKit and give you an insight on how to leverage WebKit to assist your HTML5 application development and performance workflow.

Ariya Hidayat

April 02, 2013
Tweet

More Decks by Ariya Hidayat

Other Decks in Programming

Transcript

  1. WebKit Rendering
    Overview for HTML5 Developer
    @ariyahidayat
    San Francisco, April 2, 2013
    1

    View Slide

  2. whoami
    2

    View Slide

  3. WebKit
    3

    View Slide

  4. WebKit is Everywhere
    4

    View Slide

  5. KHTML, Apple, and Community
    0
    50000
    100000
    150000
    0 3 6 9 12
    Revisions
    Years
    ~3000 commits/month
    http://ariya.ofilabs.com/2011/11/one-hundred-thousand-and-counting.html
    5

    View Slide

  6. Heavy on TDD
    Source Code
    Tests
    Repository Checkout
    6

    View Slide

  7. Involvement: Social Layers
    Contributor
    Committer
    Reviewer
    accept or reject patches
    checks in reviewed patches
    after 10-20 patches
    after 80 patches
    7

    View Slide

  8. Big Players
    http://blog.bitergia.com/2013/02/06/report-on-the-activity-of-companies-in-the-webkit-project/
    http://blog.bitergia.com/2013/03/01/reviewers-and-companies-in-webkit-project/
    8

    View Slide

  9. Browser at a High Level
    User Interface
    Browser Engine
    Graphics
    Stack
    Data Persistence
    Render Engine
    JavaScript
    Engine
    Networking
    I/O
    http://www.html5rocks.com/en/tutorials/internals/howbrowserswork
    9

    View Slide

  10. WebKit Components
    Render Engine
    WebCore
    JavaScript Engine (JSC/V8)
    Client Interface
    HTML
    DOM
    CSS
    SVG
    Canvas
    10

    View Slide

  11. Platform Abstraction
    Client Interface
    Networking
    I/O
    Graphics
    Theme
    Events
    Clipboard
    Thread Geolocation Timer
    API Calls
    Events
    Port
    (Chrome, Safari, etc.)
    http://ariya.ofilabs.com/2011/06/your-webkit-port-is-special-just-like-every-other-port.html
    11

    View Slide

  12. Loader & Parser
    12

    View Slide

  13. From Contents to Pixels
    HTML
    Parser
    DOM
    CSS
    Parser
    HTML
    Style
    Sheets
    DOM
    Tree
    Style
    Rules
    Render
    Tree
    Render
    Layout
    Paint
    http://www.html5rocks.com/en/tutorials/internals/howbrowserswork
    13

    View Slide

  14. Loader: Resource, Document, Frame, Page
    https://www.webkit.org/blog/1188/how-webkit-loads-a-web-page/
    14

    View Slide

  15. Browser Caches
    http://gent.ilcore.com/2011/02/chromes-10-caches.html
    HTTP Disk Cache
    HTTP Memory Cache
    DNS Host Cache
    Preconnect Domain Cache
    V8 Compilation Cache
    SSL Session Cache
    TCP Connections
    Cookies
    HTML5 Caches
    ....
    15

    View Slide

  16. HTML Tokenizer
    http://www.whatwg.org/specs/web-apps/current-work/multipage/tokenization.html#tokenization
    ~70 states
    Hello
    tag
    text
    16

    View Slide

  17. HTML Parser
    https://www.webkit.org/blog/1273/the-html5-parsing-algorithm/
    ~10,000 lines of code
    Work in progress: Threaded HTML Parser
    Tokenizer
    Tree Builder
    DOM Tree
    Hello
    tag
    text
    17

    View Slide

  18. Preloader Scanner
    Schedule concurrent loading of resources
    CSS @import
    HTML , <img><br/>//ajax.googleapis.com/ajax/libs/ext-core/3/ext-core.js<br/>https://use.typekit.com/bqo8qzg.js<br/>/img/sencha-large.png<br/>/css/carousel.css<br/>/js/carousel.js<br/>http://cdn.sencha.io/img/home/promos/promos-20130328-bundle-webinar.png<br/>http://cdn.sencha.io/img/home/20130219-promo-senchacon.png<br/>http://cdn.sencha.io/img/home/december-2012/icon-news.png<br/>http://cdn.sencha.io/img/home/december-2012/icon-blog.png<br/>http://cdn.sencha.io/img/20130317-mz-pivot-grid.png<br/>http://cdn.sencha.io/img/20120607-whats-new-io.png<br/>http://cdn.sencha.io/img/20130317-development-enterprise.png<br/>http://cdn.sencha.io/img/home/december-2012/icon-twitter.png<br/>http://cdn.sencha.io/img/home/december-2012/customer-logos.png<br/>//platform.twitter.com/widgets.js<br/>/forum/clientscript/vbulletin_md5.js?v=403<br/>/min/?g=main-js&1362779803&debug=1<br/>sencha.com<br/>18<br/>

    View Slide

  19. Layout
    19

    View Slide

  20. DOM Tree
    Hello
    HTMLDocument
    HTMLBodyElement
    HTMLParagraphElement attributes, children, contents
    20

    View Slide

  21. DOM Tree vs Render Tree
    There is no Render* for display:none
    HTMLDocument
    HTMLBodyElement
    HTMLParagraphElement
    RenderRoot
    RenderBody
    RenderText
    tree structure/navigation
    metrics (box model)
    hit testing
    RenderStyle
    computed style
    many:1
    21

    View Slide

  22. “Attach” Process
    Historical: Node is invisible, attach it to the main view to make it visible
    42
    View
    RenderObject RenderStyle
    tree structure/navigation
    metrics (box model)
    hit testing
    computed style
    Node/Element
    1:1 many:1
    another tree structure
    22

    View Slide

  23. It’s a forest!
    23

    View Slide

  24. http://www.webkit.org/coding/dom-element-attach.html
    24

    View Slide

  25. Style Recalc vs Layout
    • RecalcStyle refers to style
    resolution, i.e. solving style
    properties for each node.
    • Layout needs style information,
    e.g. the computed dimension
    defines the box geometry.
    • Layout will be deferred if there
    is painting in progress (not
    completed yet).
    • Layout is on-demand: triggered
    by style change.
    More complication when loading
    data
    data
    data
    data
    RecalcStyle
    RecalcStyle
    Layout
    Paint
    RecalcStyle
    Layout
    Paint
    preorder traversal
    might be
    incremental
    RenderObject
    specific
    25

    View Slide

  26. Effects of Style Recalc and Layout
    document.getElementById('box').style.top = '100px';
    document.getElementById('box').style.backgroundColor = 'red';
    Aggregated “dirty” area
    No layout necessary,
    metrics are not flagged as “changed”
    26

    View Slide

  27. Getting the Right Style
    HTMLDocument
    HTMLBodyElement
    HTMLParagraphElement
    DOM Tree
    p { color: blue }
    Stylesheet List
    RenderStyle
    CSS Style Selector: id, class, tags, ...
    27

    View Slide

  28. Minimizing Layout
    http://gent.ilcore.com/2011/03/how-not-to-trigger-layout-in-webkit.html
    http://code.google.com/p/chromium/source/search?q=%22-%3EupdateLayoutIgnorePendingStylesheets()%22
    clientHeight
    clientLeft
    clientTop
    clientWidth
    offsetHeight
    offsetLeft
    offsetParent
    offsetTop
    offsetWidth
    scrollLeft
    scrollTop
    scrollWidth
    innerText
    outerText
    Element
    focus()
    getBoundingClientRect()
    getClientRects()
    scrollByLines()
    scrollByPages()
    scrollHeight
    scrollIntoView()
    scrollIntoViewIfNeeded()
    28

    View Slide

  29. Painting
    29

    View Slide

  30. Goals of Painting
    • “Visualize” all RenderObjects (again, tree traversal)
    • All painting commands go to GraphicsContext abstraction
    • Important
    • Clipped to the visible viewport
    • Back-to-front
    Optimizations
    Transparency &
    clipping
    30

    View Slide

  31. Graphics Abstraction
    Mac & iOS
    Chromium &
    Android Qt
    CoreGraphics
    Skia
    QPainter
    graphics stack
    GraphicsContext
    http://ariya.ofilabs.com/2011/06/your-webkit-port-is-special-just-like-every-other-port.html
    Gtk+
    Cairo
    31

    View Slide

  32. CSS Painting: 10 Stages
    http://www.w3.org/TR/CSS2/zindex.html
    Appendix E. Elaborate description of Stacking Contexts

    Background

    Floats

    Foreground

    Selection

    Outline
    Done by the render objects
    (in the render tree)
    32

    View Slide

  33. Painting Flow: Browser vs WebKit
    WebKit
    Browser
    Request painting, GraphicsContext
    User scrolls
    the viewport
    Style change
    Mark “dirty” area
    Sometimes to the
    backing store
    33

    View Slide

  34. Path is Everything
    34

    View Slide

  35. Stroke and Brush
    35

    View Slide

  36. Painting Complexity
    Polygon
    Multiple levels of transparency
    36

    View Slide

  37. Transformation
    Scaling
    Rotation
    Perspective
    37

    View Slide

  38. Hardware Acceleration
    38

    View Slide

  39. translate3d
    39

    View Slide

  40. SoC = System-on-a-Chip
    CPU GPU
    40

    View Slide

  41. Graphics Processing Unit
    “Fixed” geometry
    Transformation
    Textured triangles
    Parallelism
    41

    View Slide

  42. GPU Workflow
    Vertices Rendered Textured
    Matrix
    42

    View Slide

  43. Efficient Uses of GPU
    Drawing
    primitives
    Backing
    stores
    Layer &
    compositing
    43

    View Slide

  44. Responsive Interface
    Processor
    Rendering
    User interaction
    decoupled
    44

    View Slide

  45. Maps
    Tile
    45

    View Slide

  46. Rendering vs Interaction
    Web Page
    Backing Store
    Screen
    Rendering
    User interaction
    46

    View Slide

  47. Checkerboard Pattern
    47

    View Slide

  48. Pinch to Zoom
    when you pinch...
    48

    View Slide

  49. Progressive Rendering
    Crisp but slow
    Fast but blurry
    49

    View Slide

  50. Tile Transformation
    Panning = Translation Zooming = Scaling
    50

    View Slide

  51. Perceived Responsiveness
    User pinches
    Smooth scaled
    Blocky (but fast!)
    51

    View Slide

  52. Tiling System
    CPU GPU
    ....
    ....
    Texture upload
    52

    View Slide

  53. Per Element Backing Store = Layer
    http://techblog.netflix.com/2012/01/webkit-in-your-living-room.html
    http://aerotwist.com/blog/on-translate3d-and-layer-creation-hacks/
    3-D transform/perspective
    , , plugins
    CSS animation, filters
    ....
    53

    View Slide

  54. Mechanics of Animation
    Initialization
    Animation step
    “Hey, this is good stuff. Cache it as texture #42.”
    “Apply [operation] to texture #42.”
    54

    View Slide

  55. Element and “Logical” 3-D
    http://www.webkit.org/blog-files/leaves/
    Compositor
    Tweening
    Can be in a separate thread
    55

    View Slide

  56. Debugging in Safari
    defaults write com.apple.Safari IncludeInternalDebugMenu 1
    56

    View Slide

  57. Compositing Indicators
    Composited layer
    Container layer
    No associated texture
    Overflow
    Texture (number = upload)
    57

    View Slide

  58. Frame Rate HUD
    http://ariya.ofilabs.com/2013/03/frame-rate-hud-on-chrome-for-android.html
    Frame rate chart
    GPU memory usage
    58

    View Slide

  59. Avoid Texture Reupload
    No reupload Upload
    Opacity
    Position
    Size
    Filter
    Everything else!
    “Hardware accelerated CSS”
    59

    View Slide

  60. Constant Upload = Bad
    http://codepen.io/ariya/full/xuwgy
    translate3d ???
    60

    View Slide

  61. Transformation
    http://ariya.github.com/css/spinningcube/ http://dev.sencha.com/animator/demos/cogs/
    61

    View Slide

  62. Fade In/Out
    http://dev.sencha.com/animator/demos/ions/
    http://codepen.io/ariya/full/spzad
    62

    View Slide

  63. three.js Periodic Table
    http://mrdoob.github.com/three.js/examples/css3d_periodictable.html
    63

    View Slide

  64. Montage MovieShow
    http://montagejs.github.com/montage/samples/popcorn/
    64

    View Slide

  65. Photon CSS Lighting
    http://photon.attasi.com
    65

    View Slide

  66. FPS View
    http://www.keithclark.co.uk/labs/css3-fps/
    66

    View Slide

  67. Filter
    http://html.adobe.com/webstandards/csscustomfilters/cssfilterlab/
    67

    View Slide

  68. Timer Latency
    Depending on
    user reaction
    Animation end triggers the JavaScript callback
    JavaScript code kicks the next animation
    Prepare both
    animation and hide the
    “wrong” one
    68

    View Slide

  69. Prepare and Reuse
    Hide the poster (=layer) offscreen
    Viewport
    69

    View Slide

  70. Avoid Overcapacity
    ....
    ....
    Texture upload
    Think of the GPU memory like a cache.
    70

    View Slide

  71. Color Transition
    @keyframes box {
    0% { transform: background-color: blue; }
    100% { transform: background-color: green; }
    }
    Need a new texture uploaded to
    the GPU for every in-between color
    71

    View Slide

  72. Color Transition Trick
    Blended with
    http://codepen.io/ariya/full/ofDIh
    72

    View Slide

  73. Traffic Congestion
    The speed of the car vs the traffic condition
    73

    View Slide

  74. Conclusion
    74

    View Slide

  75. Rendering Aspects
    75

    View Slide

  76. Run a thorough feature-cost analysis
    Don’t fight the rendering engine
    Never assume, always measure
    76

    View Slide

  77. More Stuff
    Jank Busters: http://www.youtube.com/watch?v=hAzhayTnhEI
    Continuous painting:
    http://updates.html5rocks.com/2013/02/Profiling-Long-Paint-Times-with-
    DevTools-Continuous-Painting-Mode
    Chrome GPU rendering benchmark (Telemetry):
    http://www.chromium.org/developers/design-documents/rendering-benchmarks
    Skia debugger:
    http://wesleyhales.com/blog/2013/02/18/Adventures-With-the-Skia-Debugger/
    77

    View Slide

  78. Thank You
    [email protected]
    @AriyaHidayat
    ariya.ofilabs.com
    speakerdeck.com/ariya
    Credits: Some artworks are from http://openclipart.org/user-cliparts/nicubunu
    78

    View Slide