WebKit Rendering: Overview for HTML5 Developer

WebKit is an open-source rendering engine for many popular desktop browsers, application run-times, and mobile devices. Do you ever wonder how WebKit pulls the bits off the server and converts them into visible pixels on the screen? Do you want to know how WebKit takes advantage of modern graphics system for hardware-accelerated animation and other effects? This session will show you a different take on WebKit and give you an insight on how to leverage WebKit to assist your HTML5 application development and performance workflow.

Transcript

1. WebKit Rendering
   Overview for HTML5 Developer
   @ariyahidayat
   San Francisco, April 2, 2013
   1
   

   View Slide

2. whoami
   2
   

   View Slide

3. WebKit
   3
   

   View Slide

4. WebKit is Everywhere
   4
   

   View Slide

5. KHTML, Apple, and Community
   0
   50000
   100000
   150000
   0 3 6 9 12
   Revisions
   Years
   ~3000 commits/month
   http://ariya.ofilabs.com/2011/11/one-hundred-thousand-and-counting.html
   5
   

   View Slide

6. Heavy on TDD
   Source Code
   Tests
   Repository Checkout
   6
   

   View Slide

7. Involvement: Social Layers
   Contributor
   Committer
   Reviewer
   accept or reject patches
   checks in reviewed patches
   after 10-20 patches
   after 80 patches
   7
   

   View Slide

8. Big Players
   http://blog.bitergia.com/2013/02/06/report-on-the-activity-of-companies-in-the-webkit-project/
   http://blog.bitergia.com/2013/03/01/reviewers-and-companies-in-webkit-project/
   8
   

   View Slide

9. Browser at a High Level
   User Interface
   Browser Engine
   Graphics
   Stack
   Data Persistence
   Render Engine
   JavaScript
   Engine
   Networking
   I/O
   http://www.html5rocks.com/en/tutorials/internals/howbrowserswork
   9
   

   View Slide

10. WebKit Components
    Render Engine
    WebCore
    JavaScript Engine (JSC/V8)
    Client Interface
    HTML
    DOM
    CSS
    SVG
    Canvas
    10
    

    View Slide

11. Platform Abstraction
    Client Interface
    Networking
    I/O
    Graphics
    Theme
    Events
    Clipboard
    Thread Geolocation Timer
    API Calls
    Events
    Port
    (Chrome, Safari, etc.)
    http://ariya.ofilabs.com/2011/06/your-webkit-port-is-special-just-like-every-other-port.html
    11
    

    View Slide

12. Loader & Parser
    12
    

    View Slide

13. From Contents to Pixels
    HTML
    Parser
    DOM
    CSS
    Parser
    HTML
    Style
    Sheets
    DOM
    Tree
    Style
    Rules
    Render
    Tree
    Render
    Layout
    Paint
    http://www.html5rocks.com/en/tutorials/internals/howbrowserswork
    13
    

    View Slide

14. Loader: Resource, Document, Frame, Page
    https://www.webkit.org/blog/1188/how-webkit-loads-a-web-page/
    14
    

    View Slide

15. Browser Caches
    http://gent.ilcore.com/2011/02/chromes-10-caches.html
    HTTP Disk Cache
    HTTP Memory Cache
    DNS Host Cache
    Preconnect Domain Cache
    V8 Compilation Cache
    SSL Session Cache
    TCP Connections
    Cookies
    HTML5 Caches
    ....
    15
    

    View Slide

16. HTML Tokenizer
    http://www.whatwg.org/specs/web-apps/current-work/multipage/tokenization.html#tokenization
    ~70 states
    Hello
    tag
    text
    16
    

    View Slide

17. HTML Parser
    https://www.webkit.org/blog/1273/the-html5-parsing-algorithm/
    ~10,000 lines of code
    Work in progress: Threaded HTML Parser
    Tokenizer
    Tree Builder
    DOM Tree
    Hello
    tag
    text
    17
    

    View Slide

18. Preloader Scanner
    Schedule concurrent loading of resources
    CSS @import
    HTML , <img><br/>//ajax.googleapis.com/ajax/libs/ext-core/3/ext-core.js<br/>https://use.typekit.com/bqo8qzg.js<br/>/img/sencha-large.png<br/>/css/carousel.css<br/>/js/carousel.js<br/>http://cdn.sencha.io/img/home/promos/promos-20130328-bundle-webinar.png<br/>http://cdn.sencha.io/img/home/20130219-promo-senchacon.png<br/>http://cdn.sencha.io/img/home/december-2012/icon-news.png<br/>http://cdn.sencha.io/img/home/december-2012/icon-blog.png<br/>http://cdn.sencha.io/img/20130317-mz-pivot-grid.png<br/>http://cdn.sencha.io/img/20120607-whats-new-io.png<br/>http://cdn.sencha.io/img/20130317-development-enterprise.png<br/>http://cdn.sencha.io/img/home/december-2012/icon-twitter.png<br/>http://cdn.sencha.io/img/home/december-2012/customer-logos.png<br/>//platform.twitter.com/widgets.js<br/>/forum/clientscript/vbulletin_md5.js?v=403<br/>/min/?g=main-js&1362779803&debug=1<br/>sencha.com<br/>18<br/>

    View Slide

19. Layout
    19
    

    View Slide

20. DOM Tree
    Hello
    HTMLDocument
    HTMLBodyElement
    HTMLParagraphElement attributes, children, contents
    20
    

    View Slide

21. DOM Tree vs Render Tree
    There is no Render* for display:none
    HTMLDocument
    HTMLBodyElement
    HTMLParagraphElement
    RenderRoot
    RenderBody
    RenderText
    tree structure/navigation
    metrics (box model)
    hit testing
    RenderStyle
    computed style
    many:1
    21
    

    View Slide

22. “Attach” Process
    Historical: Node is invisible, attach it to the main view to make it visible
    42
    View
    RenderObject RenderStyle
    tree structure/navigation
    metrics (box model)
    hit testing
    computed style
    Node/Element
    1:1 many:1
    another tree structure
    22
    

    View Slide

23. It’s a forest!
    23
    

    View Slide

24. http://www.webkit.org/coding/dom-element-attach.html
    24
    

    View Slide

25. Style Recalc vs Layout
    • RecalcStyle refers to style
    resolution, i.e. solving style
    properties for each node.
    • Layout needs style information,
    e.g. the computed dimension
    defines the box geometry.
    • Layout will be deferred if there
    is painting in progress (not
    completed yet).
    • Layout is on-demand: triggered
    by style change.
    More complication when loading
    data
    data
    data
    data
    RecalcStyle
    RecalcStyle
    Layout
    Paint
    RecalcStyle
    Layout
    Paint
    preorder traversal
    might be
    incremental
    RenderObject
    specific
    25
    

    View Slide

26. Effects of Style Recalc and Layout
    document.getElementById('box').style.top = '100px';
    document.getElementById('box').style.backgroundColor = 'red';
    Aggregated “dirty” area
    No layout necessary,
    metrics are not flagged as “changed”
    26
    

    View Slide

27. Getting the Right Style
    HTMLDocument
    HTMLBodyElement
    HTMLParagraphElement
    DOM Tree
    p { color: blue }
    Stylesheet List
    RenderStyle
    CSS Style Selector: id, class, tags, ...
    27
    

    View Slide

28. Minimizing Layout
    http://gent.ilcore.com/2011/03/how-not-to-trigger-layout-in-webkit.html
    http://code.google.com/p/chromium/source/search?q=%22-%3EupdateLayoutIgnorePendingStylesheets()%22
    clientHeight
    clientLeft
    clientTop
    clientWidth
    offsetHeight
    offsetLeft
    offsetParent
    offsetTop
    offsetWidth
    scrollLeft
    scrollTop
    scrollWidth
    innerText
    outerText
    Element
    focus()
    getBoundingClientRect()
    getClientRects()
    scrollByLines()
    scrollByPages()
    scrollHeight
    scrollIntoView()
    scrollIntoViewIfNeeded()
    28
    

    View Slide

29. Painting
    29
    

    View Slide

30. Goals of Painting
    • “Visualize” all RenderObjects (again, tree traversal)
    • All painting commands go to GraphicsContext abstraction
    • Important
    • Clipped to the visible viewport
    • Back-to-front
    Optimizations
    Transparency &
    clipping
    30
    

    View Slide

31. Graphics Abstraction
    Mac & iOS
    Chromium &
    Android Qt
    CoreGraphics
    Skia
    QPainter
    graphics stack
    GraphicsContext
    http://ariya.ofilabs.com/2011/06/your-webkit-port-is-special-just-like-every-other-port.html
    Gtk+
    Cairo
    31
    

    View Slide

32. CSS Painting: 10 Stages
    http://www.w3.org/TR/CSS2/zindex.html
    Appendix E. Elaborate description of Stacking Contexts
    •
    Background
    •
    Floats
    •
    Foreground
    •
    Selection
    •
    Outline
    Done by the render objects
    (in the render tree)
    32
    

    View Slide

33. Painting Flow: Browser vs WebKit
    WebKit
    Browser
    Request painting, GraphicsContext
    User scrolls
    the viewport
    Style change
    Mark “dirty” area
    Sometimes to the
    backing store
    33
    

    View Slide

34. Path is Everything
    34
    

    View Slide

35. Stroke and Brush
    35
    

    View Slide

36. Painting Complexity
    Polygon
    Multiple levels of transparency
    36
    

    View Slide

37. Transformation
    Scaling
    Rotation
    Perspective
    37
    

    View Slide

38. Hardware Acceleration
    38
    

    View Slide

39. translate3d
    39
    

    View Slide

40. SoC = System-on-a-Chip
    CPU GPU
    40
    

    View Slide

41. Graphics Processing Unit
    “Fixed” geometry
    Transformation
    Textured triangles
    Parallelism
    41
    

    View Slide

42. GPU Workflow
    Vertices Rendered Textured
    Matrix
    42
    

    View Slide

43. Efficient Uses of GPU
    Drawing
    primitives
    Backing
    stores
    Layer &
    compositing
    43
    

    View Slide

44. Responsive Interface
    Processor
    Rendering
    User interaction
    decoupled
    44
    

    View Slide

45. Maps
    Tile
    45
    

    View Slide

46. Rendering vs Interaction
    Web Page
    Backing Store
    Screen
    Rendering
    User interaction
    46
    

    View Slide

47. Checkerboard Pattern
    47
    

    View Slide

48. Pinch to Zoom
    when you pinch...
    48
    

    View Slide

49. Progressive Rendering
    Crisp but slow
    Fast but blurry
    49
    

    View Slide

50. Tile Transformation
    Panning = Translation Zooming = Scaling
    50
    

    View Slide

51. Perceived Responsiveness
    User pinches
    Smooth scaled
    Blocky (but fast!)
    51
    

    View Slide

52. Tiling System
    CPU GPU
    ....
    ....
    Texture upload
    52
    

    View Slide

53. Per Element Backing Store = Layer
    http://techblog.netflix.com/2012/01/webkit-in-your-living-room.html
    http://aerotwist.com/blog/on-translate3d-and-layer-creation-hacks/
    3-D transform/perspective
    , , plugins
    CSS animation, filters
    ....
    53
    

    View Slide

54. Mechanics of Animation
    Initialization
    Animation step
    “Hey, this is good stuff. Cache it as texture #42.”
    “Apply [operation] to texture #42.”
    54
    

    View Slide

55. Element and “Logical” 3-D
    http://www.webkit.org/blog-files/leaves/
    Compositor
    Tweening
    Can be in a separate thread
    55
    

    View Slide

56. Debugging in Safari
    defaults write com.apple.Safari IncludeInternalDebugMenu 1
    56
    

    View Slide

57. Compositing Indicators
    Composited layer
    Container layer
    No associated texture
    Overflow
    Texture (number = upload)
    57
    

    View Slide

58. Frame Rate HUD
    http://ariya.ofilabs.com/2013/03/frame-rate-hud-on-chrome-for-android.html
    Frame rate chart
    GPU memory usage
    58
    

    View Slide

59. Avoid Texture Reupload
    No reupload Upload
    Opacity
    Position
    Size
    Filter
    Everything else!
    “Hardware accelerated CSS”
    59
    

    View Slide

60. Constant Upload = Bad
    http://codepen.io/ariya/full/xuwgy
    translate3d ???
    60
    

    View Slide

61. Transformation
    http://ariya.github.com/css/spinningcube/ http://dev.sencha.com/animator/demos/cogs/
    61
    

    View Slide

62. Fade In/Out
    http://dev.sencha.com/animator/demos/ions/
    http://codepen.io/ariya/full/spzad
    62
    

    View Slide

63. three.js Periodic Table
    http://mrdoob.github.com/three.js/examples/css3d_periodictable.html
    63
    

    View Slide

64. Montage MovieShow
    http://montagejs.github.com/montage/samples/popcorn/
    64
    

    View Slide

65. Photon CSS Lighting
    http://photon.attasi.com
    65
    

    View Slide

66. FPS View
    http://www.keithclark.co.uk/labs/css3-fps/
    66
    

    View Slide

67. Filter
    http://html.adobe.com/webstandards/csscustomfilters/cssfilterlab/
    67
    

    View Slide

68. Timer Latency
    Depending on
    user reaction
    Animation end triggers the JavaScript callback
    JavaScript code kicks the next animation
    Prepare both
    animation and hide the
    “wrong” one
    68
    

    View Slide

69. Prepare and Reuse
    Hide the poster (=layer) offscreen
    Viewport
    69
    

    View Slide

70. Avoid Overcapacity
    ....
    ....
    Texture upload
    Think of the GPU memory like a cache.
    70
    

    View Slide

71. Color Transition
    @keyframes box {
    0% { transform: background-color: blue; }
    100% { transform: background-color: green; }
    }
    Need a new texture uploaded to
    the GPU for every in-between color
    71
    

    View Slide

72. Color Transition Trick
    Blended with
    http://codepen.io/ariya/full/ofDIh
    72
    

    View Slide

73. Traffic Congestion
    The speed of the car vs the traffic condition
    73
    

    View Slide

74. Conclusion
    74
    

    View Slide

75. Rendering Aspects
    75
    

    View Slide

76. Run a thorough feature-cost analysis
    Don’t fight the rendering engine
    Never assume, always measure
    76
    

    View Slide

77. More Stuff
    Jank Busters: http://www.youtube.com/watch?v=hAzhayTnhEI
    Continuous painting:
    http://updates.html5rocks.com/2013/02/Profiling-Long-Paint-Times-with-
    DevTools-Continuous-Painting-Mode
    Chrome GPU rendering benchmark (Telemetry):
    http://www.chromium.org/developers/design-documents/rendering-benchmarks
    Skia debugger:
    http://wesleyhales.com/blog/2013/02/18/Adventures-With-the-Skia-Debugger/
    77
    

    View Slide

78. Thank You
    ariya.hidayat@gmail.com
    @AriyaHidayat
    ariya.ofilabs.com
    speakerdeck.com/ariya
    Credits: Some artworks are from http://openclipart.org/user-cliparts/nicubunu
    78
    

    View Slide