WebKit Rendering Overview for HTML5 Developer @ariyahidayat San Francisco, April 2, 2013 1

whoami 2

WebKit 3

WebKit is Everywhere 4

KHTML, Apple, and Community 0 50000 100000 150000 0 3 6 9 12 Revisions Years ~3000 commits/month http://ariya.ofilabs.com/2011/11/one-hundred-thousand-and-counting.html 5

Heavy on TDD Source Code Tests Repository Checkout 6

Involvement: Social Layers Contributor Committer Reviewer accept or reject patches checks in reviewed patches after 10-20 patches after 80 patches 7

Big Players http://blog.bitergia.com/2013/02/06/report-on-the-activity-of-companies-in-the-webkit-project/ http://blog.bitergia.com/2013/03/01/reviewers-and-companies-in-webkit-project/ 8

Browser at a High Level User Interface Browser Engine Graphics Stack Data Persistence Render Engine JavaScript Engine Networking I/O http://www.html5rocks.com/en/tutorials/internals/howbrowserswork 9

WebKit Components Render Engine WebCore JavaScript Engine (JSC/V8) Client Interface HTML DOM CSS SVG Canvas 10

Platform Abstraction Client Interface Networking I/O Graphics Theme Events Clipboard Thread Geolocation Timer API Calls Events Port (Chrome, Safari, etc.) http://ariya.ofilabs.com/2011/06/your-webkit-port-is-special-just-like-every-other-port.html 11

Loader & Parser 12

From Contents to Pixels HTML Parser DOM CSS Parser HTML Style Sheets DOM Tree Style Rules Render Tree Render Layout Paint http://www.html5rocks.com/en/tutorials/internals/howbrowserswork 13

Loader: Resource, Document, Frame, Page https://www.webkit.org/blog/1188/how-webkit-loads-a-web-page/ 14

Browser Caches http://gent.ilcore.com/2011/02/chromes-10-caches.html HTTP Disk Cache HTTP Memory Cache DNS Host Cache Preconnect Domain Cache V8 Compilation Cache SSL Session Cache TCP Connections Cookies HTML5 Caches .... 15

HTML Tokenizer http://www.whatwg.org/specs/web-apps/current-work/multipage/tokenization.html#tokenization ~70 states

Hello

tag text 16

HTML Parser https://www.webkit.org/blog/1273/the-html5-parsing-algorithm/ ~10,000 lines of code Work in progress: Threaded HTML Parser Tokenizer Tree Builder DOM Tree

Hello

tag text 17

Preloader Scanner Schedule concurrent loading of resources CSS @import HTML , <img> //ajax.googleapis.com/ajax/libs/ext-core/3/ext-core.js https://use.typekit.com/bqo8qzg.js /img/sencha-large.png /css/carousel.css /js/carousel.js http://cdn.sencha.io/img/home/promos/promos-20130328-bundle-webinar.png http://cdn.sencha.io/img/home/20130219-promo-senchacon.png http://cdn.sencha.io/img/home/december-2012/icon-news.png http://cdn.sencha.io/img/home/december-2012/icon-blog.png http://cdn.sencha.io/img/20130317-mz-pivot-grid.png http://cdn.sencha.io/img/20120607-whats-new-io.png http://cdn.sencha.io/img/20130317-development-enterprise.png http://cdn.sencha.io/img/home/december-2012/icon-twitter.png http://cdn.sencha.io/img/home/december-2012/customer-logos.png //platform.twitter.com/widgets.js /forum/clientscript/vbulletin_md5.js?v=403 /min/?g=main-js&1362779803&debug=1 sencha.com 18

Layout 19

DOM Tree

Hello

HTMLDocument HTMLBodyElement HTMLParagraphElement attributes, children, contents 20

DOM Tree vs Render Tree There is no Render* for display:none HTMLDocument HTMLBodyElement HTMLParagraphElement RenderRoot RenderBody RenderText tree structure/navigation metrics (box model) hit testing RenderStyle computed style many:1 21

“Attach” Process Historical: Node is invisible, attach it to the main view to make it visible View RenderObject RenderStyle tree structure/navigation metrics (box model) hit testing computed style Node/Element 1:1 many:1 another tree structure 22

It’s a forest! 23

http://www.webkit.org/coding/dom-element-attach.html 24

Style Recalc vs Layout • RecalcStyle refers to style resolution, i.e. solving style properties for each node. • Layout needs style information, e.g. the computed dimension defines the box geometry. • Layout will be deferred if there is painting in progress (not completed yet). • Layout is on-demand: triggered by style change. More complication when loading data data data data RecalcStyle RecalcStyle Layout Paint RecalcStyle Layout Paint preorder traversal might be incremental RenderObject specific 25

Effects of Style Recalc and Layout document.getElementById('box').style.top = '100px'; document.getElementById('box').style.backgroundColor = 'red'; Aggregated “dirty” area No layout necessary, metrics are not flagged as “changed” 26

Getting the Right Style HTMLDocument HTMLBodyElement HTMLParagraphElement DOM Tree p { color: blue } Stylesheet List RenderStyle CSS Style Selector: id, class, tags, ... 27

Minimizing Layout http://gent.ilcore.com/2011/03/how-not-to-trigger-layout-in-webkit.html http://code.google.com/p/chromium/source/search?q=%22-%3EupdateLayoutIgnorePendingStylesheets()%22 clientHeight clientLeft clientTop clientWidth offsetHeight offsetLeft offsetParent offsetTop offsetWidth scrollLeft scrollTop scrollWidth innerText outerText Element focus() getBoundingClientRect() getClientRects() scrollByLines() scrollByPages() scrollHeight scrollIntoView() scrollIntoViewIfNeeded() 28

Painting 29

Goals of Painting • “Visualize” all RenderObjects (again, tree traversal) • All painting commands go to GraphicsContext abstraction • Important • Clipped to the visible viewport • Back-to-front Optimizations Transparency & clipping 30

Graphics Abstraction Mac & iOS Chromium & Android Qt CoreGraphics Skia QPainter graphics stack GraphicsContext http://ariya.ofilabs.com/2011/06/your-webkit-port-is-special-just-like-every-other-port.html Gtk+ Cairo 31

CSS Painting: 10 Stages http://www.w3.org/TR/CSS2/zindex.html Appendix E. Elaborate description of Stacking Contexts • Background • Floats • Foreground • Selection • Outline Done by the render objects (in the render tree) 32

Painting Flow: Browser vs WebKit WebKit Browser Request painting, GraphicsContext User scrolls the viewport Style change Mark “dirty” area Sometimes to the backing store 33

Path is Everything 34

Stroke and Brush 35

Painting Complexity Polygon Multiple levels of transparency 36

Transformation Scaling Rotation Perspective 37

Hardware Acceleration 38

translate3d 39

SoC = System-on-a-Chip CPU GPU 40

Graphics Processing Unit “Fixed” geometry Transformation Textured triangles Parallelism 41

GPU Workflow Vertices Rendered Textured Matrix 42

Efficient Uses of GPU Drawing primitives Backing stores Layer & compositing 43

Responsive Interface Processor Rendering User interaction decoupled 44

Maps Tile 45

Rendering vs Interaction Web Page Backing Store Screen Rendering User interaction 46

Checkerboard Pattern 47

Pinch to Zoom when you pinch... 48

Progressive Rendering Crisp but slow Fast but blurry 49

Tile Transformation Panning = Translation Zooming = Scaling 50

Perceived Responsiveness User pinches Smooth scaled Blocky (but fast!) 51

Tiling System CPU GPU .... .... Texture upload 52

Per Element Backing Store = Layer http://techblog.netflix.com/2012/01/webkit-in-your-living-room.html http://aerotwist.com/blog/on-translate3d-and-layer-creation-hacks/ 3-D transform/perspective , , plugins CSS animation, filters .... 53

Mechanics of Animation Initialization Animation step “Hey, this is good stuff. Cache it as texture #42.” “Apply [operation] to texture #42.” 54

Element and “Logical” 3-D http://www.webkit.org/blog-files/leaves/ Compositor Tweening Can be in a separate thread 55

Debugging in Safari defaults write com.apple.Safari IncludeInternalDebugMenu 1 56

Compositing Indicators Composited layer Container layer No associated texture Overflow Texture (number = upload) 57

Frame Rate HUD http://ariya.ofilabs.com/2013/03/frame-rate-hud-on-chrome-for-android.html Frame rate chart GPU memory usage 58

Avoid Texture Reupload No reupload Upload Opacity Position Size Filter Everything else! “Hardware accelerated CSS” 59

Constant Upload = Bad http://codepen.io/ariya/full/xuwgy translate3d ??? 60

Transformation http://ariya.github.com/css/spinningcube/ http://dev.sencha.com/animator/demos/cogs/ 61

Fade In/Out http://dev.sencha.com/animator/demos/ions/ http://codepen.io/ariya/full/spzad 62

three.js Periodic Table http://mrdoob.github.com/three.js/examples/css3d_periodictable.html 63

Montage MovieShow http://montagejs.github.com/montage/samples/popcorn/ 64

Photon CSS Lighting http://photon.attasi.com 65

FPS View http://www.keithclark.co.uk/labs/css3-fps/ 66

Filter http://html.adobe.com/webstandards/csscustomfilters/cssfilterlab/ 67

Timer Latency Depending on user reaction Animation end triggers the JavaScript callback JavaScript code kicks the next animation Prepare both animation and hide the “wrong” one 68

Prepare and Reuse Hide the poster (=layer) offscreen Viewport 69

Avoid Overcapacity .... .... Texture upload Think of the GPU memory like a cache. 70

Color Transition @keyframes box { 0% { transform: background-color: blue; } 100% { transform: background-color: green; } } Need a new texture uploaded to the GPU for every in-between color 71

Color Transition Trick Blended with http://codepen.io/ariya/full/ofDIh 72

Traffic Congestion The speed of the car vs the traffic condition 73

Conclusion 74

Rendering Aspects 75

Run a thorough feature-cost analysis Don’t fight the rendering engine Never assume, always measure 76

More Stuff Jank Busters: http://www.youtube.com/watch?v=hAzhayTnhEI Continuous painting: http://updates.html5rocks.com/2013/02/Profiling-Long-Paint-Times-with- DevTools-Continuous-Painting-Mode Chrome GPU rendering benchmark (Telemetry): http://www.chromium.org/developers/design-documents/rendering-benchmarks Skia debugger: http://wesleyhales.com/blog/2013/02/18/Adventures-With-the-Skia-Debugger/ 77

Thank You [email protected] @AriyaHidayat ariya.ofilabs.com speakerdeck.com/ariya Credits: Some artworks are from http://openclipart.org/user-cliparts/nicubunu 78