Fluid User Interface with Hardware Acceleration

Fluid User Interface with Hardware Acceleration Ariya Hidayat San Francisco
Feb 22, 2013 @ariyahidayat 1

Magical Advice Use translate3d for hardware acceleration 3

whoami 4

CSS3 5

Web Browser from 10,000 ft 6

Browser: High Level User Interface Browser Engine Graphics Stack Data
Persistence Render Engine JavaScript Engine Networking I/O 7

From Contents to Pixels HTML Parser DOM CSS Parser HTML
Style Sheets DOM Tree Style Rules Render Tree Render Layout Paint http://www.html5rocks.com/en/tutorials/internals/howbrowserswork 8

DOM Tree <html><body><p>Hello</p></body></html> HTMLDocument HTMLBodyElement HTMLParagraphElement attributes, children, contents 9

DOM Tree vs Render Tree HTMLDocument HTMLBodyElement HTMLParagraphElement RenderRoot RenderBody
RenderText tree structure/navigation metrics (box model) hit testing RenderStyle computed style many:1 10

Style Recalc vs Layout document.getElementById('box').style.top = '100px'; Aggregated “dirty” area
document.getElementById('box').style.backgroundColor = 'red'; No layout necessary, metrics are not ﬂagged as “changed” 11

Minimizing Layout http://gent.ilcore.com/2011/03/how-not-to-trigger-layout-in-webkit.html http://code.google.com/p/chromium/source/search?q=%22-%3EupdateLayoutIgnorePendingStylesheets()%22 clientHeight clientLeft clientTop clientWidth offsetHeight offsetLeft
offsetParent offsetTop offsetWidth scrollLeft scrollTop scrollWidth innerText outerText Element focus() getBoundingClientRect() getClientRects() scrollByLines() scrollByPages() scrollHeight scrollIntoView() scrollIntoViewIfNeeded() 12

CSS Painting: 10 Stages http://www.w3.org/TR/CSS2/zindex.html Appendix E. Elaborate description of
Stacking Contexts • Background • Floats • Foreground • Selection • Outline Done by the render objects (in the render tree) 13

Graphics Stack: Crash Course 14

Path is Everything 15

Stroke = Outline Solid Dashed Dotted Textured 16

Brush = Fill Solid None Gradient Textured 17

Path Approximation Curves Polygon 18

Antialiasing 19

Transformation Scaling Rotation Perspective 20

Graphics Processing Unit “Fixed” geometry Transformation Textured triangles Parallelism 21

Optimized for Games http://epicgames.com/technology/epic-citadel http://www.rage.com/ 22

GPU Workﬂow Vertices Rendered Textured Matrix 23

Web Page Rendering 24

Game vs Web Page Predictable contents (assets) ✔ Mostly text
✔ Mostly images ✔ immediate Initial response 25

GPU Physical Limitations • Memory: Can’t store too much stuff
• Speed: Quad-core CPU can do certain operations better • Bandwidth: Data transfer can be the bottleneck • Power: Electrons always need energy 26

Fluid Animation At the beginning, push as many resources as
possible to the GPU During the animation, minimize the interaction between CPU-GPU 1 2 27

Immediate vs Retained draw the shape at (x, y) x
= x + 10 blit the buffer at (x, y) x = x + 10 For every animation tick... At the beginning... draw the shape onto a buffer Off main thread 28

Animation Mechanics Initialization Animation step “Hey, this is good stuff.
Cache it as texture #42.” “Apply [operation] to texture #42.” Animation loop 29

Element = Layer http://www.webkit.org/blog-ﬁles/leaves/ 30

Compositing: “Logical” 3-D Compositor Tweening Actual operation 31

By Force transform: translateZ(0) Not needed for CSS animation! .someid
{ animation-name: somekeyframeanimation; animation-duration: 7s; transform: translateZ(0); } Don’t do that 32

Tools & Examples 33

Debugging in Safari defaults write com.apple.Safari IncludeInternalDebugMenu 1 https://developer.apple.com/videos/wwdc/2012/?id=601 34

Compositing Indicator Composited layer Container layer No associated texture Overﬂow
Texture (number = upload) 35

Texture Upload No reupload Upload Opacity Position Size Filter Everything
else! “Hardware accelerated CSS” 36

Constant Upload = Bad http://codepen.io/ariya/full/xuwgy translate3d ??? 37

Transformation http://ariya.github.com/css/spinningcube/ http://dev.sencha.com/animator/demos/cogs/ 38

Fade In/Out http://dev.sencha.com/animator/demos/ions/ http://codepen.io/ariya/full/spzad 39

three.js Periodic Table http://mrdoob.github.com/three.js/examples/css3d_periodictable.html 40

Montage MovieShow http://montagejs.github.com/montage/samples/popcorn/ 41

Photon CSS Lighting http://photon.attasi.com 42

FPS View http://www.keithclark.co.uk/labs/css3-fps/ 43

Filter http://html.adobe.com/webstandards/csscustomﬁlters/cssﬁlterlab/ 44

Hacks & Tricks 45

Trafﬁc Congestion The speed of the car vs the trafﬁc
condition 46

Avoid Overcapacity .... .... Texture upload Think of the GPU
memory like a cache. 47

Prepare and Reuse Hide the poster (=layer) offscreen Viewport 48

Color Transition @keyframes box { 0% { transform: background-color: blue;
} 100% { transform: background-color: green; } } Need a new texture uploaded to the GPU for every in-between color 49

Color Transition Trick Blended with http://codepen.io/ariya/full/ofDIh 50

Timer Latency Depending on user reaction Animation end triggers the
JavaScript callback JavaScript code kicks the next animation Prepare both animation and hide the “wrong” one 51

More Stuff Jank Busters: http://www.youtube.com/watch?v=hAzhayTnhEI Continuous painting: http://updates.html5rocks.com/2013/02/Proﬁling-Long-Paint-Times-with- DevTools-Continuous-Painting-Mode Chrome
GPU rendering benchmark (Telemetry): http://www.chromium.org/developers/design-documents/rendering-benchmarks Skia debugger: http://wesleyhales.com/blog/2013/02/18/Adventures-With-the-Skia-Debugger/ 52

Final Words 53

http://www.trollquotes.org/619-super-spider-bat-troll-quote/ 54

Use CSS3 animation & transition Be advised of texture uploads
Never assume, always measure 55

There is No Silver Bullet Traditional graphics programming has been
always full of tricks. It will always be. 56

Thank You ariya.hidayat@gmail.com @AriyaHidayat ariya.oﬁlabs.com Background artwork from http://kde-look.org/content/show.php/Sin+Flow?content=139751 CC
BY-NC-SA speakerdeck.com/ariya 57

Fluid User Interface with Hardware Acceleration

Fluid User Interface with Hardware Acceleration

More Decks by Ariya Hidayat

Other Decks in Programming

Featured

Transcript