Fluid User Interface with
Hardware Acceleration
Ariya Hidayat
San Francisco
Feb 22, 2013
@ariyahidayat
1
Slide 2
Slide 2 text
2
Slide 3
Slide 3 text
Magical Advice
Use translate3d for
hardware acceleration
3
Slide 4
Slide 4 text
whoami
4
Slide 5
Slide 5 text
CSS3
5
Slide 6
Slide 6 text
Web Browser from 10,000 ft
6
Slide 7
Slide 7 text
Browser: High Level
User Interface
Browser Engine
Graphics
Stack
Data Persistence
Render Engine
JavaScript
Engine
Networking
I/O
7
Slide 8
Slide 8 text
From Contents to Pixels
HTML
Parser
DOM
CSS
Parser
HTML
Style
Sheets
DOM
Tree
Style
Rules
Render
Tree
Render
Layout
Paint
http://www.html5rocks.com/en/tutorials/internals/howbrowserswork
8
DOM Tree vs Render Tree
HTMLDocument
HTMLBodyElement
HTMLParagraphElement
RenderRoot
RenderBody
RenderText
tree structure/navigation
metrics (box model)
hit testing
RenderStyle
computed style
many:1
10
Slide 11
Slide 11 text
Style Recalc vs Layout
document.getElementById('box').style.top = '100px';
Aggregated “dirty” area
document.getElementById('box').style.backgroundColor = 'red';
No layout necessary,
metrics are not flagged as “changed”
11
CSS Painting: 10 Stages
http://www.w3.org/TR/CSS2/zindex.html
Appendix E. Elaborate description of Stacking Contexts
•
Background
•
Floats
•
Foreground
•
Selection
•
Outline
Done by the render objects
(in the render tree)
13
Slide 14
Slide 14 text
Graphics Stack: Crash Course
14
Slide 15
Slide 15 text
Path is Everything
15
Slide 16
Slide 16 text
Stroke = Outline
Solid Dashed Dotted Textured
16
Slide 17
Slide 17 text
Brush = Fill
Solid
None Gradient Textured
17
Slide 18
Slide 18 text
Path Approximation
Curves
Polygon
18
Slide 19
Slide 19 text
Antialiasing
19
Slide 20
Slide 20 text
Transformation
Scaling
Rotation
Perspective
20
Slide 21
Slide 21 text
Graphics Processing Unit
“Fixed” geometry
Transformation
Textured triangles
Parallelism
21
Slide 22
Slide 22 text
Optimized for Games
http://epicgames.com/technology/epic-citadel http://www.rage.com/
22
Slide 23
Slide 23 text
GPU Workflow
Vertices Rendered Textured
Matrix
23
Slide 24
Slide 24 text
Web Page Rendering
24
Slide 25
Slide 25 text
Game vs Web Page
Predictable contents (assets) ✔
Mostly text
✔
Mostly images
✔
immediate
Initial response
25
Slide 26
Slide 26 text
GPU Physical Limitations
• Memory: Can’t store too much stuff
• Speed: Quad-core CPU can do certain operations better
• Bandwidth: Data transfer can be the bottleneck
• Power: Electrons always need energy
26
Slide 27
Slide 27 text
Fluid Animation
At the beginning, push as many resources
as possible to the GPU
During the animation, minimize
the interaction between CPU-GPU
1
2
27
Slide 28
Slide 28 text
Immediate vs Retained
draw the shape at (x, y)
x = x + 10
blit the buffer at (x, y)
x = x + 10
For every animation tick...
At the beginning...
draw the shape onto
a buffer
Off main thread
28
Slide 29
Slide 29 text
Animation Mechanics
Initialization
Animation step
“Hey, this is good stuff. Cache it as texture #42.”
“Apply [operation] to texture #42.”
Animation loop
29
Slide 30
Slide 30 text
Element = Layer
http://www.webkit.org/blog-files/leaves/
30
Slide 31
Slide 31 text
Compositing: “Logical” 3-D
Compositor
Tweening
Actual operation
31
Slide 32
Slide 32 text
By Force
transform: translateZ(0)
Not needed for CSS animation!
.someid {
animation-name: somekeyframeanimation;
animation-duration: 7s;
transform: translateZ(0);
}
Don’t do that
32
Slide 33
Slide 33 text
Tools & Examples
33
Slide 34
Slide 34 text
Debugging in Safari
defaults write com.apple.Safari IncludeInternalDebugMenu 1
https://developer.apple.com/videos/wwdc/2012/?id=601
34
Traffic Congestion
The speed of the car
vs the traffic condition
46
Slide 47
Slide 47 text
Avoid Overcapacity
....
....
Texture upload
Think of the GPU memory like a cache.
47
Slide 48
Slide 48 text
Prepare and Reuse
Hide the poster (=layer) offscreen
Viewport
48
Slide 49
Slide 49 text
Color Transition
@keyframes box {
0% { transform: background-color: blue; }
100% { transform: background-color: green; }
}
Need a new texture uploaded to
the GPU for every in-between color
49
Slide 50
Slide 50 text
Color Transition Trick
Blended with
http://codepen.io/ariya/full/ofDIh
50
Slide 51
Slide 51 text
Timer Latency
Depending on
user reaction
Animation end triggers the JavaScript callback
JavaScript code kicks the next animation
Prepare both
animation and hide the
“wrong” one
51
Use CSS3 animation & transition
Be advised of texture uploads
Never assume, always measure
55
Slide 56
Slide 56 text
There is No Silver Bullet
Traditional graphics
programming has been
always full of tricks.
It will always be.
56
Slide 57
Slide 57 text
Thank You
[email protected]
@AriyaHidayat
ariya.ofilabs.com
Background artwork from http://kde-look.org/content/show.php/Sin+Flow?content=139751 CC BY-NC-SA
speakerdeck.com/ariya
57