View To Pixel 2.0 (Droidcon SF 17')

On what's going on in the Android system when it created pixels on the screen, out of the Views that we write in code.

Also here: https://docs.google.com/presentation/d/1__ezgkz5DzFWxHGy4p0wx5NLr96aML5_asskM6OsEJM/present#slide=id.g299bece29b_1_927

Presented in Droidcon SF 2017.

Transcript

1. View To Pixel 2.0 Britt Barak DroidconSF 2017
3. Android Lead Android Academy TLV Women Techmakers IL Britt Barak
4. Today’s story
5. CPU new View()
6. Rasterization From a high level object into pixels on screen (or in texture).
7. GPU - For The Rescue !
8. CPU new View() GPU
9. CPU new View() GPU Polygons Textures
10. CPU new View() GPU Polygons Textures DisplayList
11. Refresh Rate 60 Hz
12. CPU new View() GPU Polygons Textures DisplayList Frame Rate 60 fps
13. What happens if...
14. CPU GPU
15. CPU GPU
16. CPU GPU
18. What do we do?
19. Double Buffer
20. Double Triple Buffer
21. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame Buffer
22. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame Buffer
23. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame Buffer
24. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame Buffer
25. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame Buffer
26. VSync
27. In some more details...
28. CPU GPU Vsync
29. CPU new View() GPU Measure
30. CPU GPU Layout
31. CPU GPU new View() DisplayList Draw
32. CPU new View() GPU Polygons Textures DisplayList Sync & Upload
33. CPU new View() GPU Polygons Textures DisplayList Execute Back Buffer
34. CPU new View() GPU Polygons Textures DisplayList Swap buffers Back Buffer
35. Measure/ Layout Draw Sync & Upload Execute Swap Buffers VSync/Misc
36. actually...
37. Input
38. Animation
39. VSync/Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
40. Let’s go a bit deeper...
41. Window Manager
42. Window Manager lifecycles, input events, focus events, screen orientation, transitions, animations, position, transforms, z-order...
43. • Knows all surfaces • Only service that can modify display’s content SurfaceFlinger
44. Status bar App UI Navigation bar
45. “Listen, I have another ‘woower’...” Window Manager SurfaceFlinger
46. “K” Window Manager SurfaceFlinger
47. Window Manager SurfaceFlinger Surface BufferQueue
48. Window Manager SurfaceFlinger Surface BufferQueue
49. Surface BufferQueue Buffer Pool Queue Gralloc (HAL interface)
50. App Window Manager “Here’s a Binder object!”
51. App “I have something to display!” Binder Buffer Queue
52. App Binder Buffer Queue “Here’s a free buffer”
53. App Binder Buffer Queue
54. App Binder Buffer Queue “Done!”
55. App Binder Buffer Queue
56. Binder Buffer Queue “I’m ready!” SurfaceFlinger Display VSYNC “K”
57. Binder Buffer Queue SurfaceFlinger Display Binder Buffer Queue Binder Buffer Queue
58. SurfaceFlinger Display Hardware Composer HAL Binder Buffer Queue Binder Buffer Queue Binder Buffer Queue
59. SurfaceFlinger Display Hardware Composer HAL Binder Buffer Queue Binder Buffer Queue Binder Buffer Queue
60. Let’s talk some more about creating the frame...
61. VSync/Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers CPU
62. 1. Order is guaranteed 2. A step is called up to once
63. Choreographer
64. I/Choreographer(1378): Skipped 12 frames! The application may be doing too much work on its main thread.
65. Process Main (/UI) Thread
66. Main Thread (UI Thread) Process System Event Input Event Service Application UI Drawing
67. Java Thread run() start()
68. Main Thread (UI Thread) Looper Message queue Process
69. Main Thread (UI Thread) Looper Message queue Handler Handle MSG Send MSG Process
70. Main Thread (UI Thread) Looper Message queue Handler1 Handle MSG Send MSG Process Handler2 Handle MSG Send MSG
71. private Choreographer (Looper looper, int vsyncSource) ;
72. Choreographer.java private final class FrameHandler extends Handler { @Override public void handleMessage(Message msg) { switch (msg.what) { case MSG_DO_FRAME: doFrame(System.nanoTime(), 0); break; case MSG_DO_SCHEDULE_VSYNC: doScheduleVsync(); break; case MSG_DO_SCHEDULE_CALLBACK: doScheduleCallback(msg.arg1); break; } } }
73. 1. Order is guaranteed. 2. Steps happen up to once Choreographer.java void doFrame(long frameTimeNanos, int frame) { //... doCallbacks(Choreographer.CALLBACK_INPUT, frameTimeNanos); doCallbacks(Choreographer.CALLBACK_ANIMATION, frameTimeNanos); doCallbacks(Choreographer.CALLBACK_TRAVERSAL, frameTimeNanos); doCallbacks(Choreographer.CALLBACK_COMMIT, frameTimeNanos); //…
74. Choreographer.java void doCallbacks(int callbackType, long frameTimeNanos) { callbacks = mCallbackQueues[callbackType] .extractDueCallbacksLocked(now / TimeUtils.NANOS_PER_MS); for (CallbackRecord c = callbacks; c != null; c = c.next) { //… c.run(frameTimeNanos);
75. Measure Layout Draw!
76. What can we do? • Flatten hierarchy • <Merge> on <include> • Avoid when near the hierarchy top • Avoid when many layouts • Minimize requestLayout() • ConstrainLayout
77. Draw()
78. View.java public void draw(Canvas canvas)
79. //Step 1: draw background drawBackground(canvas); // Step 2, save the canvas' layers (prepare Fading, optional) canvas.saveLayer(…) //Step 3, draw view's content onDraw(canvas); public void draw(Canvas canvas) {
80. // Step 4, draw children dispatchDraw(canvas); // Step 5, draw the fade effect and restore layers (optional) final Shader fade = scrollabilityCache.shader; paint.setShader(fade); canvas.drawRect(left, top, right, top , paint); public void draw(Canvas canvas) {
81. // Step 6, draw decorations (foreground, scrollbars) onDrawForeground(canvas); // Step 7, draw the default focus highlight drawDefaultFocusHighlight(canvas); public void draw(Canvas canvas) {
82. background.draw(canvas) scrollBar.draw(canvas) foreground.draw(canvas)
83. GPU Overdraw Tool https://developer.android.com/studio/profile/dev-options-overdraw.html
84. A bit more about Canvas
85. 1. Primitives Canvas. drawPath(), drawOval(), drawRect()
86. 2. PathEffect paint.setPathEffect() ComposePathEffect, CornerPathEffect, DashPathEffect, DiscretePathEffect, PathDashPathEffect, SumPathEffect
87. 3. Alpha Decide alpha channel for each pixel paint.setAlpha()
88. 4. Shading determine each pixel’s color (ARGB). paint.setColor(), paint.setColorFilter(), paint.setShader() Chiu-Ki Chan: Fun With Android Shaders And Filters
89. public void draw(Canvas canvas) { //... // Step 4, draw children dispatchDraw(canvas); // Step 5, draw the fade effect and restore layers (optional) final Shader fade = scrollabilityCache.shader; paint.setShader(fade); canvas.drawRect(left, top, right, top , paint);
90. Shader paint.setShader(shader) LinearGradient (on fade), RadialGradient, SweepGradient, BitmapShader, ComposeShader https://developer.android.com/reference/android/graphics/Shader.html
91. 5. Transfer How to transfer color to destination Bitmap paint.setXfermode() Source Image Destination Image Transfer Mode Defaults to : PorterDuffXferMode (SRC_OVER) https://developer.android.com/reference/android/graphics/PorterDuffXfermode.html
95. Binder Buffer Queue surface canvas openGL openGLES SurfaceFlinger
96. VSync/Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
97. GPU Profiling Tool • A graph per visible app • A bar per frame • 16ms benchmark • Crossing = skipping frame https://developer.android.com/studio/profile/dev-options-rendering.html
98. Frame Metrics On Android N+ - Per-Frame data: - Window.OnFrameMetricsAvailableListener() - Aggregated data: - FrameMetricsAggregator (Support lib v26) https://developer.android.com/reference/android/view/FrameMetrics.html
100. Thank you! Keep in touch! Britt Barak @brittBarak