[Refael Ozeri, Yonatan Levin] GPU. The journey from code to pixel

Presentation from GDG DevFest Ukraine 2018 - the biggest community-driven Google tech conference in the CEE.

Learn more at: https://devfest.gdg.org.ua

__

In this session we will talk about our GPU, did you ever asked yourself what happens behind the scenes when we see a text view? or how does our code turn into pixels?

Transcript

7. Romain Guy & Chet Haase
8. U and I Not really
9. Yonatan V.Levin Refael Ozeri
11. GPU - The journey from code to pixel
12. Why do we love apps?
13. Smooth Experience
14. Google Play Store
15. Houzz
16. Motion is key
17. How is motion perceived?
18. How is motion perceived? FPS frames per second
19. How is motion perceived? 12fps Flip Book 24fps Movies Fluid Motion 60fps + No Difference 60fps Smooth Motion
20. How is motion perceived? So.. 60 fps is our target. A new frame is displayed every _____ milliseconds
21. How is motion perceived? So.. 60 fps is our target. A new frame is displayed every 16.666.. milliseconds
23. How did my code turn into pixels? CPU new View()
24. Say hello to our GPU Yonatan’s GPU.. :)
25. GPU CPU new View() GPU Polygons Textures
26. GPU GPU Polygons Textures CPU new View() DisplayList
27. Wait… DisplayList?! OpenGL?! WHAT?
28. DisplayList A sequence of graphics commands needed to be executed to render a specific view These commands are a mixture of statements that can be directly mapped to OpenGL commands
29. OpenGL OpenGL is a cross-language, cross-platform API for rendering 2D and 3D vector graphics. The API is typically used to interact with a GPU, to achieve hardware-accelerated rendering.
30. So now we’re all familiar with these big names. BUT.. we have a big problem!
32. GPU GPU Polygons Textures CPU new View() DisplayList
33. What do we need to do in 16 ms ?
34. CPU new View() GPU Measure
35. GPU Layout CPU
36. GPU Draw CPU new View() DisplayList
37. Sync & Upload CPU new View() GPU Polygons Textures DisplayList
38. Execute CPU new View() GPU Polygons Textures DisplayList
39. Execute CPU new View() GPU Polygons Textures DisplayList CPU new View() GPU Polygons Textures DisplayList Frame Buffer
40. So, we have a frame buffer that loads our graphics, what can go wrong?
44. Bonus Question.. Anyone knows the name of this game?
45. Dead Rising
46. First.. let’s understand how our hardware actually works!
47. Remember this?
48. Remember this? That’s 25 fps
49. Now let’s go SLOW.. Thank you slowmoguys
50. That’s our TV captured on 1,600 frames per second !
52. That’s our TV on 146,727 frames per second !
53. CPU GPU So what happened?
54. CPU GPU Our buffer is finished
55. CPU GPU Our buffer is not done in time
57. The solution?
58. The solution? Double Buffer!
59. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame Buffer
60. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame Buffer
61. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame Buffer
62. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame Buffer
63. Performance boost! The all new RenderThread (Lollipop & above)
64. input UI Thread animations measure layout draw sync Render Thread sync execute get buffer issue swap buffer Graphics get buffer get buffer See more @ I/O 2018 - Drawn out lecture - https://youtu.be/zdQRIYOST64
65. Tools
66. Layout Inspector (Formerly Hierarchy Viewer) WHY?
67. Tools→ Layout Inspector • Display complete view hierarchy • Access to all the views’ properties • Get performance stats Layout Inspector
69. Overdraw The number of times that a given pixel is drawn in a frame.
70. Understanding Overdraw
71. Understanding Overdraw
72. Detecting Overdraw
73. https://developer.android.com/studio/profile/dev-options-overdraw.html Detecting Overdraw
75. Fixing Overdraw There are 2 common reasons for overdraw: - Redundant backgrounds / Redundant transparency - Wasteful onDraw - Things that aren’t visible at all gets drawn (not using quickReject) - Things that will be overdrawn gets drawn (not using clipRect) Colt McAnlis: https://youtu.be/vkTn3Ule4Ps
76. QuickReject A method to tell if something can be not drawn at all. Call quickReject to see if you can skip drawing of things that will be off screen. REF: https://developer.android.com/reference/android/graphics/Canvas.html
77. ClipRect ClipRect is a way to avoid OverDraw, By keeping your GPU from drawing pixels that you know that will be obscured by other stuff, you refrain from overdraw.
78. GPU Profiling Tool
79. GPU Profiling • A graph per visible app • A bar per frame • Render time corresponds height • 16 millis benchmark (green) • Crossing = skipping frame https://developer.android.com/studio/profile/dev-options-rendering.html
80. 16ms line Dropped Frames :( 14 accented if >16ms Shiney colors!
81. VSync/Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers So many colors.. What do they mean?
82. Vsync / Misc Time Misc = (vsync timestamp) - (timestamp when received) → work on the UI thread between two consecutive frames • Choreographer log : “Missed vsync by XX ms skipping XX frames” High? move work off UI Thread VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
83. Choreographer Coordinates the timing of animations, input and drawing.
84. Ever seen this? 06-29 23:11:17.796: I/Choreographer(691): Skipped 647 frames! The application may be doing too much work on its main thread.
85. Input Handling App code inside an input event callback. High? optimize/offload processing input VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
86. Animation Evaluate all running animators Often : object animator, view property animator, and transitions High?(2milis+) check custom animators / unintended work VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
87. Measure / Layout Executing layout and measure callbacks for needed view. More on that in a few slides :) VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
88. Draw update (/creates) all display lists + cache. High? • complex onDraw() logic • many views invalidated VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
89. Sync & Upload Upload bitmap to the GPU. High? • Too many images • Too big of an image VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
90. Command Issue Execution: Android's 2D renderer issuing commands to OpenGL to draw and redraw all display lists High? • Complex view (custom?) • Many views redrawing: ◦ Invalidation ◦ animation VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
91. Swap Buffers CPU done rendering and wait for GPU ack. High? A lot of GPU work VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
92. VSync/Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers So what do they mean?
93. Vitals dashboard
97. - Why 60 fps - The stages of creating a frame - Tools that helps you find out what wrong with your app What did we talk about today?
98. THANK YOU
99. Want to learn more? useful links: https://www.youtube.com/watch?v=zdQRIYOST64 https://www.youtube.com/watch?v=CaMTIgxCSqU https://www.youtube.com/watch?v=WH9AFhgwmDw https://www.youtube.com/watch?v=we6poP0kw6E https://www.youtube.com/watch?v=1iaHxmfZGGc&t https://www.youtube.com/watch?v=VzYkVL1n4M8 https://www.youtube.com/watch?v=vkTn3Ule4Ps