[Refael Ozeri, Yonatan Levin] GPU. The journey from code to pixel

[Refael Ozeri, Yonatan Levin] GPU. The journey from code to pixel

Presentation from GDG DevFest Ukraine 2018 - the biggest community-driven Google tech conference in the CEE.

Learn more at: https://devfest.gdg.org.ua

__

In this session we will talk about our GPU, did you ever asked yourself what happens behind the scenes when we see a text view? or how does our code turn into pixels?

3a6de6bc902de7f75c0e753b3202ed52?s=128

Google Developers Group Lviv

October 12, 2018
Tweet

Transcript

  1. None
  2. None
  3. None
  4. None
  5. None
  6. None
  7. Romain Guy & Chet Haase

  8. U and I Not really

  9. Yonatan V.Levin Refael Ozeri

  10. None
  11. GPU - The journey from code to pixel

  12. Why do we love apps?

  13. Smooth Experience

  14. Google Play Store

  15. Houzz

  16. Motion is key

  17. How is motion perceived?

  18. How is motion perceived? FPS frames per second

  19. How is motion perceived? 12fps Flip Book 24fps Movies Fluid

    Motion 60fps + No Difference 60fps Smooth Motion
  20. How is motion perceived? So.. 60 fps is our target.

    A new frame is displayed every _____ milliseconds
  21. How is motion perceived? So.. 60 fps is our target.

    A new frame is displayed every 16.666.. milliseconds
  22. None
  23. How did my code turn into pixels? CPU new View()

  24. Say hello to our GPU Yonatan’s GPU.. :)

  25. GPU CPU new View() GPU Polygons Textures

  26. GPU GPU Polygons Textures CPU new View() DisplayList

  27. Wait… DisplayList?! OpenGL?! WHAT?

  28. DisplayList A sequence of graphics commands needed to be executed

    to render a specific view These commands are a mixture of statements that can be directly mapped to OpenGL commands
  29. OpenGL OpenGL is a cross-language, cross-platform API for rendering 2D

    and 3D vector graphics. The API is typically used to interact with a GPU, to achieve hardware-accelerated rendering.
  30. So now we’re all familiar with these big names. BUT..

    we have a big problem!
  31. None
  32. GPU GPU Polygons Textures CPU new View() DisplayList

  33. What do we need to do in 16 ms ?

  34. CPU new View() GPU Measure

  35. GPU Layout CPU

  36. GPU Draw CPU new View() DisplayList

  37. Sync & Upload CPU new View() GPU Polygons Textures DisplayList

  38. Execute CPU new View() GPU Polygons Textures DisplayList

  39. Execute CPU new View() GPU Polygons Textures DisplayList CPU new

    View() GPU Polygons Textures DisplayList Frame Buffer
  40. So, we have a frame buffer that loads our graphics,

    what can go wrong?
  41. None
  42. None
  43. None
  44. Bonus Question.. Anyone knows the name of this game?

  45. Dead Rising

  46. First.. let’s understand how our hardware actually works!

  47. Remember this?

  48. Remember this? That’s 25 fps

  49. Now let’s go SLOW.. Thank you slowmoguys

  50. That’s our TV captured on 1,600 frames per second !

  51. None
  52. That’s our TV on 146,727 frames per second !

  53. CPU GPU So what happened?

  54. CPU GPU Our buffer is finished

  55. CPU GPU Our buffer is not done in time

  56. None
  57. The solution?

  58. The solution? Double Buffer!

  59. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame

    Buffer
  60. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame

    Buffer
  61. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame

    Buffer
  62. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame

    Buffer
  63. Performance boost! The all new RenderThread (Lollipop & above)

  64. input UI Thread animations measure layout draw sync Render Thread

    sync execute get buffer issue swap buffer Graphics get buffer get buffer See more @ I/O 2018 - Drawn out lecture - https://youtu.be/zdQRIYOST64
  65. Tools

  66. Layout Inspector (Formerly Hierarchy Viewer) WHY?

  67. Tools→ Layout Inspector • Display complete view hierarchy • Access

    to all the views’ properties • Get performance stats Layout Inspector
  68. None
  69. Overdraw The number of times that a given pixel is

    drawn in a frame.
  70. Understanding Overdraw

  71. Understanding Overdraw

  72. Detecting Overdraw

  73. https://developer.android.com/studio/profile/dev-options-overdraw.html Detecting Overdraw

  74. None
  75. Fixing Overdraw There are 2 common reasons for overdraw: -

    Redundant backgrounds / Redundant transparency - Wasteful onDraw - Things that aren’t visible at all gets drawn (not using quickReject) - Things that will be overdrawn gets drawn (not using clipRect) Colt McAnlis: https://youtu.be/vkTn3Ule4Ps
  76. QuickReject A method to tell if something can be not

    drawn at all. Call quickReject to see if you can skip drawing of things that will be off screen. REF: https://developer.android.com/reference/android/graphics/Canvas.html
  77. ClipRect ClipRect is a way to avoid OverDraw, By keeping

    your GPU from drawing pixels that you know that will be obscured by other stuff, you refrain from overdraw.
  78. GPU Profiling Tool

  79. GPU Profiling • A graph per visible app • A

    bar per frame • Render time corresponds height • 16 millis benchmark (green) • Crossing = skipping frame https://developer.android.com/studio/profile/dev-options-rendering.html
  80. 16ms line Dropped Frames :( 14 accented if >16ms Shiney

    colors!
  81. VSync/Misc Input Animation Measure/ Layout Draw Sync & Upload Command

    Issue Swap Buffers So many colors.. What do they mean?
  82. Vsync / Misc Time Misc = (vsync timestamp) - (timestamp

    when received) → work on the UI thread between two consecutive frames • Choreographer log : “Missed vsync by XX ms skipping XX frames” High? move work off UI Thread VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
  83. Choreographer Coordinates the timing of animations, input and drawing.

  84. Ever seen this? 06-29 23:11:17.796: I/Choreographer(691): Skipped 647 frames! The

    application may be doing too much work on its main thread.
  85. Input Handling App code inside an input event callback. High?

    optimize/offload processing input VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
  86. Animation Evaluate all running animators Often : object animator, view

    property animator, and transitions High?(2milis+) check custom animators / unintended work VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
  87. Measure / Layout Executing layout and measure callbacks for needed

    view. More on that in a few slides :) VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
  88. Draw update (/creates) all display lists + cache. High? •

    complex onDraw() logic • many views invalidated VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
  89. Sync & Upload Upload bitmap to the GPU. High? •

    Too many images • Too big of an image VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
  90. Command Issue Execution: Android's 2D renderer issuing commands to OpenGL

    to draw and redraw all display lists High? • Complex view (custom?) • Many views redrawing: ◦ Invalidation ◦ animation VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
  91. Swap Buffers CPU done rendering and wait for GPU ack.

    High? A lot of GPU work VSync/ Misc Input Animation Measure/ Layout Draw Sync & Upload Command Issue Swap Buffers
  92. VSync/Misc Input Animation Measure/ Layout Draw Sync & Upload Command

    Issue Swap Buffers So what do they mean?
  93. Vitals dashboard

  94. None
  95. None
  96. None
  97. - Why 60 fps - The stages of creating a

    frame - Tools that helps you find out what wrong with your app What did we talk about today?
  98. THANK YOU

  99. Want to learn more? useful links: https://www.youtube.com/watch?v=zdQRIYOST64 https://www.youtube.com/watch?v=CaMTIgxCSqU https://www.youtube.com/watch?v=WH9AFhgwmDw https://www.youtube.com/watch?v=we6poP0kw6E

    https://www.youtube.com/watch?v=1iaHxmfZGGc&t https://www.youtube.com/watch?v=VzYkVL1n4M8 https://www.youtube.com/watch?v=vkTn3Ule4Ps