View To Pixel 2.0 (Droidcon SF 17')

3142db3adb711e247e371153b5777e04?s=47 Britt Barak
November 05, 2017

View To Pixel 2.0 (Droidcon SF 17')

On what's going on in the Android system when it created pixels on the screen, out of the Views that we write in code.

Also here: https://docs.google.com/presentation/d/1__ezgkz5DzFWxHGy4p0wx5NLr96aML5_asskM6OsEJM/present#slide=id.g299bece29b_1_927

Presented in Droidcon SF 2017.

3142db3adb711e247e371153b5777e04?s=128

Britt Barak

November 05, 2017
Tweet

Transcript

  1. View To Pixel 2.0 Britt Barak DroidconSF 2017

  2. None
  3. Android Lead Android Academy TLV Women Techmakers IL Britt Barak

  4. Today’s story

  5. CPU new View()

  6. Rasterization From a high level object into pixels on screen

    (or in texture).
  7. GPU - For The Rescue !

  8. CPU new View() GPU

  9. CPU new View() GPU Polygons Textures

  10. CPU new View() GPU Polygons Textures DisplayList

  11. Refresh Rate 60 Hz

  12. CPU new View() GPU Polygons Textures DisplayList Frame Rate 60

    fps
  13. What happens if...

  14. CPU GPU

  15. CPU GPU

  16. CPU GPU

  17. None
  18. What do we do?

  19. Double Buffer

  20. Double Triple Buffer

  21. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame

    Buffer
  22. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame

    Buffer
  23. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame

    Buffer
  24. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame

    Buffer
  25. CPU new View() GPU Polygons Textures DisplayList Back Buffer Frame

    Buffer
  26. VSync

  27. In some more details...

  28. CPU GPU Vsync

  29. CPU new View() GPU Measure

  30. CPU GPU Layout

  31. CPU GPU new View() DisplayList Draw

  32. CPU new View() GPU Polygons Textures DisplayList Sync & Upload

  33. CPU new View() GPU Polygons Textures DisplayList Execute Back Buffer

  34. CPU new View() GPU Polygons Textures DisplayList Swap buffers Back

    Buffer
  35. Measure/ Layout Draw Sync & Upload Execute Swap Buffers VSync/Misc

  36. actually...

  37. Input

  38. Animation

  39. VSync/Misc Input Animation Measure/ Layout Draw Sync & Upload Command

    Issue Swap Buffers
  40. Let’s go a bit deeper...

  41. Window Manager

  42. Window Manager lifecycles, input events, focus events, screen orientation, transitions,

    animations, position, transforms, z-order...
  43. • Knows all surfaces • Only service that can modify

    display’s content SurfaceFlinger
  44. Status bar App UI Navigation bar

  45. “Listen, I have another ‘woower’...” Window Manager SurfaceFlinger

  46. “K” Window Manager SurfaceFlinger

  47. Window Manager SurfaceFlinger Surface BufferQueue

  48. Window Manager SurfaceFlinger Surface BufferQueue

  49. Surface BufferQueue Buffer Pool Queue Gralloc (HAL interface)

  50. App Window Manager “Here’s a Binder object!”

  51. App “I have something to display!” Binder Buffer Queue

  52. App Binder Buffer Queue “Here’s a free buffer”

  53. App Binder Buffer Queue

  54. App Binder Buffer Queue “Done!”

  55. App Binder Buffer Queue

  56. Binder Buffer Queue “I’m ready!” SurfaceFlinger Display VSYNC “K”

  57. Binder Buffer Queue SurfaceFlinger Display Binder Buffer Queue Binder Buffer

    Queue
  58. SurfaceFlinger Display Hardware Composer HAL Binder Buffer Queue Binder Buffer

    Queue Binder Buffer Queue
  59. SurfaceFlinger Display Hardware Composer HAL Binder Buffer Queue Binder Buffer

    Queue Binder Buffer Queue
  60. Let’s talk some more about creating the frame...

  61. VSync/Misc Input Animation Measure/ Layout Draw Sync & Upload Command

    Issue Swap Buffers CPU
  62. 1. Order is guaranteed 2. A step is called up

    to once
  63. Choreographer

  64. I/Choreographer(1378): Skipped 12 frames! The application may be doing too

    much work on its main thread.
  65. Process Main (/UI) Thread

  66. Main Thread (UI Thread) Process System Event Input Event Service

    Application UI Drawing
  67. Java Thread run() start()

  68. Main Thread (UI Thread) Looper Message queue Process

  69. Main Thread (UI Thread) Looper Message queue Handler Handle MSG

    Send MSG Process
  70. Main Thread (UI Thread) Looper Message queue Handler1 Handle MSG

    Send MSG Process Handler2 Handle MSG Send MSG
  71. private Choreographer (Looper looper, int vsyncSource) ;

  72. Choreographer.java private final class FrameHandler extends Handler { @Override public

    void handleMessage(Message msg) { switch (msg.what) { case MSG_DO_FRAME: doFrame(System.nanoTime(), 0); break; case MSG_DO_SCHEDULE_VSYNC: doScheduleVsync(); break; case MSG_DO_SCHEDULE_CALLBACK: doScheduleCallback(msg.arg1); break; } } }
  73. 1. Order is guaranteed. 2. Steps happen up to once

    Choreographer.java void doFrame(long frameTimeNanos, int frame) { //... doCallbacks(Choreographer.CALLBACK_INPUT, frameTimeNanos); doCallbacks(Choreographer.CALLBACK_ANIMATION, frameTimeNanos); doCallbacks(Choreographer.CALLBACK_TRAVERSAL, frameTimeNanos); doCallbacks(Choreographer.CALLBACK_COMMIT, frameTimeNanos); //…
  74. Choreographer.java void doCallbacks(int callbackType, long frameTimeNanos) { callbacks = mCallbackQueues[callbackType]

    .extractDueCallbacksLocked(now / TimeUtils.NANOS_PER_MS); for (CallbackRecord c = callbacks; c != null; c = c.next) { //… c.run(frameTimeNanos);
  75. Measure Layout Draw!

  76. What can we do? • Flatten hierarchy • <Merge> on

    <include> • Avoid when near the hierarchy top • Avoid when many layouts • Minimize requestLayout() • ConstrainLayout
  77. Draw()

  78. View.java public void draw(Canvas canvas)

  79. //Step 1: draw background drawBackground(canvas); // Step 2, save the

    canvas' layers (prepare Fading, optional) canvas.saveLayer(…) //Step 3, draw view's content onDraw(canvas); public void draw(Canvas canvas) {
  80. // Step 4, draw children dispatchDraw(canvas); // Step 5, draw

    the fade effect and restore layers (optional) final Shader fade = scrollabilityCache.shader; paint.setShader(fade); canvas.drawRect(left, top, right, top , paint); public void draw(Canvas canvas) {
  81. // Step 6, draw decorations (foreground, scrollbars) onDrawForeground(canvas); // Step

    7, draw the default focus highlight drawDefaultFocusHighlight(canvas); public void draw(Canvas canvas) {
  82. background.draw(canvas) scrollBar.draw(canvas) foreground.draw(canvas)

  83. GPU Overdraw Tool https://developer.android.com/studio/profile/dev-options-overdraw.html

  84. A bit more about Canvas

  85. 1. Primitives Canvas. drawPath(), drawOval(), drawRect()

  86. 2. PathEffect paint.setPathEffect() ComposePathEffect, CornerPathEffect, DashPathEffect, DiscretePathEffect, PathDashPathEffect, SumPathEffect

  87. 3. Alpha Decide alpha channel for each pixel paint.setAlpha()

  88. 4. Shading determine each pixel’s color (ARGB). paint.setColor(), paint.setColorFilter(), paint.setShader()

    Chiu-Ki Chan: Fun With Android Shaders And Filters
  89. public void draw(Canvas canvas) { //... // Step 4, draw

    children dispatchDraw(canvas); // Step 5, draw the fade effect and restore layers (optional) final Shader fade = scrollabilityCache.shader; paint.setShader(fade); canvas.drawRect(left, top, right, top , paint);
  90. Shader paint.setShader(shader) LinearGradient (on fade), RadialGradient, SweepGradient, BitmapShader, ComposeShader https://developer.android.com/reference/android/graphics/Shader.html

  91. 5. Transfer How to transfer color to destination Bitmap paint.setXfermode()

    Source Image Destination Image Transfer Mode Defaults to : PorterDuffXferMode (SRC_OVER) https://developer.android.com/reference/android/graphics/PorterDuffXfermode.html
  92. None
  93. None
  94. None
  95. Binder Buffer Queue surface canvas openGL openGLES SurfaceFlinger

  96. VSync/Misc Input Animation Measure/ Layout Draw Sync & Upload Command

    Issue Swap Buffers
  97. GPU Profiling Tool • A graph per visible app •

    A bar per frame • 16ms benchmark • Crossing = skipping frame https://developer.android.com/studio/profile/dev-options-rendering.html
  98. Frame Metrics On Android N+ - Per-Frame data: - Window.OnFrameMetricsAvailableListener()

    - Aggregated data: - FrameMetricsAggregator (Support lib v26) https://developer.android.com/reference/android/view/FrameMetrics.html
  99. None
  100. Thank you! Keep in touch! Britt Barak @brittBarak