$30 off During Our Annual Pro Sale. View Details »

View To Pixel 2.0 (Droidcon SF 17')

Britt Barak
November 05, 2017

View To Pixel 2.0 (Droidcon SF 17')

On what's going on in the Android system when it created pixels on the screen, out of the Views that we write in code.

Also here: https://docs.google.com/presentation/d/1__ezgkz5DzFWxHGy4p0wx5NLr96aML5_asskM6OsEJM/present#slide=id.g299bece29b_1_927

Presented in Droidcon SF 2017.

Britt Barak

November 05, 2017
Tweet

More Decks by Britt Barak

Other Decks in Technology

Transcript

  1. View To Pixel 2.0
    Britt Barak
    DroidconSF 2017

    View Slide

  2. View Slide

  3. Android Lead
    Android Academy TLV
    Women Techmakers IL
    Britt Barak

    View Slide

  4. Today’s story

    View Slide

  5. CPU
    new
    View()

    View Slide

  6. Rasterization
    From a high level object
    into pixels on screen (or in texture).

    View Slide

  7. GPU -
    For The Rescue !

    View Slide

  8. CPU
    new
    View()
    GPU

    View Slide

  9. CPU
    new
    View()
    GPU
    Polygons
    Textures

    View Slide

  10. CPU
    new
    View()
    GPU
    Polygons
    Textures
    DisplayList

    View Slide

  11. Refresh Rate
    60
    Hz

    View Slide

  12. CPU
    new
    View()
    GPU
    Polygons
    Textures
    DisplayList
    Frame Rate
    60
    fps

    View Slide

  13. What
    happens
    if...

    View Slide

  14. CPU GPU

    View Slide

  15. CPU GPU

    View Slide

  16. CPU GPU

    View Slide

  17. View Slide

  18. What
    do we do?

    View Slide

  19. Double Buffer

    View Slide

  20. Double Triple Buffer

    View Slide

  21. CPU
    new
    View()
    GPU
    Polygons
    Textures
    DisplayList
    Back
    Buffer
    Frame
    Buffer

    View Slide

  22. CPU
    new
    View()
    GPU
    Polygons
    Textures
    DisplayList
    Back
    Buffer
    Frame
    Buffer

    View Slide

  23. CPU
    new
    View()
    GPU
    Polygons
    Textures
    DisplayList
    Back
    Buffer
    Frame
    Buffer

    View Slide

  24. CPU
    new
    View()
    GPU
    Polygons
    Textures
    DisplayList
    Back
    Buffer
    Frame
    Buffer

    View Slide

  25. CPU
    new
    View()
    GPU
    Polygons
    Textures
    DisplayList
    Back
    Buffer
    Frame
    Buffer

    View Slide

  26. VSync

    View Slide

  27. In some more details...

    View Slide

  28. CPU GPU
    Vsync

    View Slide

  29. CPU
    new
    View()
    GPU
    Measure

    View Slide

  30. CPU GPU
    Layout

    View Slide

  31. CPU GPU
    new
    View()
    DisplayList
    Draw

    View Slide

  32. CPU
    new
    View()
    GPU
    Polygons
    Textures
    DisplayList
    Sync & Upload

    View Slide

  33. CPU
    new
    View()
    GPU
    Polygons
    Textures
    DisplayList
    Execute
    Back Buffer

    View Slide

  34. CPU
    new
    View()
    GPU
    Polygons
    Textures
    DisplayList
    Swap buffers
    Back Buffer

    View Slide

  35. Measure/ Layout
    Draw
    Sync & Upload
    Execute
    Swap Buffers
    VSync/Misc

    View Slide

  36. actually...

    View Slide

  37. Input

    View Slide

  38. Animation

    View Slide

  39. VSync/Misc
    Input
    Animation
    Measure/ Layout
    Draw
    Sync & Upload
    Command Issue
    Swap Buffers

    View Slide

  40. Let’s go a bit deeper...

    View Slide

  41. Window
    Manager

    View Slide

  42. Window
    Manager
    lifecycles, input events, focus events, screen orientation,
    transitions, animations, position, transforms, z-order...

    View Slide

  43. ● Knows all surfaces
    ● Only service that can modify display’s content
    SurfaceFlinger

    View Slide

  44. Status bar
    App UI
    Navigation bar

    View Slide

  45. “Listen, I have
    another ‘woower’...”
    Window
    Manager
    SurfaceFlinger

    View Slide

  46. “K”
    Window
    Manager
    SurfaceFlinger

    View Slide

  47. Window
    Manager
    SurfaceFlinger
    Surface
    BufferQueue

    View Slide

  48. Window
    Manager
    SurfaceFlinger
    Surface
    BufferQueue

    View Slide

  49. Surface
    BufferQueue
    Buffer Pool
    Queue
    Gralloc
    (HAL interface)

    View Slide

  50. App
    Window
    Manager
    “Here’s a Binder
    object!”

    View Slide

  51. App
    “I have something to
    display!”
    Binder
    Buffer
    Queue

    View Slide

  52. App
    Binder
    Buffer
    Queue
    “Here’s a free
    buffer”

    View Slide

  53. App
    Binder
    Buffer
    Queue

    View Slide

  54. App
    Binder
    Buffer
    Queue
    “Done!”

    View Slide

  55. App
    Binder
    Buffer
    Queue

    View Slide

  56. Binder
    Buffer
    Queue
    “I’m ready!”
    SurfaceFlinger Display
    VSYNC
    “K”

    View Slide

  57. Binder
    Buffer
    Queue SurfaceFlinger Display
    Binder
    Buffer
    Queue
    Binder
    Buffer
    Queue

    View Slide

  58. SurfaceFlinger Display
    Hardware
    Composer
    HAL
    Binder
    Buffer
    Queue
    Binder
    Buffer
    Queue
    Binder
    Buffer
    Queue

    View Slide

  59. SurfaceFlinger Display
    Hardware
    Composer
    HAL
    Binder
    Buffer
    Queue
    Binder
    Buffer
    Queue
    Binder
    Buffer
    Queue

    View Slide

  60. Let’s talk some more about
    creating the frame...

    View Slide

  61. VSync/Misc
    Input
    Animation
    Measure/ Layout
    Draw
    Sync & Upload
    Command Issue
    Swap Buffers
    CPU

    View Slide

  62. 1. Order is guaranteed
    2. A step is called up to once

    View Slide

  63. Choreographer

    View Slide

  64. I/Choreographer(1378): Skipped 12 frames!
    The application may be doing too much work on its
    main thread.

    View Slide

  65. Process
    Main (/UI) Thread

    View Slide

  66. Main Thread (UI Thread)
    Process
    System Event Input Event Service Application UI Drawing

    View Slide

  67. Java Thread
    run()
    start()

    View Slide

  68. Main Thread (UI Thread)
    Looper
    Message queue
    Process

    View Slide

  69. Main Thread (UI Thread)
    Looper
    Message queue
    Handler
    Handle
    MSG
    Send
    MSG
    Process

    View Slide

  70. Main Thread (UI Thread)
    Looper
    Message queue
    Handler1
    Handle
    MSG
    Send
    MSG
    Process
    Handler2
    Handle
    MSG
    Send
    MSG

    View Slide

  71. private Choreographer
    (Looper looper, int vsyncSource) ;

    View Slide

  72. Choreographer.java
    private final class FrameHandler extends Handler {
    @Override
    public void handleMessage(Message msg) {
    switch (msg.what) {
    case MSG_DO_FRAME:
    doFrame(System.nanoTime(), 0);
    break;
    case MSG_DO_SCHEDULE_VSYNC:
    doScheduleVsync();
    break;
    case MSG_DO_SCHEDULE_CALLBACK:
    doScheduleCallback(msg.arg1);
    break;
    }
    }
    }

    View Slide

  73. 1. Order is guaranteed.
    2. Steps happen up to once
    Choreographer.java
    void doFrame(long frameTimeNanos, int frame) {
    //...
    doCallbacks(Choreographer.CALLBACK_INPUT, frameTimeNanos);
    doCallbacks(Choreographer.CALLBACK_ANIMATION, frameTimeNanos);
    doCallbacks(Choreographer.CALLBACK_TRAVERSAL, frameTimeNanos);
    doCallbacks(Choreographer.CALLBACK_COMMIT, frameTimeNanos);
    //…

    View Slide

  74. Choreographer.java
    void doCallbacks(int callbackType, long frameTimeNanos) {
    callbacks = mCallbackQueues[callbackType]
    .extractDueCallbacksLocked(now / TimeUtils.NANOS_PER_MS);
    for (CallbackRecord c = callbacks; c != null; c = c.next) {
    //…
    c.run(frameTimeNanos);

    View Slide

  75. Measure Layout Draw!

    View Slide

  76. What can we do?
    ● Flatten hierarchy
    ● on
    ● Avoid when near the hierarchy top
    ● Avoid when many layouts
    ● Minimize requestLayout()
    ● ConstrainLayout

    View Slide

  77. Draw()

    View Slide

  78. View.java
    public void draw(Canvas canvas)

    View Slide

  79. //Step 1: draw background
    drawBackground(canvas);
    // Step 2, save the canvas' layers (prepare Fading, optional)
    canvas.saveLayer(…)
    //Step 3, draw view's content
    onDraw(canvas);
    public void draw(Canvas canvas) {

    View Slide

  80. // Step 4, draw children
    dispatchDraw(canvas);
    // Step 5, draw the fade effect and restore layers (optional)
    final Shader fade = scrollabilityCache.shader;
    paint.setShader(fade);
    canvas.drawRect(left, top, right, top , paint);
    public void draw(Canvas canvas) {

    View Slide

  81. // Step 6, draw decorations (foreground, scrollbars)
    onDrawForeground(canvas);
    // Step 7, draw the default focus highlight
    drawDefaultFocusHighlight(canvas);
    public void draw(Canvas canvas) {

    View Slide

  82. background.draw(canvas)
    scrollBar.draw(canvas)
    foreground.draw(canvas)

    View Slide

  83. GPU Overdraw Tool
    https://developer.android.com/studio/profile/dev-options-overdraw.html

    View Slide

  84. A bit more about Canvas

    View Slide

  85. 1. Primitives
    Canvas.
    drawPath(), drawOval(), drawRect()

    View Slide

  86. 2. PathEffect
    paint.setPathEffect()
    ComposePathEffect, CornerPathEffect, DashPathEffect,
    DiscretePathEffect, PathDashPathEffect, SumPathEffect

    View Slide

  87. 3. Alpha
    Decide alpha channel for each pixel
    paint.setAlpha()

    View Slide

  88. 4. Shading
    determine each pixel’s color (ARGB).
    paint.setColor(), paint.setColorFilter(), paint.setShader()
    Chiu-Ki Chan: Fun With Android Shaders And Filters

    View Slide

  89. public void draw(Canvas canvas) {
    //...
    // Step 4, draw children
    dispatchDraw(canvas);
    // Step 5, draw the fade effect and restore layers (optional)
    final Shader fade = scrollabilityCache.shader;
    paint.setShader(fade);
    canvas.drawRect(left, top, right, top , paint);

    View Slide

  90. Shader
    paint.setShader(shader)
    LinearGradient (on fade), RadialGradient,
    SweepGradient, BitmapShader, ComposeShader
    https://developer.android.com/reference/android/graphics/Shader.html

    View Slide

  91. 5. Transfer
    How to transfer color to destination Bitmap
    paint.setXfermode()
    Source Image Destination Image Transfer Mode
    Defaults to :
    PorterDuffXferMode
    (SRC_OVER)
    https://developer.android.com/reference/android/graphics/PorterDuffXfermode.html

    View Slide

  92. View Slide

  93. View Slide

  94. View Slide

  95. Binder
    Buffer
    Queue
    surface
    canvas
    openGL
    openGLES
    SurfaceFlinger

    View Slide

  96. VSync/Misc
    Input
    Animation
    Measure/ Layout
    Draw
    Sync & Upload
    Command Issue
    Swap Buffers

    View Slide

  97. GPU Profiling Tool
    ● A graph per visible app
    ● A bar per frame
    ● 16ms benchmark
    ● Crossing = skipping frame
    https://developer.android.com/studio/profile/dev-options-rendering.html

    View Slide

  98. Frame Metrics
    On Android N+
    - Per-Frame data:
    - Window.OnFrameMetricsAvailableListener()
    - Aggregated data:
    - FrameMetricsAggregator (Support lib v26)
    https://developer.android.com/reference/android/view/FrameMetrics.html

    View Slide

  99. View Slide

  100. Thank you!
    Keep in touch!
    Britt Barak
    @brittBarak

    View Slide