Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Android Graphics Performance

Android Graphics Performance

Tips, tricks, tools, and techniques used by engineers from the Android UI Graphics team for getting the best performance and smoothest UI from Android applications.

Video available at:
https://developers.google.com/events/io/sessions/325418001

Romain Guy

May 16, 2013
Tweet

More Decks by Romain Guy

Other Decks in Programming

Transcript

  1. Welcome/Android Graphics & Performance
    Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)

    View full-size slide

  2. &
    Graphics Performance
    Welcome/Android Graphics & Performance
    Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)

    View full-size slide

  3. &
    Chet Haase Romain Guy
    Welcome/Android Graphics & Performance
    Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)

    View full-size slide

  4. Architecture
    1

    View full-size slide

  5. Reordering & merging
    A future update of Android introduces a major change in the hardware accelerated 2D rendering pipeline:
    re-ordering and merging of drawing commands

    View full-size slide

  6. Save Cancel
    Maximize compatibility
    Include metadata
    This allows for more efficient rendering without changing anything in your application. Here is an example of a
    simple Android UI

    View full-size slide

  7. Until now Android would always render drawing commands in order. Android now first re-orders the commands to
    find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to
    reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of-
    order draw calls.

    View full-size slide

  8. Include metadata Maximize compatibility Cancel
    Save
    Order of drawing commands
    Until now Android would always render drawing commands in order. Android now first re-orders the commands to
    find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to
    reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of-
    order draw calls.

    View full-size slide

  9. Include metadata Maximize compatibility Cancel
    Save
    1. Re-ordering
    Until now Android would always render drawing commands in order. Android now first re-orders the commands to
    find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to
    reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of-
    order draw calls.

    View full-size slide

  10. Include metadata
    Maximize compatibility
    Cancel
    Save
    2. Merging
    Until now Android would always render drawing commands in order. Android now first re-orders the commands to
    find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to
    reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of-
    order draw calls.

    View full-size slide

  11. Let’s demo re-ordering and merging on a real device, using Google+ as an example

    View full-size slide

  12. Multi-threading
    In a future update of Android, the hardware accelerated 2D rendering pipeline makes better
    use of multi-core systems by using multiple threads to perform certain tasks.

    View full-size slide

  13. The blue bar show the UI thread performing normal rendering. However the renderer now uses
    Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.)
    Paths are also now generated on background threads, as shown in yellow.

    View full-size slide

  14. Drawing
    The blue bar show the UI thread performing normal rendering. However the renderer now uses
    Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.)
    Paths are also now generated on background threads, as shown in yellow.

    View full-size slide

  15. Drawing
    Shadows
    Shadows
    Shadows
    Shadows
    The blue bar show the UI thread performing normal rendering. However the renderer now uses
    Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.)
    Paths are also now generated on background threads, as shown in yellow.

    View full-size slide

  16. Drawing
    Paths
    Paths
    Shadows
    Shadows
    Shadows
    Shadows
    The blue bar show the UI thread performing normal rendering. However the renderer now uses
    Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.)
    Paths are also now generated on background threads, as shown in yellow.

    View full-size slide

  17. Non-rectangular clipping
    A future update of Android adds support for hardware accelerated non-rectangular clipping
    This includes clipping with paths (circles, curves, rounded rects, etc.) and transformed rects
    (3D rotations for instance)

    View full-size slide

  18. @Override
    protected void onDraw(Canvas canvas) {
    // Clip with a shape
    Path clip = getPath();
    canvas.clipPath(clip);
    // Draw the content
    for (int i = 0; i < mLines,length; i++) {
    TextLine line = mLines[i];
    canvas.drawText(line.text,
    line.x, line.y, mPaint);
    }
    }
    Non-rectangular clips can also be achieved by specifying the Region.Op parameter
    of the various clip*() methods (you can add, xor, subtract, etc.). Non-rect clipping can be triggered by rotations.

    View full-size slide

  19. Developer Tools
    2

    View full-size slide

  20. Overdraw
    In developer options you can now turn on a “Show GPU overdraw” feature
    Apps will be highlighted with various colors. Each color represents the amount of overdraw
    for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.

    View full-size slide

  21. Overdraw
    In developer options you can now turn on a “Show GPU overdraw” feature
    Apps will be highlighted with various colors. Each color represents the amount of overdraw
    for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.

    View full-size slide

  22. Overdraw
    In developer options you can now turn on a “Show GPU overdraw” feature
    Apps will be highlighted with various colors. Each color represents the amount of overdraw
    for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.

    View full-size slide

  23. Overdraw
    Blue
    Green
    Red
    Deep Red
    1x
    2x
    3x
    4x
    In developer options you can now turn on a “Show GPU overdraw” feature
    Apps will be highlighted with various colors. Each color represents the amount of overdraw
    for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.

    View full-size slide

  24. 0
    3
    6
    9
    12
    15
    Time in ms
    Frames
    Update display lists Process display lists Swap buffers
    In Android 4.1 we introduced a new rendering profiling tool in developer settings
    This tool was useful to create performance graphs such as this one
    It was however difficult to use since it required command line tools and a spreadsheet

    View full-size slide

  25. In a future update of Android profiling is a lot easier. You can now see the profiling
    graph directly on screen. The old method is also available.
    The green line indicates the 16ms threshold (60fps.)

    View full-size slide

  26. In a future update of Android profiling is a lot easier. You can now see the profiling
    graph directly on screen. The old method is also available.
    The green line indicates the 16ms threshold (60fps.)

    View full-size slide

  27. performTraversals
    draw
    getDL drawDisplayList
    systrace
    flush drawing commands
    systrace was introduced in Android 4.1 and we’ve added several improvements in 4.2
    and future updates of Android.

    View full-size slide

  28. android:sdk $ cd platform-tools/
    android:platform-tools $ 㾑
    ./systrace.py gfx view freq sched
    In a future update of Android systrace is also easier to run, no need to change developer options
    All you need is to run the script from the command line
    In this example we’re tracing graphics (gfx), UI toolkit (view), CPU frequencies (freq)
    and the kernel scheduler (sched)

    View full-size slide

  29. And since systrace is easier to use, we added more information. If you enable OpenGL tracing
    in developer options you will be able to see each individual OpenGL call in systrace (use the gfx tag when running
    systrace.) You can also enable glGetError() checks after each GL call.

    View full-size slide

  30. import android.os.Trace;
    @Override
    public View getView(int pos, View view, ViewGroup parent) {
    Trace.beginSection("getView");
    if (view == null) {
    view = createView();
    }
    // Trace time spent binding data
    Trace.beginSection("bind");
    bindView(pos, view);
    Trace.endSection();
    Trace.endSection();
    return view;
    }
    We’re introducing a new API, android.os.Trace, to let you add your own information to
    systrace. Systrace events are lightweight and add almost no overhead. Using it is easy:
    simply wrap the section of code to trace with beginSection/endSection. You can nest sections.

    View full-size slide

  31. android:sdk $ cd platform-tools/
    android:platform-tools $ 㾑
    ./systrace.py -a com.example.myapp
    You won’t see your events by default, you must specify the package name of your app
    using the -a option when invoking systrace. You can specify several packages and combine
    it with the built-in tags such as gfx and view.

    View full-size slide

  32. Tips & Tricks
    3

    View full-size slide

  33. Overdraw demo

    View full-size slide

  34. Trilinear filtering
    Trilinear filtering was introduced in Android 4.2 and can be used to improve the visual
    quality of your bitmaps if you scale them down to less than 50% of their original size

    View full-size slide

  35. Here is an example with trilinear filtering turned off (left) and on (right)
    You can see the trilinear filtering does a much better job at smoothing the image
    (The images have been scaled up many times)

    View full-size slide

  36. Off On
    Here is an example with trilinear filtering turned off (left) and on (right)
    You can see the trilinear filtering does a much better job at smoothing the image
    (The images have been scaled up many times)

    View full-size slide

  37. private void loadData() {
    // Load bitmap
    Bitmap b = getBitmap();
    // Enable trilinear filtering
    b.setHasMipMap(true);
    }
    Here is how you can enable trilinear filtering on a Bitmap
    You can also enable trilinear filtering on a BitmapDrawable if you prefer

    View full-size slide

  38. android:mipMap="true"
    android:src="@drawable/my_drawable" />
    You can of course enable trilinear filtering in your XML drawable definition

    View full-size slide

  39. Canvas layers
    Let’s talk a bit about Canvas layers (as opposed to View.setLayerType.)
    There are two types of layers and using them can create interesting performance issues

    View full-size slide

  40. @Override
    protected void onDraw(Canvas canvas) {
    // Create a clipped layer
    canvas.save();
    canvas.saveLayer(x, y, width, height,
    Canvas.CLIP_TO_LAYER_SAVE_FLAG);
    // Draw stuff
    canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null);
    canvas.restore();
    }
    The first type of layers is called “clipped layer”. Such layers are created by passing the flag
    Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer().

    View full-size slide

  41. saveLayer()
    Here is how clipped layers work. After calling saveLayer(), only the drawing commands that intersect with that layer
    will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an
    offscreen render target (bitmap in software, FBO+texture in hardware.)

    View full-size slide

  42. Here is how clipped layers work. After calling saveLayer(), only the drawing commands that intersect with that layer
    will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an
    offscreen render target (bitmap in software, FBO+texture in hardware.)

    View full-size slide

  43. Here is how clipped layers work. After calling saveLayer(), only the drawing commands that intersect with that layer
    will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an
    offscreen render target (bitmap in software, FBO+texture in hardware.)

    View full-size slide

  44. @Override
    protected void onDraw(Canvas canvas) {
    // Create an unclipped layer
    canvas.save();
    canvas.saveLayer(x, y, width, height, 0);
    // Draw stuff
    canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null);
    canvas.restore();
    }
    The second type of layers is called “unclipped layer”. Such layers are created by NOT passing the flag
    Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer().

    View full-size slide

  45. saveLayer()
    With unclipped layers, a drawing command will be drawn in any layer it intersects, including the original canvas. In
    this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This
    is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N
    times (N=number of active layers.)

    View full-size slide

  46. With unclipped layers, a drawing command will be drawn in any layer it intersects, including the original canvas. In
    this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This
    is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N
    times (N=number of active layers.)

    View full-size slide

  47. With unclipped layers, a drawing command will be drawn in any layer it intersects, including the original canvas. In
    this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This
    is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N
    times (N=number of active layers.)

    View full-size slide

  48. Using alpha with care
    Android offers many easy ways to apply alpha on a view. Using alpha can however have a measurable impact
    the performance of your application.

    View full-size slide

  49. Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is
    they all trigger the same side effect: the creation of an expensive offscreen buffer.

    View full-size slide

  50. view.setAlpha(0.5f);
    View.ALPHA.set(view, 0.5f);
    ObjectAnimation.ofFloat(view, "alpha", 0.5f)
    view.animate().alpha(0.5f);
    view.setAnimation(new AlphaAnimation(1.0f, 0.5f));
    Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is
    they all trigger the same side effect: the creation of an expensive offscreen buffer.

    View full-size slide

  51. view.setAlpha(0.5f);
    View.ALPHA.set(view, 0.5f);
    ObjectAnimation.ofFloat(view, "alpha", 0.5f)
    view.animate().alpha(0.5f);
    view.setAnimation(new AlphaAnimation(1.0f, 0.5f));
    Canvas.saveLayerAlpha(l, t, r, b, 127,
    Canvas.CLIP_TO_LAYER_SAVE_FLAG);
    ==
    Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is
    they all trigger the same side effect: the creation of an expensive offscreen buffer.

    View full-size slide

  52. Why is it important to use a separate buffer to apply the alpha? Here is an example. Let’s imagine a View containing 3
    photos (bitmaps.) If we animate the opacity to 50% without using a separate buffer the result would look like this one.

    View full-size slide

  53. Why is it important to use a separate buffer to apply the alpha? Here is an example. Let’s imagine a View containing 3
    photos (bitmaps.) If we animate the opacity to 50% without using a separate buffer the result would look like this one.

    View full-size slide

  54. Instead of applying the alpha separately, we can use a separate buffer in which we draw the 3 photos and then we
    apply alpha to that buffer. The result can be seen above and looks a lot better.

    View full-size slide

  55. Instead of applying the alpha separately, we can use a separate buffer in which we draw the 3 photos and then we
    apply alpha to that buffer. The result can be seen above and looks a lot better.

    View full-size slide

  56. lternatives
    Let’s examine various solutions you can use to maximize performance when applying alpha on a View.

    View full-size slide

  57. // Not this
    textView.setAlpha(alpha);
    // But this
    int newTextColor = (int) (0xFF * alpha) << 24 |
    baseTextColor & 0xFFFFFF;
    textView.setTextColor(newTextColor);
    If you are setting alpha on a TextView and you don’t have a background you can simply
    bake the alpha in the text color instead. This also applies to background colors, drawing
    primitives, etc.

    View full-size slide

  58. // Not this
    imageView.setAlpha(alpha);
    // But this
    imageView.setImageAlpha((int) (alpha * 255));
    Similarly with ImageView, you can apply the alpha directly to the image instead of the
    View itself.

    View full-size slide

  59. // Not this
    customView.setAlpha(alpha);
    // But this
    int alpha = (int)
    (255 * slider.getProgress() / 100.0f);
    paint.setAlpha(alpha);
    canvas.draw*(..., paint);
    If you are writing a custom view and the drawing commands don’t overlap, you can simply
    set the alpha on the paint.

    View full-size slide

  60. // Or use a layer
    view.setLayerType(View.LAYER_TYPE_HARDWARE, null);
    // Transient layer
    view.animate().alpha(0).withLayer();
    By far the easiest thing to do is to use a hardware layer. The extra buffer copy will happen
    only once (or every time the view changes) instead of on every frame.

    View full-size slide

  61. // API level 16+
    @Override
    public boolean hasOverlappingRendering() {
    // Don't lie to us!
    return false;
    }
    If you have a custom view and you know your content does not overlap, override this method
    and return false. This will trigger an automatic optimization in the hardware rendering pipeline
    that will bypass the use of a separate buffer.

    View full-size slide

  62. 640 dp
    400 dp
    Canvas
    Canvas is the API used to draw on Android
    You should however be very careful with its dimensions

    View full-size slide

  63. @Override
    protected void onDraw(Canvas canvas) {
    // Get the dimensions of the Canvas
    int w = canvas.getWidth();
    int h = canvas.getHeight();
    canvas.drawRect(0, 0, w, h, mPaint);
    }
    Here is code I have seen in many applications
    The result of these calls might surprise you

    View full-size slide

  64. 1280 px
    800 px
    View
    300 px
    600 px
    Canvas is the API used to draw on Android
    You should however be very careful with its dimensions
    What will the previous code sample yield as a result?

    View full-size slide

  65. In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap

    View full-size slide

  66. With hardware rendering
    600x300 px (size of the View)
    In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap

    View full-size slide

  67. With hardware rendering
    With software rendering
    600x300 px
    1280x800 px
    (size of the View)
    (size of the window)
    In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap

    View full-size slide


  68. Clipping
    Let’s talk about clipping

    View full-size slide

  69. @Override
    protected void onDraw(Canvas canvas) {
    // Keep the jellybeans
    canvas.clipRect(l, t, r, b);
    // Rotate the jar
    canvas.rotate(-30.0f, pX, pY);
    // Draw the jar
    canvas.drawBitmap(mJellyBeans, x, y, null);
    }
    You should be very careful when applying rotations with clipping
    Here is a simple example of code that clips first, then rotates before drawing a bitmap

    View full-size slide

  70. The previous code will produce something like this. Note that the clip rect remains screen aligned.
    This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

    View full-size slide

  71. 1. Clip
    The previous code will produce something like this. Note that the clip rect remains screen aligned.
    This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

    View full-size slide

  72. 2. Rotate
    The previous code will produce something like this. Note that the clip rect remains screen aligned.
    This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

    View full-size slide

  73. 3. Draw
    The previous code will produce something like this. Note that the clip rect remains screen aligned.
    This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

    View full-size slide

  74. The previous code will produce something like this. Note that the clip rect remains screen aligned.
    This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

    View full-size slide

  75. @Override
    protected void onDraw(Canvas canvas) {
    // Rotate the jar
    canvas.rotate(-30.0f, pX, pY);
    // Keep the jellybeans
    canvas.clipRect(l, t, r, b);
    // Draw the jar
    canvas.drawBitmap(mJellyBeans, x, y, null);
    }
    What if we change the order of operations and rotate then clip?

    View full-size slide

  76. This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore.
    We need to take a more expensive code path that relies on the stencil buffer.

    View full-size slide

  77. 1. Rotate
    This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore.
    We need to take a more expensive code path that relies on the stencil buffer.

    View full-size slide

  78. 2. Clip
    This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore.
    We need to take a more expensive code path that relies on the stencil buffer.

    View full-size slide

  79. 3. Draw
    This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore.
    We need to take a more expensive code path that relies on the stencil buffer.

    View full-size slide

  80. This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore.
    We need to take a more expensive code path that relies on the stencil buffer.

    View full-size slide

  81. Stencil buffer
    The stencil buffer is a mask in the GPU. To use the stencil buffer we must clear it and draw into it.
    This means that transformed clipping operations cause extra drawing commands to be executed
    This also means we’re using more fillrate

    View full-size slide

  82. Stencil buffer
    The stencil buffer is a mask in the GPU. To use the stencil buffer we must clear it and draw into it.
    This means that transformed clipping operations cause extra drawing commands to be executed
    This also means we’re using more fillrate

    View full-size slide

  83. 640 px
    400 px
    View
    Imagine a view with the following dimensions
    Let’s now invalidate that View partially

    View full-size slide

  84. Invalidate
    640 px
    400 px
    (170,125)
    (470,275)
    Imagine a view with the following dimensions
    Let’s now invalidate that View partially

    View full-size slide

  85. @Override
    protected void onDraw(Canvas canvas) {
    // Query the current clip
    Rect clip = canvas.getClipBounds();
    // ???
    Log.d("I/O", "clip = " + clip);
    }
    What would you expect the getClipBounds() API to return?
    (Note: use the variant that takes a REct to avoid allocations)

    View full-size slide

  86. In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the dirty rect
    Discussion about multiple invalidates (dirty unions)

    View full-size slide

  87. With hardware rendering
    0, 0, 640, 400
    (bounds of the View)
    In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the dirty rect
    Discussion about multiple invalidates (dirty unions)

    View full-size slide

  88. With hardware rendering
    With software rendering
    0, 0, 640, 400
    170,125, 470, 275
    (bounds of the View)
    (bounds of the dirty rect)
    In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the dirty rect
    Discussion about multiple invalidates (dirty unions)

    View full-size slide

  89. Reordering barriers
    Reordering and merging can optimize your applications but specific Canvas calls can prevent reordering.
    We refer to these calls as “barriers”. Reordering can only happen before or after but cannot work across
    barriers.

    View full-size slide

  90. Non-rectangular clips
    The first type of barrier is caused by non-rectangular clips. Be careful as non-rect clips can be introduced by
    rotations. For instance, rotating a View without setting a layer on that view first will cause non-rect clipping to occur.

    View full-size slide

  91. saveLayer()
    The second type of barrier is created by calling saveLayer() (or its variant saveLayerAlpha().) Be careful as
    saveLayerAlpha() can be triggered on your behalf by calling View.setAlpha() as discussed earlier.

    View full-size slide

  92. More info
    Parleys.com
    For Butter or Worse
    Google I/O 2012
    Various Android GUI & performance talks
    Accelerated Android Rendering
    Google I/O 2011

    View full-size slide

  93. More info
    Romain’s Tips & Tricks
    www.curious-creature.org
    Chet’s Tips & Tricks
    goo.gl/y9JZr
    Android Performance Case Study
    graphics-geek.blogspot.com

    View full-size slide

  94. Q&A
    google.com/+ChetHaase
    google.com/+RomainGuy
    @chethaase
    @romainguy

    View full-size slide