Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Android Graphics Performance

Android Graphics Performance

Tips, tricks, tools, and techniques used by engineers from the Android UI Graphics team for getting the best performance and smoothest UI from Android applications.

Video available at:
https://developers.google.com/events/io/sessions/325418001

Romain Guy

May 16, 2013
Tweet

More Decks by Romain Guy

Other Decks in Programming

Transcript

  1. View Slide

  2. Welcome/Android Graphics & Performance
    Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)

    View Slide

  3. &
    Graphics Performance
    Welcome/Android Graphics & Performance
    Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)

    View Slide

  4. &
    Chet Haase Romain Guy
    Welcome/Android Graphics & Performance
    Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)

    View Slide

  5. Architecture
    1

    View Slide

  6. Reordering & merging
    A future update of Android introduces a major change in the hardware accelerated 2D rendering pipeline:
    re-ordering and merging of drawing commands

    View Slide

  7. Save Cancel
    Maximize compatibility
    Include metadata
    This allows for more efficient rendering without changing anything in your application. Here is an example of a
    simple Android UI

    View Slide

  8. Until now Android would always render drawing commands in order. Android now first re-orders the commands to
    find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to
    reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of-
    order draw calls.

    View Slide

  9. Include metadata Maximize compatibility Cancel
    Save
    Order of drawing commands
    Until now Android would always render drawing commands in order. Android now first re-orders the commands to
    find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to
    reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of-
    order draw calls.

    View Slide

  10. Include metadata Maximize compatibility Cancel
    Save
    1. Re-ordering
    Until now Android would always render drawing commands in order. Android now first re-orders the commands to
    find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to
    reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of-
    order draw calls.

    View Slide

  11. Include metadata
    Maximize compatibility
    Cancel
    Save
    2. Merging
    Until now Android would always render drawing commands in order. Android now first re-orders the commands to
    find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to
    reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of-
    order draw calls.

    View Slide

  12. Let’s demo re-ordering and merging on a real device, using Google+ as an example

    View Slide

  13. Multi-threading
    In a future update of Android, the hardware accelerated 2D rendering pipeline makes better
    use of multi-core systems by using multiple threads to perform certain tasks.

    View Slide

  14. The blue bar show the UI thread performing normal rendering. However the renderer now uses
    Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.)
    Paths are also now generated on background threads, as shown in yellow.

    View Slide

  15. Drawing
    The blue bar show the UI thread performing normal rendering. However the renderer now uses
    Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.)
    Paths are also now generated on background threads, as shown in yellow.

    View Slide

  16. Drawing
    Shadows
    Shadows
    Shadows
    Shadows
    The blue bar show the UI thread performing normal rendering. However the renderer now uses
    Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.)
    Paths are also now generated on background threads, as shown in yellow.

    View Slide

  17. Drawing
    Paths
    Paths
    Shadows
    Shadows
    Shadows
    Shadows
    The blue bar show the UI thread performing normal rendering. However the renderer now uses
    Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.)
    Paths are also now generated on background threads, as shown in yellow.

    View Slide

  18. Non-rectangular clipping
    A future update of Android adds support for hardware accelerated non-rectangular clipping
    This includes clipping with paths (circles, curves, rounded rects, etc.) and transformed rects
    (3D rotations for instance)

    View Slide

  19. @Override
    protected void onDraw(Canvas canvas) {
    // Clip with a shape
    Path clip = getPath();
    canvas.clipPath(clip);
    // Draw the content
    for (int i = 0; i < mLines,length; i++) {
    TextLine line = mLines[i];
    canvas.drawText(line.text,
    line.x, line.y, mPaint);
    }
    }
    Non-rectangular clips can also be achieved by specifying the Region.Op parameter
    of the various clip*() methods (you can add, xor, subtract, etc.). Non-rect clipping can be triggered by rotations.

    View Slide

  20. Developer Tools
    2

    View Slide

  21. Overdraw
    In developer options you can now turn on a “Show GPU overdraw” feature
    Apps will be highlighted with various colors. Each color represents the amount of overdraw
    for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.

    View Slide

  22. Overdraw
    In developer options you can now turn on a “Show GPU overdraw” feature
    Apps will be highlighted with various colors. Each color represents the amount of overdraw
    for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.

    View Slide

  23. Overdraw
    In developer options you can now turn on a “Show GPU overdraw” feature
    Apps will be highlighted with various colors. Each color represents the amount of overdraw
    for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.

    View Slide

  24. Overdraw
    Blue
    Green
    Red
    Deep Red
    1x
    2x
    3x
    4x
    In developer options you can now turn on a “Show GPU overdraw” feature
    Apps will be highlighted with various colors. Each color represents the amount of overdraw
    for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.

    View Slide

  25. 0
    3
    6
    9
    12
    15
    Time in ms
    Frames
    Update display lists Process display lists Swap buffers
    In Android 4.1 we introduced a new rendering profiling tool in developer settings
    This tool was useful to create performance graphs such as this one
    It was however difficult to use since it required command line tools and a spreadsheet

    View Slide

  26. In a future update of Android profiling is a lot easier. You can now see the profiling
    graph directly on screen. The old method is also available.
    The green line indicates the 16ms threshold (60fps.)

    View Slide

  27. In a future update of Android profiling is a lot easier. You can now see the profiling
    graph directly on screen. The old method is also available.
    The green line indicates the 16ms threshold (60fps.)

    View Slide

  28. performTraversals
    draw
    getDL drawDisplayList
    systrace
    flush drawing commands
    systrace was introduced in Android 4.1 and we’ve added several improvements in 4.2
    and future updates of Android.

    View Slide

  29. android:sdk $ cd platform-tools/
    android:platform-tools $ 㾑
    ./systrace.py gfx view freq sched
    In a future update of Android systrace is also easier to run, no need to change developer options
    All you need is to run the script from the command line
    In this example we’re tracing graphics (gfx), UI toolkit (view), CPU frequencies (freq)
    and the kernel scheduler (sched)

    View Slide

  30. And since systrace is easier to use, we added more information. If you enable OpenGL tracing
    in developer options you will be able to see each individual OpenGL call in systrace (use the gfx tag when running
    systrace.) You can also enable glGetError() checks after each GL call.

    View Slide

  31. import android.os.Trace;
    @Override
    public View getView(int pos, View view, ViewGroup parent) {
    Trace.beginSection("getView");
    if (view == null) {
    view = createView();
    }
    // Trace time spent binding data
    Trace.beginSection("bind");
    bindView(pos, view);
    Trace.endSection();
    Trace.endSection();
    return view;
    }
    We’re introducing a new API, android.os.Trace, to let you add your own information to
    systrace. Systrace events are lightweight and add almost no overhead. Using it is easy:
    simply wrap the section of code to trace with beginSection/endSection. You can nest sections.

    View Slide

  32. android:sdk $ cd platform-tools/
    android:platform-tools $ 㾑
    ./systrace.py -a com.example.myapp
    You won’t see your events by default, you must specify the package name of your app
    using the -a option when invoking systrace. You can specify several packages and combine
    it with the built-in tags such as gfx and view.

    View Slide

  33. Tips & Tricks
    3

    View Slide

  34. Overdraw demo

    View Slide

  35. Trilinear filtering
    Trilinear filtering was introduced in Android 4.2 and can be used to improve the visual
    quality of your bitmaps if you scale them down to less than 50% of their original size

    View Slide

  36. Here is an example with trilinear filtering turned off (left) and on (right)
    You can see the trilinear filtering does a much better job at smoothing the image
    (The images have been scaled up many times)

    View Slide

  37. Off On
    Here is an example with trilinear filtering turned off (left) and on (right)
    You can see the trilinear filtering does a much better job at smoothing the image
    (The images have been scaled up many times)

    View Slide

  38. private void loadData() {
    // Load bitmap
    Bitmap b = getBitmap();
    // Enable trilinear filtering
    b.setHasMipMap(true);
    }
    Here is how you can enable trilinear filtering on a Bitmap
    You can also enable trilinear filtering on a BitmapDrawable if you prefer

    View Slide

  39. android:mipMap="true"
    android:src="@drawable/my_drawable" />
    You can of course enable trilinear filtering in your XML drawable definition

    View Slide

  40. Canvas layers
    Let’s talk a bit about Canvas layers (as opposed to View.setLayerType.)
    There are two types of layers and using them can create interesting performance issues

    View Slide

  41. @Override
    protected void onDraw(Canvas canvas) {
    // Create a clipped layer
    canvas.save();
    canvas.saveLayer(x, y, width, height,
    Canvas.CLIP_TO_LAYER_SAVE_FLAG);
    // Draw stuff
    canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null);
    canvas.restore();
    }
    The first type of layers is called “clipped layer”. Such layers are created by passing the flag
    Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer().

    View Slide

  42. saveLayer()
    Here is how clipped layers work. After calling saveLayer(), only the drawing commands that intersect with that layer
    will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an
    offscreen render target (bitmap in software, FBO+texture in hardware.)

    View Slide

  43. Here is how clipped layers work. After calling saveLayer(), only the drawing commands that intersect with that layer
    will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an
    offscreen render target (bitmap in software, FBO+texture in hardware.)

    View Slide

  44. Here is how clipped layers work. After calling saveLayer(), only the drawing commands that intersect with that layer
    will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an
    offscreen render target (bitmap in software, FBO+texture in hardware.)

    View Slide

  45. @Override
    protected void onDraw(Canvas canvas) {
    // Create an unclipped layer
    canvas.save();
    canvas.saveLayer(x, y, width, height, 0);
    // Draw stuff
    canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null);
    canvas.restore();
    }
    The second type of layers is called “unclipped layer”. Such layers are created by NOT passing the flag
    Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer().

    View Slide

  46. saveLayer()
    With unclipped layers, a drawing command will be drawn in any layer it intersects, including the original canvas. In
    this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This
    is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N
    times (N=number of active layers.)

    View Slide

  47. With unclipped layers, a drawing command will be drawn in any layer it intersects, including the original canvas. In
    this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This
    is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N
    times (N=number of active layers.)

    View Slide

  48. With unclipped layers, a drawing command will be drawn in any layer it intersects, including the original canvas. In
    this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This
    is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N
    times (N=number of active layers.)

    View Slide

  49. Using alpha with care
    Android offers many easy ways to apply alpha on a view. Using alpha can however have a measurable impact
    the performance of your application.

    View Slide

  50. Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is
    they all trigger the same side effect: the creation of an expensive offscreen buffer.

    View Slide

  51. view.setAlpha(0.5f);
    View.ALPHA.set(view, 0.5f);
    ObjectAnimation.ofFloat(view, "alpha", 0.5f)
    view.animate().alpha(0.5f);
    view.setAnimation(new AlphaAnimation(1.0f, 0.5f));
    Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is
    they all trigger the same side effect: the creation of an expensive offscreen buffer.

    View Slide

  52. view.setAlpha(0.5f);
    View.ALPHA.set(view, 0.5f);
    ObjectAnimation.ofFloat(view, "alpha", 0.5f)
    view.animate().alpha(0.5f);
    view.setAnimation(new AlphaAnimation(1.0f, 0.5f));
    Canvas.saveLayerAlpha(l, t, r, b, 127,
    Canvas.CLIP_TO_LAYER_SAVE_FLAG);
    ==
    Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is
    they all trigger the same side effect: the creation of an expensive offscreen buffer.

    View Slide

  53. Why is it important to use a separate buffer to apply the alpha? Here is an example. Let’s imagine a View containing 3
    photos (bitmaps.) If we animate the opacity to 50% without using a separate buffer the result would look like this one.

    View Slide

  54. Why is it important to use a separate buffer to apply the alpha? Here is an example. Let’s imagine a View containing 3
    photos (bitmaps.) If we animate the opacity to 50% without using a separate buffer the result would look like this one.

    View Slide

  55. Instead of applying the alpha separately, we can use a separate buffer in which we draw the 3 photos and then we
    apply alpha to that buffer. The result can be seen above and looks a lot better.

    View Slide

  56. Instead of applying the alpha separately, we can use a separate buffer in which we draw the 3 photos and then we
    apply alpha to that buffer. The result can be seen above and looks a lot better.

    View Slide

  57. lternatives
    Let’s examine various solutions you can use to maximize performance when applying alpha on a View.

    View Slide

  58. // Not this
    textView.setAlpha(alpha);
    // But this
    int newTextColor = (int) (0xFF * alpha) << 24 |
    baseTextColor & 0xFFFFFF;
    textView.setTextColor(newTextColor);
    If you are setting alpha on a TextView and you don’t have a background you can simply
    bake the alpha in the text color instead. This also applies to background colors, drawing
    primitives, etc.

    View Slide

  59. // Not this
    imageView.setAlpha(alpha);
    // But this
    imageView.setImageAlpha((int) (alpha * 255));
    Similarly with ImageView, you can apply the alpha directly to the image instead of the
    View itself.

    View Slide

  60. // Not this
    customView.setAlpha(alpha);
    // But this
    int alpha = (int)
    (255 * slider.getProgress() / 100.0f);
    paint.setAlpha(alpha);
    canvas.draw*(..., paint);
    If you are writing a custom view and the drawing commands don’t overlap, you can simply
    set the alpha on the paint.

    View Slide

  61. // Or use a layer
    view.setLayerType(View.LAYER_TYPE_HARDWARE, null);
    // Transient layer
    view.animate().alpha(0).withLayer();
    By far the easiest thing to do is to use a hardware layer. The extra buffer copy will happen
    only once (or every time the view changes) instead of on every frame.

    View Slide

  62. // API level 16+
    @Override
    public boolean hasOverlappingRendering() {
    // Don't lie to us!
    return false;
    }
    If you have a custom view and you know your content does not overlap, override this method
    and return false. This will trigger an automatic optimization in the hardware rendering pipeline
    that will bypass the use of a separate buffer.

    View Slide

  63. 640 dp
    400 dp
    Canvas
    Canvas is the API used to draw on Android
    You should however be very careful with its dimensions

    View Slide

  64. @Override
    protected void onDraw(Canvas canvas) {
    // Get the dimensions of the Canvas
    int w = canvas.getWidth();
    int h = canvas.getHeight();
    canvas.drawRect(0, 0, w, h, mPaint);
    }
    Here is code I have seen in many applications
    The result of these calls might surprise you

    View Slide

  65. 1280 px
    800 px
    View
    300 px
    600 px
    Canvas is the API used to draw on Android
    You should however be very careful with its dimensions
    What will the previous code sample yield as a result?

    View Slide

  66. In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap

    View Slide

  67. With hardware rendering
    600x300 px (size of the View)
    In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap

    View Slide

  68. With hardware rendering
    With software rendering
    600x300 px
    1280x800 px
    (size of the View)
    (size of the window)
    In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap

    View Slide


  69. Clipping
    Let’s talk about clipping

    View Slide

  70. @Override
    protected void onDraw(Canvas canvas) {
    // Keep the jellybeans
    canvas.clipRect(l, t, r, b);
    // Rotate the jar
    canvas.rotate(-30.0f, pX, pY);
    // Draw the jar
    canvas.drawBitmap(mJellyBeans, x, y, null);
    }
    You should be very careful when applying rotations with clipping
    Here is a simple example of code that clips first, then rotates before drawing a bitmap

    View Slide

  71. The previous code will produce something like this. Note that the clip rect remains screen aligned.
    This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

    View Slide

  72. 1. Clip
    The previous code will produce something like this. Note that the clip rect remains screen aligned.
    This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

    View Slide

  73. 2. Rotate
    The previous code will produce something like this. Note that the clip rect remains screen aligned.
    This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

    View Slide

  74. 3. Draw
    The previous code will produce something like this. Note that the clip rect remains screen aligned.
    This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

    View Slide

  75. The previous code will produce something like this. Note that the clip rect remains screen aligned.
    This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

    View Slide

  76. @Override
    protected void onDraw(Canvas canvas) {
    // Rotate the jar
    canvas.rotate(-30.0f, pX, pY);
    // Keep the jellybeans
    canvas.clipRect(l, t, r, b);
    // Draw the jar
    canvas.drawBitmap(mJellyBeans, x, y, null);
    }
    What if we change the order of operations and rotate then clip?

    View Slide

  77. This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore.
    We need to take a more expensive code path that relies on the stencil buffer.

    View Slide

  78. 1. Rotate
    This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore.
    We need to take a more expensive code path that relies on the stencil buffer.

    View Slide

  79. 2. Clip
    This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore.
    We need to take a more expensive code path that relies on the stencil buffer.

    View Slide

  80. 3. Draw
    This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore.
    We need to take a more expensive code path that relies on the stencil buffer.

    View Slide

  81. This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore.
    We need to take a more expensive code path that relies on the stencil buffer.

    View Slide

  82. Stencil buffer
    The stencil buffer is a mask in the GPU. To use the stencil buffer we must clear it and draw into it.
    This means that transformed clipping operations cause extra drawing commands to be executed
    This also means we’re using more fillrate

    View Slide

  83. Stencil buffer
    The stencil buffer is a mask in the GPU. To use the stencil buffer we must clear it and draw into it.
    This means that transformed clipping operations cause extra drawing commands to be executed
    This also means we’re using more fillrate

    View Slide

  84. 640 px
    400 px
    View
    Imagine a view with the following dimensions
    Let’s now invalidate that View partially

    View Slide

  85. Invalidate
    640 px
    400 px
    (170,125)
    (470,275)
    Imagine a view with the following dimensions
    Let’s now invalidate that View partially

    View Slide

  86. @Override
    protected void onDraw(Canvas canvas) {
    // Query the current clip
    Rect clip = canvas.getClipBounds();
    // ???
    Log.d("I/O", "clip = " + clip);
    }
    What would you expect the getClipBounds() API to return?
    (Note: use the variant that takes a REct to avoid allocations)

    View Slide

  87. In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the dirty rect
    Discussion about multiple invalidates (dirty unions)

    View Slide

  88. With hardware rendering
    0, 0, 640, 400
    (bounds of the View)
    In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the dirty rect
    Discussion about multiple invalidates (dirty unions)

    View Slide

  89. With hardware rendering
    With software rendering
    0, 0, 640, 400
    170,125, 470, 275
    (bounds of the View)
    (bounds of the dirty rect)
    In hardware, the API returns the dimensions of the View
    In software, the API returns the dimensions of the dirty rect
    Discussion about multiple invalidates (dirty unions)

    View Slide

  90. Reordering barriers
    Reordering and merging can optimize your applications but specific Canvas calls can prevent reordering.
    We refer to these calls as “barriers”. Reordering can only happen before or after but cannot work across
    barriers.

    View Slide

  91. Non-rectangular clips
    The first type of barrier is caused by non-rectangular clips. Be careful as non-rect clips can be introduced by
    rotations. For instance, rotating a View without setting a layer on that view first will cause non-rect clipping to occur.

    View Slide

  92. saveLayer()
    The second type of barrier is created by calling saveLayer() (or its variant saveLayerAlpha().) Be careful as
    saveLayerAlpha() can be triggered on your behalf by calling View.setAlpha() as discussed earlier.

    View Slide

  93. More info
    Parleys.com
    For Butter or Worse
    Google I/O 2012
    Various Android GUI & performance talks
    Accelerated Android Rendering
    Google I/O 2011

    View Slide

  94. More info
    Romain’s Tips & Tricks
    www.curious-creature.org
    Chet’s Tips & Tricks
    goo.gl/y9JZr
    Android Performance Case Study
    graphics-geek.blogspot.com

    View Slide

  95. Q&A
    google.com/+ChetHaase
    google.com/+RomainGuy
    @chethaase
    @romainguy

    View Slide

  96. Developers

    View Slide