Share
Embed
Android Graphics Performance
Tips, tricks, tools, and techniques used by engineers from the Android UI Graphics team for getting the best performance and smoothest UI from Android applications.
Video available at:
https://developers.google.com/events/io/sessions/325418001
Romain Guy
May 16, 2013
Transcript
96 slides
-
Welcome/Android Graphics & Performance Chet Haase & Romain Guy, Android
Framework engineers (Graphics & animations) -
& Graphics Performance Welcome/Android Graphics & Performance Chet Haase &
Romain Guy, Android Framework engineers (Graphics & animations) -
& Chet Haase Romain Guy Welcome/Android Graphics & Performance Chet
Haase & Romain Guy, Android Framework engineers (Graphics & animations) -
Reordering & merging A future update of Android introduces a
major change in the hardware accelerated 2D rendering pipeline: re-ordering and merging of drawing commands -
Save Cancel Maximize compatibility Include metadata This allows for more
efficient rendering without changing anything in your application. Here is an example of a simple Android UI -
Until now Android would always render drawing commands in order.
Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls. -
Include metadata Maximize compatibility Cancel Save Order of drawing commands
Until now Android would always render drawing commands in order. Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls. -
Include metadata Maximize compatibility Cancel Save 1. Re-ordering Until now
Android would always render drawing commands in order. Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls. -
Include metadata Maximize compatibility Cancel Save 2. Merging Until now
Android would always render drawing commands in order. Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls. -
Let’s demo re-ordering and merging on a real device, using
Google+ as an example -
Multi-threading In a future update of Android, the hardware accelerated
2D rendering pipeline makes better use of multi-core systems by using multiple threads to perform certain tasks. -
The blue bar show the UI thread performing normal rendering.
However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow. -
Drawing The blue bar show the UI thread performing normal
rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow. -
Drawing Shadows Shadows Shadows Shadows The blue bar show the
UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow. -
Drawing Paths Paths Shadows Shadows Shadows Shadows The blue bar
show the UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow. -
Non-rectangular clipping A future update of Android adds support for
hardware accelerated non-rectangular clipping This includes clipping with paths (circles, curves, rounded rects, etc.) and transformed rects (3D rotations for instance) -
@Override protected void onDraw(Canvas canvas) { // Clip with a
shape Path clip = getPath(); canvas.clipPath(clip); // Draw the content for (int i = 0; i < mLines,length; i++) { TextLine line = mLines[i]; canvas.drawText(line.text, line.x, line.y, mPaint); } } Non-rectangular clips can also be achieved by specifying the Region.Op parameter of the various clip*() methods (you can add, xor, subtract, etc.). Non-rect clipping can be triggered by rotations. -
Overdraw In developer options you can now turn on a
“Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc. -
Overdraw In developer options you can now turn on a
“Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc. -
Overdraw In developer options you can now turn on a
“Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc. -
Overdraw Blue Green Red Deep Red 1x 2x 3x 4x
In developer options you can now turn on a “Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc. -
0 3 6 9 12 15 Time in ms Frames
Update display lists Process display lists Swap buffers In Android 4.1 we introduced a new rendering profiling tool in developer settings This tool was useful to create performance graphs such as this one It was however difficult to use since it required command line tools and a spreadsheet -
In a future update of Android profiling is a lot
easier. You can now see the profiling graph directly on screen. The old method is also available. The green line indicates the 16ms threshold (60fps.) -
In a future update of Android profiling is a lot
easier. You can now see the profiling graph directly on screen. The old method is also available. The green line indicates the 16ms threshold (60fps.) -
performTraversals draw getDL drawDisplayList systrace flush drawing commands systrace was
introduced in Android 4.1 and we’ve added several improvements in 4.2 and future updates of Android. -
android:sdk $ cd platform-tools/ android:platform-tools $ 㾑 ./systrace.py gfx view
freq sched In a future update of Android systrace is also easier to run, no need to change developer options All you need is to run the script from the command line In this example we’re tracing graphics (gfx), UI toolkit (view), CPU frequencies (freq) and the kernel scheduler (sched) -
And since systrace is easier to use, we added more
information. If you enable OpenGL tracing in developer options you will be able to see each individual OpenGL call in systrace (use the gfx tag when running systrace.) You can also enable glGetError() checks after each GL call. -
import android.os.Trace; @Override public View getView(int pos, View view, ViewGroup
parent) { Trace.beginSection("getView"); if (view == null) { view = createView(); } // Trace time spent binding data Trace.beginSection("bind"); bindView(pos, view); Trace.endSection(); Trace.endSection(); return view; } We’re introducing a new API, android.os.Trace, to let you add your own information to systrace. Systrace events are lightweight and add almost no overhead. Using it is easy: simply wrap the section of code to trace with beginSection/endSection. You can nest sections. -
android:sdk $ cd platform-tools/ android:platform-tools $ 㾑 ./systrace.py -a com.example.myapp
You won’t see your events by default, you must specify the package name of your app using the -a option when invoking systrace. You can specify several packages and combine it with the built-in tags such as gfx and view. -
Trilinear filtering Trilinear filtering was introduced in Android 4.2 and
can be used to improve the visual quality of your bitmaps if you scale them down to less than 50% of their original size -
Here is an example with trilinear filtering turned off (left)
and on (right) You can see the trilinear filtering does a much better job at smoothing the image (The images have been scaled up many times) -
Off On Here is an example with trilinear filtering turned
off (left) and on (right) You can see the trilinear filtering does a much better job at smoothing the image (The images have been scaled up many times) -
private void loadData() { // Load bitmap Bitmap b =
getBitmap(); // Enable trilinear filtering b.setHasMipMap(true); } Here is how you can enable trilinear filtering on a Bitmap You can also enable trilinear filtering on a BitmapDrawable if you prefer -
<bitmap android:mipMap="true" android:src="@drawable/my_drawable" /> You can of course enable trilinear
filtering in your XML drawable definition -
Canvas layers Let’s talk a bit about Canvas layers (as
opposed to View.setLayerType.) There are two types of layers and using them can create interesting performance issues -
@Override protected void onDraw(Canvas canvas) { // Create a clipped
layer canvas.save(); canvas.saveLayer(x, y, width, height, Canvas.CLIP_TO_LAYER_SAVE_FLAG); // Draw stuff canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null); canvas.restore(); } The first type of layers is called “clipped layer”. Such layers are created by passing the flag Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer(). -
saveLayer() Here is how clipped layers work. After calling saveLayer(),
only the drawing commands that intersect with that layer will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an offscreen render target (bitmap in software, FBO+texture in hardware.) -
Here is how clipped layers work. After calling saveLayer(), only
the drawing commands that intersect with that layer will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an offscreen render target (bitmap in software, FBO+texture in hardware.) -
Here is how clipped layers work. After calling saveLayer(), only
the drawing commands that intersect with that layer will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an offscreen render target (bitmap in software, FBO+texture in hardware.) -
@Override protected void onDraw(Canvas canvas) { // Create an unclipped
layer canvas.save(); canvas.saveLayer(x, y, width, height, 0); // Draw stuff canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null); canvas.restore(); } The second type of layers is called “unclipped layer”. Such layers are created by NOT passing the flag Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer(). -
saveLayer() With unclipped layers, a drawing command will be drawn
in any layer it intersects, including the original canvas. In this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N times (N=number of active layers.) -
With unclipped layers, a drawing command will be drawn in
any layer it intersects, including the original canvas. In this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N times (N=number of active layers.) -
With unclipped layers, a drawing command will be drawn in
any layer it intersects, including the original canvas. In this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N times (N=number of active layers.) -
Using alpha with care Android offers many easy ways to
apply alpha on a view. Using alpha can however have a measurable impact the performance of your application. -
Each line shows a different way to set alpha on
a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer. -
view.setAlpha(0.5f); View.ALPHA.set(view, 0.5f); ObjectAnimation.ofFloat(view, "alpha", 0.5f) view.animate().alpha(0.5f); view.setAnimation(new AlphaAnimation(1.0f, 0.5f));
Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer. -
view.setAlpha(0.5f); View.ALPHA.set(view, 0.5f); ObjectAnimation.ofFloat(view, "alpha", 0.5f) view.animate().alpha(0.5f); view.setAnimation(new AlphaAnimation(1.0f, 0.5f));
Canvas.saveLayerAlpha(l, t, r, b, 127, Canvas.CLIP_TO_LAYER_SAVE_FLAG); == Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer. -
Why is it important to use a separate buffer to
apply the alpha? Here is an example. Let’s imagine a View containing 3 photos (bitmaps.) If we animate the opacity to 50% without using a separate buffer the result would look like this one. -
Why is it important to use a separate buffer to
apply the alpha? Here is an example. Let’s imagine a View containing 3 photos (bitmaps.) If we animate the opacity to 50% without using a separate buffer the result would look like this one. -
Instead of applying the alpha separately, we can use a
separate buffer in which we draw the 3 photos and then we apply alpha to that buffer. The result can be seen above and looks a lot better. -
Instead of applying the alpha separately, we can use a
separate buffer in which we draw the 3 photos and then we apply alpha to that buffer. The result can be seen above and looks a lot better. -
lternatives Let’s examine various solutions you can use to maximize
performance when applying alpha on a View. -
// Not this textView.setAlpha(alpha); // But this int newTextColor =
(int) (0xFF * alpha) << 24 | baseTextColor & 0xFFFFFF; textView.setTextColor(newTextColor); If you are setting alpha on a TextView and you don’t have a background you can simply bake the alpha in the text color instead. This also applies to background colors, drawing primitives, etc. -
// Not this imageView.setAlpha(alpha); // But this imageView.setImageAlpha((int) (alpha *
255)); Similarly with ImageView, you can apply the alpha directly to the image instead of the View itself. -
// Not this customView.setAlpha(alpha); // But this int alpha =
(int) (255 * slider.getProgress() / 100.0f); paint.setAlpha(alpha); canvas.draw*(..., paint); If you are writing a custom view and the drawing commands don’t overlap, you can simply set the alpha on the paint. -
// Or use a layer view.setLayerType(View.LAYER_TYPE_HARDWARE, null); // Transient layer
view.animate().alpha(0).withLayer(); By far the easiest thing to do is to use a hardware layer. The extra buffer copy will happen only once (or every time the view changes) instead of on every frame. -
// API level 16+ @Override public boolean hasOverlappingRendering() { //
Don't lie to us! return false; } If you have a custom view and you know your content does not overlap, override this method and return false. This will trigger an automatic optimization in the hardware rendering pipeline that will bypass the use of a separate buffer. -
640 dp 400 dp Canvas Canvas is the API used
to draw on Android You should however be very careful with its dimensions -
@Override protected void onDraw(Canvas canvas) { // Get the dimensions
of the Canvas int w = canvas.getWidth(); int h = canvas.getHeight(); canvas.drawRect(0, 0, w, h, mPaint); } Here is code I have seen in many applications The result of these calls might surprise you -
1280 px 800 px View 300 px 600 px Canvas
is the API used to draw on Android You should however be very careful with its dimensions What will the previous code sample yield as a result? -
In hardware, the API returns the dimensions of the View
In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap -
With hardware rendering 600x300 px (size of the View) In
hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap -
With hardware rendering With software rendering 600x300 px 1280x800 px
(size of the View) (size of the window) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap -
@Override protected void onDraw(Canvas canvas) { // Keep the jellybeans
canvas.clipRect(l, t, r, b); // Rotate the jar canvas.rotate(-30.0f, pX, pY); // Draw the jar canvas.drawBitmap(mJellyBeans, x, y, null); } You should be very careful when applying rotations with clipping Here is a simple example of code that clips first, then rotates before drawing a bitmap -
The previous code will produce something like this. Note that
the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation. -
1. Clip The previous code will produce something like this.
Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation. -
2. Rotate The previous code will produce something like this.
Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation. -
3. Draw The previous code will produce something like this.
Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation. -
The previous code will produce something like this. Note that
the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation. -
@Override protected void onDraw(Canvas canvas) { // Rotate the jar
canvas.rotate(-30.0f, pX, pY); // Keep the jellybeans canvas.clipRect(l, t, r, b); // Draw the jar canvas.drawBitmap(mJellyBeans, x, y, null); } What if we change the order of operations and rotate then clip? -
This is a much more expensive operation. As you can
see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer. -
1. Rotate This is a much more expensive operation. As
you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer. -
2. Clip This is a much more expensive operation. As
you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer. -
3. Draw This is a much more expensive operation. As
you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer. -
This is a much more expensive operation. As you can
see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer. -
Stencil buffer The stencil buffer is a mask in the
GPU. To use the stencil buffer we must clear it and draw into it. This means that transformed clipping operations cause extra drawing commands to be executed This also means we’re using more fillrate -
Stencil buffer The stencil buffer is a mask in the
GPU. To use the stencil buffer we must clear it and draw into it. This means that transformed clipping operations cause extra drawing commands to be executed This also means we’re using more fillrate -
640 px 400 px View Imagine a view with the
following dimensions Let’s now invalidate that View partially -
Invalidate 640 px 400 px (170,125) (470,275) Imagine a view
with the following dimensions Let’s now invalidate that View partially -
@Override protected void onDraw(Canvas canvas) { // Query the current
clip Rect clip = canvas.getClipBounds(); // ??? Log.d("I/O", "clip = " + clip); } What would you expect the getClipBounds() API to return? (Note: use the variant that takes a REct to avoid allocations) -
In hardware, the API returns the dimensions of the View
In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions) -
With hardware rendering 0, 0, 640, 400 (bounds of the
View) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions) -
With hardware rendering With software rendering 0, 0, 640, 400
170,125, 470, 275 (bounds of the View) (bounds of the dirty rect) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions) -
Reordering barriers Reordering and merging can optimize your applications but
specific Canvas calls can prevent reordering. We refer to these calls as “barriers”. Reordering can only happen before or after but cannot work across barriers. -
Non-rectangular clips The first type of barrier is caused by
non-rectangular clips. Be careful as non-rect clips can be introduced by rotations. For instance, rotating a View without setting a layer on that view first will cause non-rect clipping to occur. -
saveLayer() The second type of barrier is created by calling
saveLayer() (or its variant saveLayerAlpha().) Be careful as saveLayerAlpha() can be triggered on your behalf by calling View.setAlpha() as discussed earlier. -
More info Parleys.com For Butter or Worse Google I/O 2012
Various Android GUI & performance talks Accelerated Android Rendering Google I/O 2011 -
More info Romain’s Tips & Tricks www.curious-creature.org Chet’s Tips &
Tricks goo.gl/y9JZr Android Performance Case Study graphics-geek.blogspot.com