Android Graphics Performance

Tips, tricks, tools, and techniques used by engineers from the Android UI Graphics team for getting the best performance and smoothest UI from Android applications.

Video available at:
https://developers.google.com/events/io/sessions/325418001

Transcript

2. Welcome/Android Graphics & Performance Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)
3. & Graphics Performance Welcome/Android Graphics & Performance Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)
4. & Chet Haase Romain Guy Welcome/Android Graphics & Performance Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)
5. Architecture 1
6. Reordering & merging A future update of Android introduces a major change in the hardware accelerated 2D rendering pipeline: re-ordering and merging of drawing commands
7. Save Cancel Maximize compatibility Include metadata This allows for more efficient rendering without changing anything in your application. Here is an example of a simple Android UI
8. Until now Android would always render drawing commands in order. Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.
9. Include metadata Maximize compatibility Cancel Save Order of drawing commands Until now Android would always render drawing commands in order. Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.
10. Include metadata Maximize compatibility Cancel Save 1. Re-ordering Until now Android would always render drawing commands in order. Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.
11. Include metadata Maximize compatibility Cancel Save 2. Merging Until now Android would always render drawing commands in order. Android now first re-orders the commands to find optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.
12. Let’s demo re-ordering and merging on a real device, using Google+ as an example
13. Multi-threading In a future update of Android, the hardware accelerated 2D rendering pipeline makes better use of multi-core systems by using multiple threads to perform certain tasks.
14. The blue bar show the UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.
15. Drawing The blue bar show the UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.
16. Drawing Shadows Shadows Shadows Shadows The blue bar show the UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.
17. Drawing Paths Paths Shadows Shadows Shadows Shadows The blue bar show the UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.
18. Non-rectangular clipping A future update of Android adds support for hardware accelerated non-rectangular clipping This includes clipping with paths (circles, curves, rounded rects, etc.) and transformed rects (3D rotations for instance)
19. @Override protected void onDraw(Canvas canvas) { // Clip with a shape Path clip = getPath(); canvas.clipPath(clip); // Draw the content for (int i = 0; i < mLines,length; i++) { TextLine line = mLines[i]; canvas.drawText(line.text, line.x, line.y, mPaint); } } Non-rectangular clips can also be achieved by specifying the Region.Op parameter of the various clip*() methods (you can add, xor, subtract, etc.). Non-rect clipping can be triggered by rotations.
20. Developer Tools 2
21. Overdraw In developer options you can now turn on a “Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.
22. Overdraw In developer options you can now turn on a “Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.
23. Overdraw In developer options you can now turn on a “Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.
24. Overdraw Blue Green Red Deep Red 1x 2x 3x 4x In developer options you can now turn on a “Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.
25. 0 3 6 9 12 15 Time in ms Frames Update display lists Process display lists Swap buffers In Android 4.1 we introduced a new rendering profiling tool in developer settings This tool was useful to create performance graphs such as this one It was however difficult to use since it required command line tools and a spreadsheet
26. In a future update of Android profiling is a lot easier. You can now see the profiling graph directly on screen. The old method is also available. The green line indicates the 16ms threshold (60fps.)
27. In a future update of Android profiling is a lot easier. You can now see the profiling graph directly on screen. The old method is also available. The green line indicates the 16ms threshold (60fps.)
28. performTraversals draw getDL drawDisplayList systrace flush drawing commands systrace was introduced in Android 4.1 and we’ve added several improvements in 4.2 and future updates of Android.
29. android:sdk $ cd platform-tools/ android:platform-tools $ 㾑 ./systrace.py gfx view freq sched In a future update of Android systrace is also easier to run, no need to change developer options All you need is to run the script from the command line In this example we’re tracing graphics (gfx), UI toolkit (view), CPU frequencies (freq) and the kernel scheduler (sched)
30. And since systrace is easier to use, we added more information. If you enable OpenGL tracing in developer options you will be able to see each individual OpenGL call in systrace (use the gfx tag when running systrace.) You can also enable glGetError() checks after each GL call.
31. import android.os.Trace; @Override public View getView(int pos, View view, ViewGroup parent) { Trace.beginSection("getView"); if (view == null) { view = createView(); } // Trace time spent binding data Trace.beginSection("bind"); bindView(pos, view); Trace.endSection(); Trace.endSection(); return view; } We’re introducing a new API, android.os.Trace, to let you add your own information to systrace. Systrace events are lightweight and add almost no overhead. Using it is easy: simply wrap the section of code to trace with beginSection/endSection. You can nest sections.
32. android:sdk $ cd platform-tools/ android:platform-tools $ 㾑 ./systrace.py -a com.example.myapp You won’t see your events by default, you must specify the package name of your app using the -a option when invoking systrace. You can specify several packages and combine it with the built-in tags such as gfx and view.
33. Tips & Tricks 3
34. Overdraw demo
35. Trilinear filtering Trilinear filtering was introduced in Android 4.2 and can be used to improve the visual quality of your bitmaps if you scale them down to less than 50% of their original size
36. Here is an example with trilinear filtering turned off (left) and on (right) You can see the trilinear filtering does a much better job at smoothing the image (The images have been scaled up many times)
37. Off On Here is an example with trilinear filtering turned off (left) and on (right) You can see the trilinear filtering does a much better job at smoothing the image (The images have been scaled up many times)
38. private void loadData() { // Load bitmap Bitmap b = getBitmap(); // Enable trilinear filtering b.setHasMipMap(true); } Here is how you can enable trilinear filtering on a Bitmap You can also enable trilinear filtering on a BitmapDrawable if you prefer
39. <bitmap android:mipMap="true" android:src="@drawable/my_drawable" /> You can of course enable trilinear filtering in your XML drawable definition
40. Canvas layers Let’s talk a bit about Canvas layers (as opposed to View.setLayerType.) There are two types of layers and using them can create interesting performance issues
41. @Override protected void onDraw(Canvas canvas) { // Create a clipped layer canvas.save(); canvas.saveLayer(x, y, width, height, Canvas.CLIP_TO_LAYER_SAVE_FLAG); // Draw stuff canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null); canvas.restore(); } The first type of layers is called “clipped layer”. Such layers are created by passing the flag Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer().
42. saveLayer() Here is how clipped layers work. After calling saveLayer(), only the drawing commands that intersect with that layer will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an offscreen render target (bitmap in software, FBO+texture in hardware.)
43. Here is how clipped layers work. After calling saveLayer(), only the drawing commands that intersect with that layer will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an offscreen render target (bitmap in software, FBO+texture in hardware.)
44. Here is how clipped layers work. After calling saveLayer(), only the drawing commands that intersect with that layer will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an offscreen render target (bitmap in software, FBO+texture in hardware.)
45. @Override protected void onDraw(Canvas canvas) { // Create an unclipped layer canvas.save(); canvas.saveLayer(x, y, width, height, 0); // Draw stuff canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null); canvas.restore(); } The second type of layers is called “unclipped layer”. Such layers are created by NOT passing the flag Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer().
46. saveLayer() With unclipped layers, a drawing command will be drawn in any layer it intersects, including the original canvas. In this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N times (N=number of active layers.)
47. With unclipped layers, a drawing command will be drawn in any layer it intersects, including the original canvas. In this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N times (N=number of active layers.)
48. With unclipped layers, a drawing command will be drawn in any layer it intersects, including the original canvas. In this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N times (N=number of active layers.)
49. Using alpha with care Android offers many easy ways to apply alpha on a view. Using alpha can however have a measurable impact the performance of your application.
50. Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer.
51. view.setAlpha(0.5f); View.ALPHA.set(view, 0.5f); ObjectAnimation.ofFloat(view, "alpha", 0.5f) view.animate().alpha(0.5f); view.setAnimation(new AlphaAnimation(1.0f, 0.5f)); Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer.
52. view.setAlpha(0.5f); View.ALPHA.set(view, 0.5f); ObjectAnimation.ofFloat(view, "alpha", 0.5f) view.animate().alpha(0.5f); view.setAnimation(new AlphaAnimation(1.0f, 0.5f)); Canvas.saveLayerAlpha(l, t, r, b, 127, Canvas.CLIP_TO_LAYER_SAVE_FLAG); == Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer.
53. Why is it important to use a separate buffer to apply the alpha? Here is an example. Let’s imagine a View containing 3 photos (bitmaps.) If we animate the opacity to 50% without using a separate buffer the result would look like this one.
54. Why is it important to use a separate buffer to apply the alpha? Here is an example. Let’s imagine a View containing 3 photos (bitmaps.) If we animate the opacity to 50% without using a separate buffer the result would look like this one.
55. Instead of applying the alpha separately, we can use a separate buffer in which we draw the 3 photos and then we apply alpha to that buffer. The result can be seen above and looks a lot better.
56. Instead of applying the alpha separately, we can use a separate buffer in which we draw the 3 photos and then we apply alpha to that buffer. The result can be seen above and looks a lot better.
57. lternatives Let’s examine various solutions you can use to maximize performance when applying alpha on a View.
58. // Not this textView.setAlpha(alpha); // But this int newTextColor = (int) (0xFF * alpha) << 24 | baseTextColor & 0xFFFFFF; textView.setTextColor(newTextColor); If you are setting alpha on a TextView and you don’t have a background you can simply bake the alpha in the text color instead. This also applies to background colors, drawing primitives, etc.
59. // Not this imageView.setAlpha(alpha); // But this imageView.setImageAlpha((int) (alpha * 255)); Similarly with ImageView, you can apply the alpha directly to the image instead of the View itself.
60. // Not this customView.setAlpha(alpha); // But this int alpha = (int) (255 * slider.getProgress() / 100.0f); paint.setAlpha(alpha); canvas.draw*(..., paint); If you are writing a custom view and the drawing commands don’t overlap, you can simply set the alpha on the paint.
61. // Or use a layer view.setLayerType(View.LAYER_TYPE_HARDWARE, null); // Transient layer view.animate().alpha(0).withLayer(); By far the easiest thing to do is to use a hardware layer. The extra buffer copy will happen only once (or every time the view changes) instead of on every frame.
62. // API level 16+ @Override public boolean hasOverlappingRendering() { // Don't lie to us! return false; } If you have a custom view and you know your content does not overlap, override this method and return false. This will trigger an automatic optimization in the hardware rendering pipeline that will bypass the use of a separate buffer.
63. 640 dp 400 dp Canvas Canvas is the API used to draw on Android You should however be very careful with its dimensions
64. @Override protected void onDraw(Canvas canvas) { // Get the dimensions of the Canvas int w = canvas.getWidth(); int h = canvas.getHeight(); canvas.drawRect(0, 0, w, h, mPaint); } Here is code I have seen in many applications The result of these calls might surprise you
65. 1280 px 800 px View 300 px 600 px Canvas is the API used to draw on Android You should however be very careful with its dimensions What will the previous code sample yield as a result?
66. In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap
67. With hardware rendering 600x300 px (size of the View) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap
68. With hardware rendering With software rendering 600x300 px 1280x800 px (size of the View) (size of the window) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap
69. ✂ Clipping Let’s talk about clipping
70. @Override protected void onDraw(Canvas canvas) { // Keep the jellybeans canvas.clipRect(l, t, r, b); // Rotate the jar canvas.rotate(-30.0f, pX, pY); // Draw the jar canvas.drawBitmap(mJellyBeans, x, y, null); } You should be very careful when applying rotations with clipping Here is a simple example of code that clips first, then rotates before drawing a bitmap
71. The previous code will produce something like this. Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.
72. 1. Clip The previous code will produce something like this. Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.
73. 2. Rotate The previous code will produce something like this. Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.
74. 3. Draw The previous code will produce something like this. Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.
75. The previous code will produce something like this. Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.
76. @Override protected void onDraw(Canvas canvas) { // Rotate the jar canvas.rotate(-30.0f, pX, pY); // Keep the jellybeans canvas.clipRect(l, t, r, b); // Draw the jar canvas.drawBitmap(mJellyBeans, x, y, null); } What if we change the order of operations and rotate then clip?
77. This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.
78. 1. Rotate This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.
79. 2. Clip This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.
80. 3. Draw This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.
81. This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.
82. Stencil buffer The stencil buffer is a mask in the GPU. To use the stencil buffer we must clear it and draw into it. This means that transformed clipping operations cause extra drawing commands to be executed This also means we’re using more fillrate
83. Stencil buffer The stencil buffer is a mask in the GPU. To use the stencil buffer we must clear it and draw into it. This means that transformed clipping operations cause extra drawing commands to be executed This also means we’re using more fillrate
84. 640 px 400 px View Imagine a view with the following dimensions Let’s now invalidate that View partially
85. Invalidate 640 px 400 px (170,125) (470,275) Imagine a view with the following dimensions Let’s now invalidate that View partially
86. @Override protected void onDraw(Canvas canvas) { // Query the current clip Rect clip = canvas.getClipBounds(); // ??? Log.d("I/O", "clip = " + clip); } What would you expect the getClipBounds() API to return? (Note: use the variant that takes a REct to avoid allocations)
87. In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions)
88. With hardware rendering 0, 0, 640, 400 (bounds of the View) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions)
89. With hardware rendering With software rendering 0, 0, 640, 400 170,125, 470, 275 (bounds of the View) (bounds of the dirty rect) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions)
90. Reordering barriers Reordering and merging can optimize your applications but specific Canvas calls can prevent reordering. We refer to these calls as “barriers”. Reordering can only happen before or after but cannot work across barriers.
91. Non-rectangular clips The first type of barrier is caused by non-rectangular clips. Be careful as non-rect clips can be introduced by rotations. For instance, rotating a View without setting a layer on that view first will cause non-rect clipping to occur.
92. saveLayer() The second type of barrier is created by calling saveLayer() (or its variant saveLayerAlpha().) Be careful as saveLayerAlpha() can be triggered on your behalf by calling View.setAlpha() as discussed earlier.
93. More info Parleys.com For Butter or Worse Google I/O 2012 Various Android GUI & performance talks Accelerated Android Rendering Google I/O 2011
94. More info Romain’s Tips & Tricks www.curious-creature.org Chet’s Tips & Tricks goo.gl/y9JZr Android Performance Case Study graphics-geek.blogspot.com
95. Q&A google.com/+ChetHaase google.com/+RomainGuy @chethaase @romainguy
96. Developers