Android Graphics Performance

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Welcome/Android Graphics & Performance Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)

Slide 3

Slide 3 text

& Graphics Performance Welcome/Android Graphics & Performance Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)

Slide 4

Slide 4 text

& Chet Haase Romain Guy Welcome/Android Graphics & Performance Chet Haase & Romain Guy, Android Framework engineers (Graphics & animations)

Slide 5

Slide 5 text

Architecture 1

Slide 6

Slide 6 text

Reordering & merging A future update of Android introduces a major change in the hardware accelerated 2D rendering pipeline: re-ordering and merging of drawing commands

Slide 7

Slide 7 text

Save Cancel Maximize compatibility Include metadata This allows for more efficient rendering without changing anything in your application. Here is an example of a simple Android UI

Slide 8

Slide 8 text

Until now Android would always render drawing commands in order. Android now ﬁrst re-orders the commands to ﬁnd optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.

Slide 9

Slide 9 text

Include metadata Maximize compatibility Cancel Save Order of drawing commands Until now Android would always render drawing commands in order. Android now ﬁrst re-orders the commands to ﬁnd optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.

Slide 10

Slide 10 text

Include metadata Maximize compatibility Cancel Save 1. Re-ordering Until now Android would always render drawing commands in order. Android now ﬁrst re-orders the commands to ﬁnd optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.

Slide 11

Slide 11 text

Include metadata Maximize compatibility Cancel Save 2. Merging Until now Android would always render drawing commands in order. Android now ﬁrst re-orders the commands to ﬁnd optimal batches for the GPU. This avoids changing GPU state. Then Android merges commands together to reduce the number of draw calls to the GPU. In this example we start with 8 in-order calls and end up with 3 out-of- order draw calls.

Slide 12

Slide 12 text

Let’s demo re-ordering and merging on a real device, using Google+ as an example

Slide 13

Slide 13 text

Multi-threading In a future update of Android, the hardware accelerated 2D rendering pipeline makes better use of multi-core systems by using multiple threads to perform certain tasks.

Slide 14

Slide 14 text

The blue bar show the UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.

Slide 15

Slide 15 text

Drawing The blue bar show the UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.

Slide 16

Slide 16 text

Drawing Shadows Shadows Shadows Shadows The blue bar show the UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.

Slide 17

Slide 17 text

Drawing Paths Paths Shadows Shadows Shadows Shadows The blue bar show the UI thread performing normal rendering. However the renderer now uses Renderscript to generate drop shadows and thus use all the available core (4 cores in this example.) Paths are also now generated on background threads, as shown in yellow.

Slide 18

Slide 18 text

Non-rectangular clipping A future update of Android adds support for hardware accelerated non-rectangular clipping This includes clipping with paths (circles, curves, rounded rects, etc.) and transformed rects (3D rotations for instance)

Slide 19

Slide 19 text

@Override protected void onDraw(Canvas canvas) { // Clip with a shape Path clip = getPath(); canvas.clipPath(clip); // Draw the content for (int i = 0; i < mLines,length; i++) { TextLine line = mLines[i]; canvas.drawText(line.text, line.x, line.y, mPaint); } } Non-rectangular clips can also be achieved by specifying the Region.Op parameter of the various clip*() methods (you can add, xor, subtract, etc.). Non-rect clipping can be triggered by rotations.

Slide 20

Slide 20 text

Developer Tools 2

Slide 21

Slide 21 text

Overdraw In developer options you can now turn on a “Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Overdraw Blue Green Red Deep Red 1x 2x 3x 4x In developer options you can now turn on a “Show GPU overdraw” feature Apps will be highlighted with various colors. Each color represents the amount of overdraw for each pixel. 1x means the pixel was drawn twice, 2x means the pixel was drawn 3 times, etc.

Slide 25

Slide 25 text

0 3 6 9 12 15 Time in ms Frames Update display lists Process display lists Swap buffers In Android 4.1 we introduced a new rendering proﬁling tool in developer settings This tool was useful to create performance graphs such as this one It was however difficult to use since it required command line tools and a spreadsheet

Slide 26

Slide 26 text

In a future update of Android proﬁling is a lot easier. You can now see the proﬁling graph directly on screen. The old method is also available. The green line indicates the 16ms threshold (60fps.)

Slide 27

Slide 27 text

Slide 28

Slide 28 text

performTraversals draw getDL drawDisplayList systrace flush drawing commands systrace was introduced in Android 4.1 and we’ve added several improvements in 4.2 and future updates of Android.

Slide 29

Slide 29 text

android:sdk $ cd platform-tools/ android:platform-tools $ 㾑 ./systrace.py gfx view freq sched In a future update of Android systrace is also easier to run, no need to change developer options All you need is to run the script from the command line In this example we’re tracing graphics (gfx), UI toolkit (view), CPU frequencies (freq) and the kernel scheduler (sched)

Slide 30

Slide 30 text

And since systrace is easier to use, we added more information. If you enable OpenGL tracing in developer options you will be able to see each individual OpenGL call in systrace (use the gfx tag when running systrace.) You can also enable glGetError() checks after each GL call.

Slide 31

Slide 31 text

import android.os.Trace; @Override public View getView(int pos, View view, ViewGroup parent) { Trace.beginSection("getView"); if (view == null) { view = createView(); } // Trace time spent binding data Trace.beginSection("bind"); bindView(pos, view); Trace.endSection(); Trace.endSection(); return view; } We’re introducing a new API, android.os.Trace, to let you add your own information to systrace. Systrace events are lightweight and add almost no overhead. Using it is easy: simply wrap the section of code to trace with beginSection/endSection. You can nest sections.

Slide 32

Slide 32 text

android:sdk $ cd platform-tools/ android:platform-tools $ 㾑 ./systrace.py -a com.example.myapp You won’t see your events by default, you must specify the package name of your app using the -a option when invoking systrace. You can specify several packages and combine it with the built-in tags such as gfx and view.

Slide 33

Slide 33 text

Tips & Tricks 3

Slide 34

Slide 34 text

Overdraw demo

Slide 35

Slide 35 text

Trilinear ﬁltering Trilinear ﬁltering was introduced in Android 4.2 and can be used to improve the visual quality of your bitmaps if you scale them down to less than 50% of their original size

Slide 36

Slide 36 text

Here is an example with trilinear ﬁltering turned off (left) and on (right) You can see the trilinear ﬁltering does a much better job at smoothing the image (The images have been scaled up many times)

Slide 37

Slide 37 text

Off On Here is an example with trilinear ﬁltering turned off (left) and on (right) You can see the trilinear ﬁltering does a much better job at smoothing the image (The images have been scaled up many times)

Slide 38

Slide 38 text

private void loadData() { // Load bitmap Bitmap b = getBitmap(); // Enable trilinear filtering b.setHasMipMap(true); } Here is how you can enable trilinear ﬁltering on a Bitmap You can also enable trilinear ﬁltering on a BitmapDrawable if you prefer

Slide 39

Slide 39 text

You can of course enable trilinear ﬁltering in your XML drawable deﬁnition

Slide 40

Slide 40 text

Canvas layers Let’s talk a bit about Canvas layers (as opposed to View.setLayerType.) There are two types of layers and using them can create interesting performance issues

Slide 41

Slide 41 text

@Override protected void onDraw(Canvas canvas) { // Create a clipped layer canvas.save(); canvas.saveLayer(x, y, width, height, Canvas.CLIP_TO_LAYER_SAVE_FLAG); // Draw stuff canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null); canvas.restore(); } The ﬁrst type of layers is called “clipped layer”. Such layers are created by passing the ﬂag Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer().

Slide 42

Slide 42 text

saveLayer() Here is how clipped layers work. After calling saveLayer(), only the drawing commands that intersect with that layer will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an offscreen render target (bitmap in software, FBO+texture in hardware.)

Slide 43

Slide 43 text

Here is how clipped layers work. After calling saveLayer(), only the drawing commands that intersect with that layer will be rendered. They are also rendered only in that layer. In practice this means the renderer must create an offscreen render target (bitmap in software, FBO+texture in hardware.)

Slide 44

Slide 44 text

Slide 45

Slide 45 text

@Override protected void onDraw(Canvas canvas) { // Create an unclipped layer canvas.save(); canvas.saveLayer(x, y, width, height, 0); // Draw stuff canvas.drawBitmap(bugDroid, 0.0f, 0.0f, null); canvas.restore(); } The second type of layers is called “unclipped layer”. Such layers are created by NOT passing the ﬂag Canvas.CLIP_TO_LAYER_SAVE_FLAG when calling Canvas.saveLayer().

Slide 46

Slide 46 text

saveLayer() With unclipped layers, a drawing command will be drawn in any layer it intersects, including the original canvas. In this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N times (N=number of active layers.)

Slide 47

Slide 47 text

With unclipped layers, a drawing command will be drawn in any layer it intersects, including the original canvas. In this example you can see the bitmap is drawn in both the layer we’ve created and the original drawing surface. This is how fading edges are implemented on Android. Unclipped layers are expensive: each command is executed N times (N=number of active layers.)

Slide 48

Slide 48 text

Slide 49

Slide 49 text

Using alpha with care Android offers many easy ways to apply alpha on a view. Using alpha can however have a measurable impact the performance of your application.

Slide 50

Slide 50 text

Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer.

Slide 51

Slide 51 text

view.setAlpha(0.5f); View.ALPHA.set(view, 0.5f); ObjectAnimation.ofFloat(view, "alpha", 0.5f) view.animate().alpha(0.5f); view.setAnimation(new AlphaAnimation(1.0f, 0.5f)); Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer.

Slide 52

Slide 52 text

view.setAlpha(0.5f); View.ALPHA.set(view, 0.5f); ObjectAnimation.ofFloat(view, "alpha", 0.5f) view.animate().alpha(0.5f); view.setAnimation(new AlphaAnimation(1.0f, 0.5f)); Canvas.saveLayerAlpha(l, t, r, b, 127, Canvas.CLIP_TO_LAYER_SAVE_FLAG); == Each line shows a different way to set alpha on a View. Some will trigger an animation, some won’t. What matters is they all trigger the same side effect: the creation of an expensive offscreen buffer.

Slide 53

Slide 53 text

Why is it important to use a separate buffer to apply the alpha? Here is an example. Let’s imagine a View containing 3 photos (bitmaps.) If we animate the opacity to 50% without using a separate buffer the result would look like this one.

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Instead of applying the alpha separately, we can use a separate buffer in which we draw the 3 photos and then we apply alpha to that buffer. The result can be seen above and looks a lot better.

Slide 56

Slide 56 text

Instead of applying the alpha separately, we can use a separate buffer in which we draw the 3 photos and then we apply alpha to that buffer. The result can be seen above and looks a lot better.

Slide 57

Slide 57 text

lternatives Let’s examine various solutions you can use to maximize performance when applying alpha on a View.

Slide 58

Slide 58 text

// Not this textView.setAlpha(alpha); // But this int newTextColor = (int) (0xFF * alpha) << 24 | baseTextColor & 0xFFFFFF; textView.setTextColor(newTextColor); If you are setting alpha on a TextView and you don’t have a background you can simply bake the alpha in the text color instead. This also applies to background colors, drawing primitives, etc.

Slide 59

Slide 59 text

// Not this imageView.setAlpha(alpha); // But this imageView.setImageAlpha((int) (alpha * 255)); Similarly with ImageView, you can apply the alpha directly to the image instead of the View itself.

Slide 60

Slide 60 text

// Not this customView.setAlpha(alpha); // But this int alpha = (int) (255 * slider.getProgress() / 100.0f); paint.setAlpha(alpha); canvas.draw*(..., paint); If you are writing a custom view and the drawing commands don’t overlap, you can simply set the alpha on the paint.

Slide 61

Slide 61 text

// Or use a layer view.setLayerType(View.LAYER_TYPE_HARDWARE, null); // Transient layer view.animate().alpha(0).withLayer(); By far the easiest thing to do is to use a hardware layer. The extra buffer copy will happen only once (or every time the view changes) instead of on every frame.

Slide 62

Slide 62 text

// API level 16+ @Override public boolean hasOverlappingRendering() { // Don't lie to us! return false; } If you have a custom view and you know your content does not overlap, override this method and return false. This will trigger an automatic optimization in the hardware rendering pipeline that will bypass the use of a separate buffer.

Slide 63

Slide 63 text

640 dp 400 dp Canvas Canvas is the API used to draw on Android You should however be very careful with its dimensions

Slide 64

Slide 64 text

@Override protected void onDraw(Canvas canvas) { // Get the dimensions of the Canvas int w = canvas.getWidth(); int h = canvas.getHeight(); canvas.drawRect(0, 0, w, h, mPaint); } Here is code I have seen in many applications The result of these calls might surprise you

Slide 65

Slide 65 text

1280 px 800 px View 300 px 600 px Canvas is the API used to draw on Android You should however be very careful with its dimensions What will the previous code sample yield as a result?

Slide 66

Slide 66 text

In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap

Slide 67

Slide 67 text

With hardware rendering 600x300 px (size of the View) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap

Slide 68

Slide 68 text

With hardware rendering With software rendering 600x300 px 1280x800 px (size of the View) (size of the window) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the window... or the Bitmap if you draw into a bitmap

Slide 69

Slide 69 text

✂ Clipping Let’s talk about clipping

Slide 70

Slide 70 text

@Override protected void onDraw(Canvas canvas) { // Keep the jellybeans canvas.clipRect(l, t, r, b); // Rotate the jar canvas.rotate(-30.0f, pX, pY); // Draw the jar canvas.drawBitmap(mJellyBeans, x, y, null); } You should be very careful when applying rotations with clipping Here is a simple example of code that clips ﬁrst, then rotates before drawing a bitmap

Slide 71

Slide 71 text

The previous code will produce something like this. Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

Slide 72

Slide 72 text

1. Clip The previous code will produce something like this. Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

Slide 73

Slide 73 text

2. Rotate The previous code will produce something like this. Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

Slide 74

Slide 74 text

3. Draw The previous code will produce something like this. Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

Slide 75

Slide 75 text

The previous code will produce something like this. Note that the clip rect remains screen aligned. This is trivial to handle for the OpenGL pipeline, a simple scissoring operation.

Slide 76

Slide 76 text

@Override protected void onDraw(Canvas canvas) { // Rotate the jar canvas.rotate(-30.0f, pX, pY); // Keep the jellybeans canvas.clipRect(l, t, r, b); // Draw the jar canvas.drawBitmap(mJellyBeans, x, y, null); } What if we change the order of operations and rotate then clip?

Slide 77

Slide 77 text

This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.

Slide 78

Slide 78 text

1. Rotate This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.

Slide 79

Slide 79 text

2. Clip This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.

Slide 80

Slide 80 text

3. Draw This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.

Slide 81

Slide 81 text

This is a much more expensive operation. As you can see the rectangle is not screen aligned anymore. We need to take a more expensive code path that relies on the stencil buffer.

Slide 82

Slide 82 text

Stencil buffer The stencil buffer is a mask in the GPU. To use the stencil buffer we must clear it and draw into it. This means that transformed clipping operations cause extra drawing commands to be executed This also means we’re using more ﬁllrate

Slide 83

Slide 83 text

Slide 84

Slide 84 text

640 px 400 px View Imagine a view with the following dimensions Let’s now invalidate that View partially

Slide 85

Slide 85 text

Invalidate 640 px 400 px (170,125) (470,275) Imagine a view with the following dimensions Let’s now invalidate that View partially

Slide 86

Slide 86 text

@Override protected void onDraw(Canvas canvas) { // Query the current clip Rect clip = canvas.getClipBounds(); // ??? Log.d("I/O", "clip = " + clip); } What would you expect the getClipBounds() API to return? (Note: use the variant that takes a REct to avoid allocations)

Slide 87

Slide 87 text

In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions)

Slide 88

Slide 88 text

With hardware rendering 0, 0, 640, 400 (bounds of the View) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions)

Slide 89

Slide 89 text

With hardware rendering With software rendering 0, 0, 640, 400 170,125, 470, 275 (bounds of the View) (bounds of the dirty rect) In hardware, the API returns the dimensions of the View In software, the API returns the dimensions of the dirty rect Discussion about multiple invalidates (dirty unions)

Slide 90

Slide 90 text

Reordering barriers Reordering and merging can optimize your applications but speciﬁc Canvas calls can prevent reordering. We refer to these calls as “barriers”. Reordering can only happen before or after but cannot work across barriers.

Slide 91

Slide 91 text

Non-rectangular clips The ﬁrst type of barrier is caused by non-rectangular clips. Be careful as non-rect clips can be introduced by rotations. For instance, rotating a View without setting a layer on that view ﬁrst will cause non-rect clipping to occur.

Slide 92

Slide 92 text

saveLayer() The second type of barrier is created by calling saveLayer() (or its variant saveLayerAlpha().) Be careful as saveLayerAlpha() can be triggered on your behalf by calling View.setAlpha() as discussed earlier.

Slide 93

Slide 93 text

More info Parleys.com For Butter or Worse Google I/O 2012 Various Android GUI & performance talks Accelerated Android Rendering Google I/O 2011

Slide 94

Slide 94 text

More info Romain’s Tips & Tricks www.curious-creature.org Chet’s Tips & Tricks goo.gl/y9JZr Android Performance Case Study graphics-geek.blogspot.com