A dive into the Android Window Manager

Slide 1

Slide 1 text

Behind the scene A Dive into the Window Manager WINDOW TOKEN INPUT MANAGER FRAME BUFFER SURFACE FLINGER ACTIVITY MANAGER Happy to be here and see you all here. Feels good to be back in person. We hope to teach you a thing or two about the WM and provide you with a deeper understanding and appreciation of Android. Lots to talk about, so let's get started.

Slide 2

Slide 2 text

This is Vadim. And I’m Pablo.

Slide 3

Slide 3 text

Android framework engineers from Zurich from the WM team.

Slide 4

Slide 4 text

Window Manager Our goal today is to give you an overview of what the Window Manager is, what are its responsibilities, and how you control it. As you know, software developers are really bad at naming things (or keeping things compartmentalized…). Window Manager is no exception.

Slide 5

Slide 5 text

Window Manager Activity Lifecycle Insets Splash Screen Foldables Desktop Task Layout Params Flags OEMs Status Bar Animations Now, the WM does manage windows. But, it does way more than that. It's also responsible for the Activity lifecycle, the system bars, dealing with insets, multiple screen handling and so on. We will do our best to tie back what the Window Manager does to what you already know, going back and forth between the developer facing API and the underlying mechanism. Once we have a good general understanding of the Window Manager we’ll go over some APIs that are related to the WM and try to bring a new light and deeper understanding to them.

Slide 6

Slide 6 text

The kind of question you wouldn't think to ask yourself one day What is a Window ? VADIM The ﬁrst thing for us to understand is what a Window is. Despite our extensive use of illustrations depicting house window, we won't be talking about them.

Slide 7

Slide 7 text

All consumer operating systems with a user interface have a concept of windows. It's very obvious on desktop, like here on this early version of MacOS, where windows have border and titles.

Slide 8

Slide 8 text

Michael Vincent michaelv.org Some people even thought it was not obvious enough so they literally named their windowed operating system "Windows"

Slide 9

Slide 9 text

But window are not that obvious on mobile operating systems, like Android. They don't have borders, they don't have title anymore, they often overlap and share a common background. Who can tell how many windows are currently visible on screen 1 ? 2 ? 3 ? More ?

Slide 10

Slide 10 text

Switching to the overview screen, it's a little more easy to identify our window, so if we exclude the overview screen itself, let's see what windows we have

Slide 11

Slide 11 text

Application Of course we have our application window, containing the content of our Activity

Slide 12

Slide 12 text

Status Bar A little less obvious are the system bars which each live in a separate window, with its own content and populated by the system UI process. Here we have the status bar…

Slide 13

Slide 13 text

Navigation Bar And down below we have the navigation bar. You can also note that both them are transparent and share the background of the application underneath

Slide 14

Slide 14 text

Let's now get back to our application and introduce another concept

Slide 15

Slide 15 text

Have you ever heard about the Activity Lifecycle? Have you ever heard about the Activity Lifecycle ? NEXT SLIDE

Slide 16

Slide 16 text

Have you ever heard about the Activity Lifecycle? If you haven't, you should leave the room because you are at the wrong conference If you haven't, you should leave the room because you are at the wrong conference, unless you've skipped the first lesson of your Android course and you are here to make up for it

Slide 17

Slide 17 text

onStart() onResume() onCreate() onPause() For the others who actually were good students. you are all familiar with this diagram. You are also familiar about the events that triggers change the the lifecycle state But have you ever wonder …. NEXT SLIDE

Slide 18

Slide 18 text

onStart() onResume() onCreate() onPause() Who is pulling the strings ? ….Who is pulling the strings? Who is actually creating you Activity instance ? Who is making the call to the lifecycle callbacks ? And you probably guessed that the answer to this is "The System", but since you are sitting here today, you might have also understood that it is more specifically invoked by the Window Manager

Slide 19

Slide 19 text

“The window manager allows apps to safely share the display.” That right so the window manager is responsible for orchestrating everything to get your app started, running, on screen, and stopped. Fundamentally, all this is to control when and how the app gets displayed. So taking a step back, really, the main role of the window manager is to allow apps to safely share the display. Just like we have mechanisms to share CPU, memory or any other shared resource, the window manager provides a safe mechanism for different apps, and the system UI to share the display. So to allow apps to safely share the display we need two key things:

Slide 20

Slide 20 text

Provide apps with a way to draw on screen. 1. 2. Enforce rules around what is shown on screen. 1. Of course, a mechanism for apps to draw what they want to display for the user onto the screen. 2. Robust mechanism to enforce rules about what is shown on screen so apps can’t just take control of your screen.

Slide 21

Slide 21 text

Provide apps with a way to draw on screen. 1. 2. Enforce rules around what is shown on screen. So let’s start with that first part and explain how the Window Manager provides apps with a way to draw on screen. Going back to the lifecycles – you probably noticed that by the time onResume is called your activity is visible on screen. But what you might not know is how do we get from no app to having your activity visible on the screen in onResume?

Slide 22

Slide 22 text

To explain this – we will take this sample example. Where we tap on chrome on the launcher to continue learning from the NYTs about how a window can possibly change our view of the earth. And what we will try to explain is that magic step in between being on the launcher and the NYTs article being open.

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Window Manager Window Manager State Start activity requested Create Task Create ActivityRecord Create Process for app Start activity Thread Run lifecyle callbacks Application Not Running Launcher Window Manager Intent Category = LAUNCHER Action = MAIN Package = Chrome And in our example, the way that happens is, when you click on that Chrome icon on the launcher, it will send an intent with package = com.google.android.apps.chrome, action = MAIN (start as a main entry point), category = LAUNCHER (should appear in the Launcher as a top-level application). Working with the package manager to figure out exactly what needs to run that intent will be processed by the WM.

Slide 25

Slide 25 text

Window Manager Window Manager State Start activity requested Create Task Create ActivityRecord Create Process for app Start activity Thread Run lifecyle callbacks Task Application Not Running Once, the WM receives the request. It starts off by requesting that the WindowManager create a Task, since we don’t have one of those yet. Notice how a Task just popped up there on the right hand side - that will keep track of what the internal state of the window manager looks like for this specific application as we go along. So now we have this Task there to store stateful information about the task (if it’s running, it’s bounds, position, TODO whatelse?).

Slide 26

Slide 26 text

Window Manager Window Manager State Start activity requested Create Task Create ActivityRecord Create Process for app Start activity Thread Run lifecyle callbacks Task Application Not Running Task Activity A Activity B Activity C Back Stack Just briefly though, as a reminder of what a Task is. Simply put, a task is a collection of activities. Conceptually, anything you see on the recents page is its own task. In most cases you have one task per app, but in some case start multiple task. Like in Gmail when you start a new email. The activities in a task also form the backstack. So when you open a new activity in a task like opening an email to read in Gmail, it will add a new activity on that stack and going back will pop it.

Slide 27

Slide 27 text

Window Manager Window Manager State Start activity requested Create Task Create ActivityRecord Create Process for app Start activity Thread Run lifecyle callbacks Task ActivityRecord Application Not Running So naturally we now need something to store our activity and its state. So the next step is to create an ActivityRecord, which is literally just a container to hold windows within our activity and any stateful information about the activity. Just like the Task. They don’t actually directly present themselves on the screen. The purpose of the Task and ActivityRecord window containers is to enforce rules based on their state onto their children. So the Task might have bounds which the ActivityRecord and its children won’t be able to draw out of. The activity record might be hidden propagating the effect down to the children, hiding any children windows.

Slide 28

Slide 28 text

Window Manager Window Manager State Start activity requested Create Task Create ActivityRecord Create Process for app Start activity Thread Run lifecyle callbacks Task ActivityRecord Application Not Running We now got a representation of our app in the Window Manager to manage how we want it to show on screen. But notice that our app process here on the left, is still not running, so we still don’t actually have a place to run this new activity in. So since there isn’t already a process for the app that we can use, the Window Manager will request to start a new process for Chrome. So what does that look like?

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Window Manager Window Manager State Start activity requested Create Task Create ActivityRecord Create Process for app Start activity Thread Run lifecyle callbacks Task ActivityRecord Application Not Running Zygote and if you look up the definition in Wikipedia it is “the initial cell from which a new organism is produced”. And that’s kind of what it is. Zygote is a process that is setup to be ready to run an application, it includes everything an app may need at runtime, including all the activity code. And it’s just always waiting and listening for a request to fork itself.

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Window Manager Window Manager State Start activity requested Create Task Create ActivityRecord Create Process for app Start activity Thread Run lifecyle callbacks Task ActivityRecord Application Not Running Zygote Window Manager fork() App Process System Process Zygote, listening on those requests will fork itself. We have an app process that is exactly the same as the Zygote process.

Slide 33

Slide 33 text

Window Manager Application Window Manager State Start activity requested Create Task Create ActivityRecord Create Process for app Start activity Thread Run lifecyle callbacks Task ActivityRecord So now our application is finally started and is just a forked (so a clone) of the Zygote process not doing anything yet.

Slide 34

Slide 34 text

Window Manager Application Window Manager State Start activity requested Create Task Create ActivityRecord Create Process for app Start activity Thread onResume onCreate onStart Run lifecyle callbacks Task ActivityRecord And now, the WM can send a request to start the activity thread in the newly created app process. And that will initialize a few things to get the activity ready. Including registering itself back with the WM to receive lifecycle callbacks.

Slide 35

Slide 35 text

Slide 36

Slide 36 text

onResume onCreate onStart WindowManager Application main_activity.xml onCreate() { setContentView( R.layout.main_activity) } Activity Task ActivityRecord Window Manager State Task ActivityRecord Great, we have a process running, and our Activity is created. The system finally gave us control and we start executing our code. Not far from the start of your Activity, you are setting the contentView. This is actually the first interaction your application will have with the window manager.

Slide 37

Slide 37 text

main_activity.xml onCreate() { setContentView(R.layout.main_activity) } ActivityRecord Activity Great, we have a process running, and our Activity is created. The system finally gave us control and we start executing our code. Not far from the start of your Activity, you are setting the contentView. This is actually the first interaction your application will have with the window manager.

Slide 38

Slide 38 text

main_activity.xml Activity PhoneWindow ActivityRecord - Title - Action Bar - Menu Under the hood, the PhoneWindow is created. Historically, the Phone window was used to display the ActionBar, Menu, and Window Title

Slide 39

Slide 39 text

main_activity.xml Activity PhoneWindow ActivityRecord PhoneWindow But this only happen in your application process, and the true master of window is the window manager,

Slide 40

Slide 40 text

main_activity.xml Activity PhoneWindow ActivityRecord PhoneWindow WindowManager .addWindow(PhoneWindow)* *Not accurate, but that's the idea so your application needs to request that an actual window gets created. This is slightly simplified, but that's the idea. In reality, when your application communicates with the window manager, the call needs to go from you application process to the system process where the window manager lives.

Slide 41

Slide 41 text

Binder main_activity.xml Activity PhoneWindow ActivityRecord PhoneWindow WindowManager .addWindow(PhoneWindow) To achieve this, Android has a Inter Proocess Communication sytem, or IPC, called binder. This is a true marvel of the Android operating system, and very complex, … NEXT SLIDE

Slide 42

Slide 42 text

Binder main_activity.xml Activity PhoneWindow ActivityRecord PhoneWindow Binder but I'd like to give you a very very very simplified overview, because honestly, I'm way less intelligent than the people, or shall I say the person, who came with this idea

Slide 43

Slide 43 text

Binder main_activity.xml Activity PhoneWindow ActivityRecord PhoneWindow App Process System Process MyService.foo(Image, Bar) MyServiceImpl.foo(Image, Bar) So one side you have you app process, who wants to call a service in the System process. On the other side, you have the actual service implementation. In our case, the window managers service You application only sees a fake implementation of the service, and if you step into the method with the debugger, you'll see it's a stub, with no actual code

Slide 44

Slide 44 text

Binder main_activity.xml Activity PhoneWindow ActivityRecord PhoneWindow BinderProxy Kernel App Process System Process BinderStub MyService.foo(Image, Bar) transaction[bytes, bytes] transaction[bytes, bytes] MyServiceImpl.foo(Image, Bar) Serialization What's happening is that the parameters of your methods are serialized by the BinderProxy, created from the Service interface and then passed down at the kernel level where the binder will transfer a copy of the data to the system process. Then them mirror operation happens, where the data is deserialized using the BinderStub of the actual service and finally the actual service implementation get's called with the recreated copy of the original data. Note that this operation is costly, and the system tries to limit them when possible, and are usually ran outside of the main thread

Slide 45

Slide 45 text

main_activity.xml Activity PhoneWindow ActivityRecord mWindowSession .addToDisplayAsUser(IWindow) Binder PhoneWindow So now you understand what's actually happening. The window creation request goes to the window manager with some payload information including the IWindow, which is used as a token to uniquely identify our window in the window manager

Slide 46

Slide 46 text

main_activity.xml Activity PhoneWindow ActivityRecord mWindowSession .addToDisplayAsUser(IWindow) Binder PhoneWindow WindowState IWindow On the system side, a WindowState instance is created. From now on, this will be the source of truth for all data related to our window such as the bounds, the orientation, the owning process and many more.

Slide 47

Slide 47 text

main_activity.xml Activity PhoneWindow ActivityRecord WindowState installDecor() PhoneWindow The next step happens the application side. The PhoneWindow calls install decor

Slide 48

Slide 48 text

main_activity.xml Activity PhoneWindow ActivityRecord WindowState DecorView PhoneWindow This will have the effect to create the first Actual View of our hierarchy called the decor view. It's role is to display your application content and everything that goes around.

Slide 49

Slide 49 text

main_activity.xml Activity PhoneWindow ActivityRecord WindowState DecorView PhoneWindow - Title - Action Bar - Menu If you remember, I told you that the phone window was responsible for drawing the window title, action bar and menues

Slide 50

Slide 50 text

main_activity.xml Activity PhoneWindow ActivityRecord WindowState DecorView PhoneWindow - Title - Action Bar - Menu OBSOLETE But all of this elements are obsolete on modern applications who feel more immersive by drawing from edge to edge.

Slide 51

Slide 51 text

main_activity.xml Activity PhoneWindow ActivityRecord WindowState DecorView PhoneWindow To Achieve this edge to edge feeling, you can see that the decor view has some colored area on top and bottom. Let's take a step back in time and see this evolution

Slide 52

Slide 52 text

On the left hand side, we have an Android Gingerbread screenshot. Annectotelly, [NEXT SLIDE] this was the android version of my first android device: a very French Archos 7

Slide 53

Slide 53 text

this was the android version of my first android device: a very French Archos 7

Slide 54

Slide 54 text

App View In the case, the location of our application content hasn't much change.

Slide 55

Slide 55 text

B App View Status Bar It still sits below the status bar. But you'll notice that since API 21 applications can customize the status bar color. I'm not teaching you anything new if you bought a new phone in the last 8 years

Slide 56

Slide 56 text

B App View Phone Window Status Bar Wi Window Title The notable change relevant to the window manager happened on the Phone window. On Gingerbread, the status bar and our application window were clearly separated, and the phone window was responsible for showing the window title

Slide 57

Slide 57 text

B App View Phone Window Status Bar Wi windowDrawsSystemBarBackgrounds DecorView with Status Bar Background But on modern phones, the status and navigation bar sit on top of our application window, which now has the responsibility to draw the status bar background when you set the flag windowDrawsSystemBarBackground. And this name now makes send because our window draws the background of the system bar

Slide 58

Slide 58 text

main_activity.xml Activity PhoneWindow ActivityRecord WindowState DecorView PhoneWindow Let's go back to our application, now that we now why we have a phone window and a decor view, we need to attach them together

Slide 59

Slide 59 text

Activity ActivityRecord WindowState PhoneWindow main_activity.xml DecorView And we finally need to add the content view of our application.

Slide 60

Slide 60 text

main_activity.xml Activity ActivityRecord WindowState mContentParent PhoneWindow DecorView The decor view has some other views including the content parent that will be the actual parent of our layout.

Slide 61

Slide 61 text

Activity ActivityRecord WindowState PhoneWindow DecorView mContentParent main_activity.xml And there we go, our application now has its view hierarchy attached a the window hierarchy. The window manager knows about us

Slide 62

Slide 62 text

PhoneWindow DecorView ? Activity PABLO Ok that’s cool – so we’ve now ended up with a nice little structure on the app side to represent what we want our final activity to look like. But how exactly does that object representation of the Activity in the app process end up on the screen?

Slide 63

Slide 63 text

PhoneWindow DecorView Application Hardware ? Activity Window Manager Ok that’s cool – so we’ve now ended up with a nice little structure on the app side to represent what we want our final activity to look like. But how exactly does that object representation of the Activity in the app process end up on the screen?

Slide 64

Slide 64 text

Application Application Hardware Window Manager View Hierarchy Window The Where The Whole The What Ok that’s cool – so we’ve now ended up with a nice little structure on the app side to represent what we want our final activity to look like. But how exactly does that object representation of the Activity in the app process end up on the screen?

Slide 65

Slide 65 text

Application Application Hardware Window Manager DecorView Window The Where The Whole The What SurfaceFlinger View Hierarchy Ok that’s cool – so we’ve now ended up with a nice little structure on the app side to represent what we want our final activity to look like. But how exactly does that object representation of the Activity in the app process end up on the screen?

Slide 66

Slide 66 text

Application Application Hardware Window Manager DecorView Window The Where The Whole The What SurfaceFlinger ViewRootImpl View Hierarchy Ok that’s cool – so we’ve now ended up with a nice little structure on the app side to represent what we want our final activity to look like. But how exactly does that object representation of the Activity in the app process end up on the screen?

Slide 67

Slide 67 text

Application Application Hardware Window Manager DecorView Window The Where The Whole The What ViewRootImpl View Hierarchy SurfaceFlinger So we know a bit about how the Windows are laid out and how the views are structured in our application, but we don’t know about how the SF works to get stuff on display. So let’s explain that and look at how the surface flinger get’s things onto the display and how the app and WM interact with it.

Slide 68

Slide 68 text

Task ActivityRecord WindowState Window Manager Remember this from the WM – this is how our app is represented in the WM.

Slide 69

Slide 69 text

Task ActivityRecord WindowState Window Manager Container Layer Container Layer Buffer Layer Surface Flinger The surface flinger has a similar tree structure to the WM. Layers instead of windows. Windows might hold information about abstract states of the windows, enforce rules about what apps can change, process inputs, and much more. The layers are very concrete and only hold information that is directly relevant to figure out what needs to be drawn on screen. You can think of it as the direct tree representation of what you see on screen. So that is things like opacity, position, translations, and bounds that need to be applied. And importantly buffer layers contain graphic buffers containing pixels. Like in WM – parent layer rules are enforced on children as you would expect. Visiblily, bounds, etc. Each window that is visible has an associated layer in surface flinger.

Slide 70

Slide 70 text

composed pixels in memory reads Hardware Display Frame Buffer Container Layer Container Layer Buffer Layer Surface Flinger Tree The surface flinger is responsible for composing all those layers into a single final buffer that the display can show. That buffer is called the Frame buffer, and is just a place in memory that the hardware display is configured to read form everytime it refreshes and display those pixels on screen.

Slide 71

Slide 71 text

Task ActivityRecord WindowState Window Manager Container Layer Container Layer Buffer Layer Surface Flinger Application DecorView So what we want is to get the contents of the DecorView from the application into this buffer layer here. So that it can get composed with the rest of the surface flinger tree into the frame buffer to be shown on screen.

Slide 72

Slide 72 text

Task ActivityRecord WindowState Window Manager Container Layer Container Layer Buffer Layer Surface Flinger Application SurfaceView Buffer Layer To bring back these layers in SF to something you might be familiar with – the most directly related object to a layer we have in the application side is a SurfaceView. And when you post a canvas to it you are updating the buffer of the layer directly. So if that helps you can see buffer layers as surface views.

Slide 73

Slide 73 text

Window Layer So, we said each window, if visible, has a layer, but how does a window control a layer? And how does it allow applications to control that layer for its window so it can draw in it?

Slide 74

Slide 74 text

WindowState Buffer Layer SurfaceControl Well, when a window is created it creates an associated layer in the surface flinger which then provides it with a SurfaceControl which is what is used to control the layer.

Slide 75

Slide 75 text

Task ActivityRecord WindowState SurfaceControl SurfaceControl SurfaceControl Window Manager So every window in the Window Manager tree has a surface control that it uses to manage its layer counterpart to specify how to display itself on screen. Now how do you use that surface control to update the SF tree and define what and how should end up on screen?

Slide 76

Slide 76 text

Transaction setVisibility( SurfaceControl , true) setPosition( SurfaceControl , x, y) setBuffer( SurfaceControl ,[1001100100]) Container Layer Container Layer Buffer Layer Buffer Layer Buffer Layer Surface Flinger That would be the role of transactions. So with the SurfaceControl we can create a Transaction which is just a batch of operations to apply in SurfaceFlinger and send that over to update the Surface Flinger hierarchy. In particular we are interested here in the setBuffer operation which allows us to update the buffer of the layer associated with a surface control. And so that’s what we use to send over the activity’s content that we want to show on screen to surface flinger to be composed.

Slide 77

Slide 77 text

WindowState Buffer Layer SurfaceControl Application If you remember, the visual content of our activity is in the DecorView in the application process and not in the WindowManager. So we need to get this surface control for the app’s buffer layer into the application process. How does that happen exactly?

Slide 78

Slide 78 text

Hardware Window Manager ViewRootImpl#performTraversals Application Window Manager ViewRootImpl#relayoutWindow Render Thread Task ActivityRecord WindowState SurfaceControl SurfaceControl SurfaceControl ViewRootImpl SurfaceControl Remember ViewRootImpl is our link between the WM and the application (holding a reference to the WM to interact with it). So going back to the app, once the app is ready to draw (after on resume), performTraversals (which traverses the entire view hierarchy in the app and get’s it ready to show on screen) is run in the render thread. And this on the first run and whenever a relayout is required will trigger relayoutWindow which will get the SurfaceControl associated with the WindowState, which is the Window for the activity’s content.

Slide 79

Slide 79 text

Hardware Window Manager ViewRootImpl#performTraversals Application Window Manager ViewRootImpl#relayoutWindow Render Thread Task ActivityRecord WindowState SurfaceControl SurfaceControl SurfaceControl ViewRootImpl SurfaceControl So now that the ViewRootImpl has access to that surface control it can use it to control that layer. (note that it can’t just do whatever it wants because the task and activity record’s respective layers will be set up to have bounds which will restrict what the app can do with the SurfaceControl which is a child of those). Can use it to create children views of that but nothing above (so limited by parent layers, which WM controls), so WM still keeps full control.

Slide 80

Slide 80 text

Hardware Window Manager ViewRootImpl#performTraversals Application Window Manager ViewRootImpl#relayoutWindow Render Thread Task ActivityRecord WindowState SurfaceControl SurfaceControl SurfaceControl ViewRootImpl ViewRootImpl#performDraw SurfaceControl [1001100100] Now at some point performDraw will be called which will render the activity’s view hierarchy into a buffer (1s and 0s representing pixels).

Slide 81

Slide 81 text

Hardware Window Manager ViewRootImpl#performTraversals Application Window Manager ViewRootImpl#relayoutWindow Render Thread Task ActivityRecord WindowState SurfaceControl SurfaceControl SurfaceControl ViewRootImpl ViewRootImpl#performDraw SurfaceControl [1001100100] If you want an explanation of how that works exactly – all you have to do is go back in time 4 years and check out Romain's and Chet's talk that they gave here. Or I guess YouTube also works if that’s too hard.

Slide 82

Slide 82 text

Hardware Window Manager ViewRootImpl#performTraversals Application Window Manager ViewRootImpl#relayoutWindow Render Thread Task ActivityRecord WindowState Container Layer Container Layer Buffer Layer ViewRootImpl ViewRootImpl#performDraw SurfaceControl [1001100100] Transaction So finally after onDraw we can send a transansaction to set the buffer of activity’s root buffer layer.

Slide 83

Slide 83 text

Hardware Window Manager ViewRootImpl#performTraversals Application Window Manager ViewRootImpl#relayoutWindow Render Thread Task ActivityRecord WindowState SurfaceControl SurfaceControl SurfaceControl ViewRootImpl ViewRootImpl#performDraw SurfaceControl [1001100100] Transaction Transaction setBuffer( SurfaceControl ,[1001100100]) And that’s what the transaction we would send looks like. Taking the SurfaceControl and drawn buffer we have in the application.

Slide 84

Slide 84 text

Hardware Window Manager ViewRootImpl#performTraversals Application Window Manager ViewRootImpl#relayoutWindow Render Thread Task ActivityRecord WindowState SurfaceControl SurfaceControl SurfaceControl ViewRootImpl ViewRootImpl#performDraw SurfaceControl [1001100100] Transaction Transaction setBuffer( SurfaceControl ,[1001100100]) SurfaceControl [1001100100]

Slide 85

Slide 85 text

Application Hardware ? Activity PhoneWindow DecorView Window Manager Ok so let’s recap –

Slide 86

Slide 86 text

Application Hardware Activity PhoneWindow DecorView Window Manager SurfaceControl The first step is for the app to render itself into a buffer. (TODO: What’s the function that’s called to do this?) (TODO: Traversal of view hierarchy to do that) Now with the SurfaceControl we’ve got in ViewRootImpl and the buffer we just created a transaction gets created setting the buffer of the layer the surface control is associated with.

Slide 87

Slide 87 text

Application Hardware Activity PhoneWindow DecorView Window Manager SurfaceControl Buffer onDraw(canvas) [1001100100] So that transaction is sent over the Surface Flinger and applied there. Which in this case means setting the buffer of the buffer layer for the activity.

Slide 88

Slide 88 text

Application Hardware Activity PhoneWindow DecorView Window Manager SurfaceControl Buffer [1001100100] Transaction setBuffer( ) onDraw(canvas) Surface flinger then composes the tree which now includes the activity’s buffer into the frame buffer. Which the display is configured to read from.

Slide 89

Slide 89 text

Application Hardware Activity PhoneWindow DecorView Window Manager SurfaceControl Buffer rendered into [1001100100] Transaction SurfaceControl Buffer setBuffer( ) And tada - our activity ends up on screen.

Slide 90

Slide 90 text

Application Hardware Window Manager Transaction SurfaceControl Buffer setBuffer( ) Surface Flinger

Slide 91

Slide 91 text

Application Hardware Window Manager Transaction SurfaceControl Buffer setBuffer( ) Surface Flinger Container Layer Container Layer Buffer Layer

Slide 92

Slide 92 text

Application Hardware Window Manager Transaction SurfaceControl Buffer setBuffer( ) Surface Flinger Container Layer Container Layer Buffer Layer Frame Buffer compose

Slide 93

Slide 93 text

Application Hardware Window Manager Frame Buffer reads

Slide 94

Slide 94 text

Application Hardware Window Manager Success

Slide 95

Slide 95 text

Provide apps with a way to draw on screen. 1. 2. Enforce rules about what is shown on screen. So now we have a general understanding of how with the help and orchestration of the Window Manager we go from having no app open to having it’s main activity on screen. And this process can be extrapolated out to pretty much anything that gets drawn on screen. But we haven’t properly introduced the mechanisms that ensure apps safe sharing between what’s on screen. And how the Window Manager ultimately has full control of what gets to be shown on screen even though as we’ve seen apps are able to post their buffer pretty much directly to the display to be shown.

Slide 96

Slide 96 text

Task WindowContainer ActivityRecord WindowToken Gmail Activity WindowState Back to our WM hierachy. Windows have states – state contain rules above how to be displayed. Propagated down – e.g. visibility, position, and more abstract stuff that influences those properties sent over the surface flinger.

Slide 97

Slide 97 text

Task WindowContainer ActivityRecord WindowToken Gmail Activity WindowState That how we prevent the app form doing whatever it wants. The application only have access to the bottom most window – and it’s associated layer. And can do whatever it wants so still has a lot of flexibility. But because the parents are controled by WM. WM utilimately decides what to show.

Slide 98

Slide 98 text

DisplayContent RootDisplayArea DisplayArea WindowedMagnification DisplayArea WindowTokens DisplayArea OneHanded WindowToken WindowContainer Wallpaper WindowState DisplayArea FullscreenMagnification StatusBar WindowState DisplayArea WindowTokens WindowToken WindowContainer DisplayArea HideDisplayCutout DisplayArea OneHanded DisplayArea FullscreenMagnification DisplayArea DefaultTaskDisplayArea Task WindowContainer ActivityRecord WindowToken Gmail Activity WindowState DisplayArea WindowTokens WindowToken WindowContainer NavigationBar WindowState This part in the dotted rectangle, it the part that should now be familiar to you - it’s what our simplified view on the window manager state has been throughout the slides. But in reality the window manager state is bigger and more complex than that (as you can see on screen), and in fact this is only part of it. Z-index is defined by hierarchy DisplayAreas allow easy manipulation for different features (generated, don’t need to know how)

Slide 99

Slide 99 text

No content

Slide 100

Slide 100 text

User launches app It takes time Application is ready That's a lot to digest! Not only for us, but also for the system. The whole process and window creation process takes time

Slide 101

Slide 101 text

User launches app It takes time Application is ready 400ms ~ 1000ms Running around the 400ms on high end phones to a magnitude of seconds on lower end phones

Slide 102

Slide 102 text

User launches app App is ready to be shown (layout drawn and populated) App startup time App process is created and window hierarchy are created Application is drawing Controlled by the app This delay, called the app startup time, is unavoidable

Slide 103

Slide 103 text

User launches app App is ready to be shown (layout drawn and populated) App startup time App process is created and window hierarchy are created 1st frame is drawn but layout is not populated Short background task run by the app (e.g local database fetch, template layout) Application is drawing Controlled by the app On top of that, you application might also be doing some initialization work such as local database fetch or even network call, we'll need to have a discussion after the talk if you are doing this.

Slide 104

Slide 104 text

User launches app App is ready to be shown (layout drawn and populated) App startup time 1st frame is drawn but layout is not populated Short background task run by the app (e.g local database fetch, template layout) Application is drawing Delay Flickers Controlled by the app This waiting time is noticeable by users who will see a delay between the time they click on a launcher icon and the time when the app is actually showing up on screen. They might even see some flickering of the element on the screen if you populate your layout incrementally while you application is already visible To solve this, Android 12 introduced the splash screen

Slide 105

Slide 105 text

Android 12 provides a default splash screen for all applications The Splash Screen If you fell asleep it’s time to wake up :P

Slide 106

Slide 106 text

SplashScreen User launches app App is ready to be shown (layout drawn and populated) In the SystemUI process Controlled by the app App startup time 1st frame is drawn but layout is not populated Short background task run by the app (e.g local database fetch, template layout) Application is drawing The splash screen is a special kind of window called a starting window. As opposed to you application window, the splash screen is created in the system ui process, who is in charge to draw all the system element like the status bar, and is always running. This gives the advantage of always being available to show a window with no delay By default, all application on Android 12 have a splash screen and it s shown until the first frame of the application is drawn.

Slide 107

Slide 107 text

Customize the Icon Fill the background with a speciﬁc single color. windowSplashScreenBackground: color Set a background behind the splash screen icon. This is useful if there is not enough contrast between the window background and the icon. windowSplashScreenIconBackground: color Replace an icon in the center of the starting window, if the object is animatable and drawable(e.g. AnimationDrawable, AnimatedVectorDrawable), then it will also play the animation while showing the starting window windowSplashScreenAnimatedIcon: drawable The SplashScreen UI is customized using theme attributes Vadim By default, the launcher icon is used and the windowBackground

Slide 108

Slide 108 text

build.gradle: dependencies { ... implementation 'androidx.core:core-splashscreen:1.0.0-beta01' } Setting up the splash screen

Slide 109

Slide 109 text

build.gradle: dependencies { ... implementation 'androidx.core:core-splashscreen:1.0.0-beta01' } Setting up the splash screen

Slide 110

Slide 110 text

res/values/themes.xml: ... // Set the splash screen background <item name="windowSplashScreenBackground">@color/...</item> // Use to add either a drawable or animated drawable. <item name="windowSplashScreenAnimatedIcon">@drawable/...</item> //Required for animated icons <item name="windowSplashScreenAnimationDuration">200</item> // Set the actual App theme <item name="postSplashScreenTheme">@style/Theme.App</item> # Required. Setting up the splash screen

Slide 111

Slide 111 text

Slide 112

Slide 112 text

fun onCreate() { val splashScreen = installSplashScreen() super.onCreate() ... } Setting up the splash screen MainActivity.kt

Slide 113

Slide 113 text

fun onCreate() { val splashScreen = installSplashScreen() super.onCreate() ... viewModel.isLoading = true splashScreen.setKeepOnScreenCondition{ viewModel.isLoading } ... } class ViewModel { fun loadStuff() { . . . viewModel.isLoading = false } } Wait for the application to be ready MainActivity.kt

Slide 114

Slide 114 text

Slide 115

Slide 115 text

How to deal with a routing Activity

Slide 116

Slide 116 text

How to deal with a routing Activity Don't do it

Slide 117

Slide 117 text

How to deal with a routing Activity Don't do it Compose NavigationComponent

Slide 118

Slide 118 text

How to deal with a routing Activity class RoutingActivity : Activity { fun onCreate() { super.onCreate() ... startNextActivity() } } class HomeActivity : Activity { fun onCreate() { super.onCreate() ... } Here is an example, you can pause on this slide. Don’t need to talk about example, just showing to listener so they can pause.

Slide 119

Slide 119 text

How to deal with a routing Activity class RoutingActivity : Activity { fun onCreate() { val splashScreen = installSplashScreen() super.onCreate() ... splashScreen .setKeepOnScreenCondition{ true } startNextActivity() } } class HomeActivity : Activity { fun onCreate() { val splashScreen = installSplashScreen() super.onCreate() ... } Here is an example, you can pause on this slide. Don’t need to talk about example, just showing to listener so they can pause.

Slide 120

Slide 120 text

Slide 121

Slide 121 text

Slide 122

Slide 122 text

Customize exit animation 1. SplashScreen with AVD 2. Exit Animation Listener 3. Lottie Animation 4. Circular Reveal 5. SplashScreenViewProvider .remove() Here is an example, you can pause on this slide. Don’t need to talk about example, just showing to listener so they can pause.

Slide 123

Slide 123 text

fun onCreate() { val splashScreen = installSplashScreen() super.onCreate() ... splashScreen.onExitAnimationListener = this::onSplashScreenExit } private fun onSplashScreenExit(viewProvider: SplashScreenViewProvider) { applyLottiAnimation(viewProvider.iconView) .doOnEnd { cicularRevealHideAnimation(viewProvider.view) } .doOnEnd { viewProvider.remove() } } Customize exit animation Here is an example, you can pause on this slide. Don’t need to talk about example, just showing to listener so they can pause.

Slide 124

Slide 124 text

Exit Animation potential overlap potential overlap potential overlap SplashScreen window displayed Animated Splash Screen icon User launches app App is ready to be shown (layout drawn and populated) End of transition Splash screen customization Controlled by the app App startup time 1st frame is drawn but layout is not populated Short background task run by the app (e.g local database fetch, template layout) SplashScreen removed In app only Animation (e.g shared element transition) In the SystemUI process It can also use animate the content of the app to provide a nice transition between the splash screen and the content like a share element transition with the icon

Slide 125

Slide 125 text

Thank you Pablo Gamito Software Engineer @pablo_gamito Vadim Caen Software Engineer @vadimcaen