Droidcon Berlin - A ride through AOSP's new colors

Dynamic theming on the Pixel is an amazing feature, but we as Android engineers always care about how it works under the hood!

In this talk, we will understand “Monet”, the wallpaper-based dynamic theming system.

We will see:

What is Monet?
Dynamic behavior of system-wide theming
Wallpaper color extraction
K-means and Celebi quantizers
Color extraction strategy
Runtime dynamic color updation
Dynamic themes with M3 components for Jetpack Compose
In this session, we will aim to understand how the Android operating system works to enable dynamic theming (basics fundamentals of Material You). We will deep dive into AOSP to examine parts of the framework that form Monet and ultimately understand how dynamic system colors work.

Join me on the colourful ride!

Transcript

1. siddroid Droidcon Berlin, July 2022 A ride through AOSP’s new colors Sid Patil
2. siddroid siddroid About Me Android Engineer at Delivery Hero based in Berlin Built Paytm, a mobile payments platform in India, ~ 100m users Organizes and hosts communities like Kotlin Mumbai & Android Worldwide Know more at siddroid.com A ride through AOSP’s new colors!
3. siddroid siddroid Questions? Do you recognise this logo? How many of you use a Google Pixel? Have you heard about Monet? Do you use Material 3 in your apps? M3 logo from material.io
4. siddroid siddroid Material You, the face of the Android OS A ride through AOSP’s new colors
5. siddroid siddroid
6. siddroid siddroid .. the current face of Android A ride through AOSP’s new colors!
7. siddroid siddroid A promo From material.io
8. siddroid siddroid What is Monet? The dynamic theme engine on Android Enables features like dynamic color and themed icons on Android 12 and 13 First introduced with Android 12 and Google Pixel devices Fully added in AOSP from Android 12L A promo From material.io
9. siddroid siddroid How would you build it? Read and observe changes in the wallpaper Make sense of information from the device wallpaper Extract dominant colors of the wallpaper Map them to a design speci fi cation Propagate them to the apps Picture by Sagar Viradiya
10. siddroid siddroid Let’s step into the Android framework A ride through AOSP’s new colors!
11. siddroid siddroid Android framework components System UI 
 ThemeOverlayController & ThemeOverlayApplier WallpaperManager & WallpaperManagerService Core 
 WallpaperColors System UI 
 Monet Graphics 
 Palette Graphics 
 Quantizers A ride through AOSP’s new colors!
12. siddroid siddroid Wallpaper Manager Service public class WallpaperManagerService extends IWallpaperManager.Stub implements IWallpaperManagerService class WallpaperObserver extends FileObserver frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
13. siddroid siddroid Wallpaper Manager Service class WallpaperObserver extends FileObserver { mWallpaperFile = new File(mWallpaperDir, WALLPAPER); mWallpaperLockFile = new File(mWallpaperDir, WALLPAPER_LOCK_ORIG); final boolean sysWallpaperChanged = (mWallpaperFile.equals(changedFile)); final boolean lockWallpaperChanged = (mWallpaperLockFile.equals(changedFile)); } frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
14. siddroid siddroid Wallpaper Manager Service class WallpaperObserver extends FileObserver { // If home wallpaper has changed notifyColorsWhich |= FLAG_SYSTEM; // If lockscreen wallpaper has changed notifyColorsWhich |= FLAG_LOCK; if (notifyColorsWhich != 0) { notifyWallpaperColorsChanged(wallpaper, notifyColorsWhich); } } frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
15. siddroid siddroid Wallpaper Manager Service void notifyWallpaperColorsChanged( WallpaperData wallpaper, int which) { wallpaper.connection.forEachDisplayConnector(connector -> { notifyWallpaperColorsChangedOnDisplay( wallpaper, which, connector.mDisplayId ); }); } frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
16. siddroid siddroid Monet & System-UI A ride through AOSP’s new colors!
17. siddroid siddroid System-UI & Feature flags // Flags.java public static final BooleanFlag MONET = new BooleanFlag(800, true, R.bool.flag_monet); // FeatureFlags.java public boolean isMonetEnabled() { return isEnabled(Flags.MONET); } // ThemeOverlayController.java mIsMonetEnabled = featureFlags.isMonetEnabled(); frameworks/base/packages/SystemUI/src/com/android/systemui/ fl ags/FeatureFlags.java & Flags.java frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java

18. siddroid siddroid Theme Overlay Controller OnColorsChangedListener mOnColorsChangedListener = new OnColorsChangedListener() { @Override public void onColorsChanged( WallpaperColors wallpaperColors, int which, int userId) { // Post performing user Id specific logic handleWallpaperColors(wallpaperColors, which, userId); } }; frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
19. siddroid siddroid Wallpaper Manager Service private void notifyWallpaperColorsChangedOnDisplay( WallpaperData wallpaper, int which, int displayId) { boolean needsExtraction; needsExtraction = wallpaper.primaryColors == null; if (needsExtraction) { extractColors(wallpaper); } notifyColorListeners( ... ); } frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
20. siddroid siddroid Wallpaper Manager Service private void extractColors(WallpaperData wallpaper) { WallpaperColors colors = null; if (cropFile != null) { Bitmap bitmap = BitmapFactory.decodeFile(cropFile); if (bitmap != null) { colors = WallpaperColors.fromBitmap(bitmap); } } else if (defaultImageWallpaper) { colors = extractDefaultImageWallpaperColors(); } } frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
21. siddroid siddroid Extracting colors from the wallpaper A ride through AOSP’s new colors!
22. siddroid siddroid Wallpaper Colors public static WallpaperColors fromBitmap(@NonNull Bitmap bitmap) { final int bitmapArea = bitmap.getWidth() * bitmap.getHeight(); if (bitmapArea > MAX_WALLPAPER_EXTRACTION_AREA) { Size optimalSize = calculateOptimalSize( bitmap.getWidth(), bitmap.getHeight()); bitmap = Bitmap.createScaledBitmap(bitmap, optimalSize.getWidth(), optimalSize.getHeight(), false); } } frameworks/base/core/java/android/app/WallpaperColors.java
23. siddroid siddroid Wallpaper Colors if (ActivityManager.isLowRamDeviceStatic()) { palette = Palette .from(bitmap, new VariationalKMeansQuantizer()) .maximumColorCount(5) .resizeBitmapArea(MAX_WALLPAPER_EXTRACTION_AREA) .generate(); } else { palette = Palette .from(bitmap, new CelebiQuantizer()) .maximumColorCount(128) .resizeBitmapArea(MAX_WALLPAPER_EXTRACTION_AREA) .generate(); } frameworks/base/core/java/android/app/WallpaperColors.java
24. siddroid siddroid Dominant colors and Quantizers A ride through AOSP’s new colors!
25. siddroid siddroid Image quantization An image is a set of Pixels A quantization algorithm reduces the number of pixels Each pixel consists of a single color Android uses K-means clustering (Variational and Celebi) Device RAM determines which technique is used Pixelated image from cs.princeton.edu
26. siddroid siddroid Color science A color denotes a given value from the spectrum of light For example, RGB and CMYK are the most commonly used The HSL model uses hue, saturation and lightness of a color Android uses a custom HCT model for calculating colors
27. siddroid siddroid Material Design’s HCT model Stands for Hue, Chroma and Tone A blend of LCH and CAM16 color spaces Best of both color systems Better contrast ratio calculations with tone values Android uses this custom HCT model for calculating Material You theme colors via Monet From material.io “The science of color and design” by James O'Leary
28. siddroid siddroid Why Monet? Original Older Algorithm Celebi Quantizer From material.io “The science of color and design” by James O'Leary Claude Monet’s “Impression Sunrise” A ride through AOSP’s new colors!
29. siddroid siddroid Palette & Swatches Palette(List<Swatch> swatches) { mSwatches = swatches; mDominantSwatch = findDominantSwatch(); } public Swatch(@ColorInt int colorInt, int population) { mColor = Color.valueOf(colorInt); mPopulation = population; } frameworks/base/core/java/com/android/internal/graphics/palette/Palette.java
30. siddroid siddroid Finding the seed color Swatches are fi rst sorted by population count Each color is assigned a “score” Colors are then sorted in descending order by the score Iterate through the scored colors and identify seed color Seed color is used to build the fi nal theme via Monet
31. siddroid siddroid Re-evaluating System Theme A ride through AOSP’s new colors!
32. siddroid siddroid When to re-evaluate? First boot of the device Via broadcast receiver when Intent.ACTION_WALLPAPER_CHANGED is received On a new user setup When the active user is changed private void reevaluateSystemTheme
33. siddroid siddroid Theme Overlay Controller private final BroadcastReceiver mBroadcastReceiver = new BroadcastReceiver() { @Override public void onReceive(Context context, Intent intent) { if (Intent.ACTION_WALLPAPER_CHANGED.equals(intent.getAction())) { . .. mAcceptColorEvents = true; } } }; frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
34. siddroid siddroid Runtime updates of System colors A ride through AOSP’s new colors!
35. siddroid siddroid Theme Overlay Controller protected @Nullable FabricatedOverlay getOverlay(int color, int type) frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
36. siddroid siddroid Monet Color Scheme // Build Monet ColorScheme mColorScheme = new ColorScheme(color, nightMode); List<Integer> colorShades = mColorScheme.getAllAccentColors(); // OR List<Integer> colorShades = mColorScheme.getAllNeutralColors(); frameworks/base/packages/SystemUI/monet/src/com/android/systemui/monet/ColorScheme.kt frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
37. siddroid siddroid Monet Color Scheme String name = type == ACCENT ? "accent" : "neutral"; for (int i = 0; i < colorShades.size(); i ++ ) { int luminosity = i % paletteSize; int paletteIndex = i / paletteSize + 1; String resourceName; switch (luminosity) { case 0: resourceName = "android:color/system_" + name + paletteIndex + "_10"; break; case 1: resourceName = "android:color/system_" + name + paletteIndex + "_50"; break; default: int l = luminosity - 1; resourceName = "android:color/system_" + name + paletteIndex + "_" + l + "00"; } } frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
38. siddroid siddroid System color resources public static final int system_accent3_600 = 17170521; public static final int system_accent3_700 = 17170522; public static final int system_accent3_800 = 17170523; public static final int system_accent3_900 = 17170524; public static final int system_neutral1_0 = 17170461; public static final int system_neutral1_10 = 17170462; public static final int system_neutral1_100 = 17170464; Android .R fi le
39. siddroid siddroid Theme Overlay Applier /* * All theme customization categories used by the system, in order that they should be applied, * starts with launcher and grouped by target package. */ static final List<String> THEME_CATEGORIES = Lists.newArrayList( OVERLAY_CATEGORY_SYSTEM_PALETTE, OVERLAY_CATEGORY_ICON_LAUNCHER, OVERLAY_CATEGORY_SHAPE, OVERLAY_CATEGORY_FONT, OVERLAY_CATEGORY_ACCENT_COLOR, OVERLAY_CATEGORY_ICON_ANDROID, OVERLAY_CATEGORY_ICON_SYSUI, OVERLAY_CATEGORY_ICON_SETTINGS, OVERLAY_CATEGORY_ICON_THEME_PICKER); frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayApplier.java
40. siddroid siddroid Monet and Material 3 with compose A ride through AOSP’s new colors!
41. siddroid siddroid Tonal Palette internal class TonalPalette( // 13 shades of each val neutral: Color, val neutralVariant: Color, val primary: Color, val secondary: Color, val tertiary: Color, ) androidx.compose.material3.TonalPalette.kt
42. siddroid siddroid An image From material.io
43. siddroid siddroid Dynamic Tonal Palette internal fun dynamicTonalPalette(context: Context): TonalPalette = TonalPalette( neutral100 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_0), neutral99 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_10), neutral95 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_50), .. ) androidx.compose.material3.DynamicTonalPalette.kt
44. siddroid siddroid Dynamic Tonal Palette @RequiresApi(23) private object ColorResourceHelper { @DoNotInline fun getColor(context: Context, @ColorRes id: Int): Color { return Color(context.resources.getColor(id, context.theme)) } } androidx.compose.material3.DynamicTonalPalette.kt
45. siddroid siddroid Color Schemes fun dynamicLightColorScheme(context: Context): ColorScheme { val tonalPalette = dynamicTonalPalette(context) return lightColorScheme( primary = tonalPalette.primary40, onPrimary = tonalPalette.primary100, primaryContainer = tonalPalette.primary90, androidx.compose.material3.DynamicTonalPalette.kt
46. siddroid siddroid Material Color Utilities https://github.com/material-foundation/material-color-utilities
47. siddroid siddroid Material Color Utilities https://github.com/material-foundation/material-color-utilities
48. siddroid siddroid A promo From material.io
49. siddroid siddroid Shout out to! The Android and Material design teams at Google! James O’leary for his work on color science behind Material You material.io & Guru Singh for his help with my questions! Android code search A video of Monet in Action from Material foundation
50. siddroid siddroid Further reading
51. siddroid siddroid Danke Schone! Droidcon Berlin, July 2022