$30 off During Our Annual Pro Sale. View Details »

Droidcon Berlin - A ride through AOSP's new colors

Sid Patil
July 11, 2022

Droidcon Berlin - A ride through AOSP's new colors

Dynamic theming on the Pixel is an amazing feature, but we as Android engineers always care about how it works under the hood!

In this talk, we will understand “Monet”, the wallpaper-based dynamic theming system.

We will see:

What is Monet?
Dynamic behavior of system-wide theming
Wallpaper color extraction
K-means and Celebi quantizers
Color extraction strategy
Runtime dynamic color updation
Dynamic themes with M3 components for Jetpack Compose
In this session, we will aim to understand how the Android operating system works to enable dynamic theming (basics fundamentals of Material You). We will deep dive into AOSP to examine parts of the framework that form Monet and ultimately understand how dynamic system colors work.

Join me on the colourful ride!

Sid Patil

July 11, 2022
Tweet

Other Decks in Science

Transcript

  1. siddroid siddroid About Me Android Engineer at Delivery Hero based

    in Berlin Built Paytm, a mobile payments platform in India, ~ 100m users Organizes and hosts communities like Kotlin Mumbai & Android Worldwide Know more at siddroid.com A ride through AOSP’s new colors!
  2. siddroid siddroid Questions? Do you recognise this logo? How many

    of you use a Google Pixel? Have you heard about Monet? Do you use Material 3 in your apps? M3 logo from material.io
  3. siddroid siddroid What is Monet? The dynamic theme engine on

    Android Enables features like dynamic color and themed icons on Android 12 and 13 First introduced with Android 12 and Google Pixel devices Fully added in AOSP from Android 12L A promo From material.io
  4. siddroid siddroid How would you build it? Read and observe

    changes in the wallpaper Make sense of information from the device wallpaper Extract dominant colors of the wallpaper Map them to a design speci fi cation Propagate them to the apps Picture by Sagar Viradiya
  5. siddroid siddroid Android framework components System UI 
 ThemeOverlayController &

    ThemeOverlayApplier WallpaperManager & WallpaperManagerService Core 
 WallpaperColors System UI 
 Monet Graphics 
 Palette Graphics 
 Quantizers A ride through AOSP’s new colors!
  6. siddroid siddroid Wallpaper Manager Service public class WallpaperManagerService extends IWallpaperManager.Stub

    implements IWallpaperManagerService class WallpaperObserver extends FileObserver frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
  7. siddroid siddroid Wallpaper Manager Service class WallpaperObserver extends FileObserver {

    mWallpaperFile = new File(mWallpaperDir, WALLPAPER); mWallpaperLockFile = new File(mWallpaperDir, WALLPAPER_LOCK_ORIG); final boolean sysWallpaperChanged = (mWallpaperFile.equals(changedFile)); final boolean lockWallpaperChanged = (mWallpaperLockFile.equals(changedFile)); } frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
  8. siddroid siddroid Wallpaper Manager Service class WallpaperObserver extends FileObserver {

    // If home wallpaper has changed notifyColorsWhich |= FLAG_SYSTEM; // If lockscreen wallpaper has changed notifyColorsWhich |= FLAG_LOCK; if (notifyColorsWhich != 0) { notifyWallpaperColorsChanged(wallpaper, notifyColorsWhich); } } frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
  9. siddroid siddroid Wallpaper Manager Service void notifyWallpaperColorsChanged( WallpaperData wallpaper, int

    which) { wallpaper.connection.forEachDisplayConnector(connector -> { notifyWallpaperColorsChangedOnDisplay( wallpaper, which, connector.mDisplayId ); }); } frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
  10. siddroid siddroid System-UI & Feature flags // Flags.java public static

    final BooleanFlag MONET = new BooleanFlag(800, true, R.bool.flag_monet); // FeatureFlags.java public boolean isMonetEnabled() { return isEnabled(Flags.MONET); } // ThemeOverlayController.java mIsMonetEnabled = featureFlags.isMonetEnabled(); frameworks/base/packages/SystemUI/src/com/android/systemui/ fl ags/FeatureFlags.java & Flags.java frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java

  11. siddroid siddroid Theme Overlay Controller OnColorsChangedListener mOnColorsChangedListener = new OnColorsChangedListener()

    { @Override public void onColorsChanged( WallpaperColors wallpaperColors, int which, int userId) { // Post performing user Id specific logic handleWallpaperColors(wallpaperColors, which, userId); } }; frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
  12. siddroid siddroid Wallpaper Manager Service private void notifyWallpaperColorsChangedOnDisplay( WallpaperData wallpaper,

    int which, int displayId) { boolean needsExtraction; needsExtraction = wallpaper.primaryColors == null; if (needsExtraction) { extractColors(wallpaper); } notifyColorListeners( ... ); } frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
  13. siddroid siddroid Wallpaper Manager Service private void extractColors(WallpaperData wallpaper) {

    WallpaperColors colors = null; if (cropFile != null) { Bitmap bitmap = BitmapFactory.decodeFile(cropFile); if (bitmap != null) { colors = WallpaperColors.fromBitmap(bitmap); } } else if (defaultImageWallpaper) { colors = extractDefaultImageWallpaperColors(); } } frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
  14. siddroid siddroid Wallpaper Colors public static WallpaperColors fromBitmap(@NonNull Bitmap bitmap)

    { final int bitmapArea = bitmap.getWidth() * bitmap.getHeight(); if (bitmapArea > MAX_WALLPAPER_EXTRACTION_AREA) { Size optimalSize = calculateOptimalSize( bitmap.getWidth(), bitmap.getHeight()); bitmap = Bitmap.createScaledBitmap(bitmap, optimalSize.getWidth(), optimalSize.getHeight(), false); } } frameworks/base/core/java/android/app/WallpaperColors.java
  15. siddroid siddroid Wallpaper Colors if (ActivityManager.isLowRamDeviceStatic()) { palette = Palette

    .from(bitmap, new VariationalKMeansQuantizer()) .maximumColorCount(5) .resizeBitmapArea(MAX_WALLPAPER_EXTRACTION_AREA) .generate(); } else { palette = Palette .from(bitmap, new CelebiQuantizer()) .maximumColorCount(128) .resizeBitmapArea(MAX_WALLPAPER_EXTRACTION_AREA) .generate(); } frameworks/base/core/java/android/app/WallpaperColors.java
  16. siddroid siddroid Image quantization An image is a set of

    Pixels A quantization algorithm reduces the number of pixels Each pixel consists of a single color Android uses K-means clustering (Variational and Celebi) Device RAM determines which technique is used Pixelated image from cs.princeton.edu
  17. siddroid siddroid Color science A color denotes a given value

    from the spectrum of light For example, RGB and CMYK are the most commonly used The HSL model uses hue, saturation and lightness of a color Android uses a custom HCT model for calculating colors
  18. siddroid siddroid Material Design’s HCT model Stands for Hue, Chroma

    and Tone A blend of LCH and CAM16 color spaces Best of both color systems Better contrast ratio calculations with tone values Android uses this custom HCT model for calculating Material You theme colors via Monet From material.io “The science of color and design” by James O'Leary
  19. siddroid siddroid Why Monet? Original Older Algorithm Celebi Quantizer From

    material.io “The science of color and design” by James O'Leary Claude Monet’s “Impression Sunrise” A ride through AOSP’s new colors!
  20. siddroid siddroid Palette & Swatches Palette(List<Swatch> swatches) { mSwatches =

    swatches; mDominantSwatch = findDominantSwatch(); } public Swatch(@ColorInt int colorInt, int population) { mColor = Color.valueOf(colorInt); mPopulation = population; } frameworks/base/core/java/com/android/internal/graphics/palette/Palette.java
  21. siddroid siddroid Finding the seed color Swatches are fi rst

    sorted by population count Each color is assigned a “score” Colors are then sorted in descending order by the score Iterate through the scored colors and identify seed color Seed color is used to build the fi nal theme via Monet
  22. siddroid siddroid When to re-evaluate? First boot of the device

    Via broadcast receiver when Intent.ACTION_WALLPAPER_CHANGED is received On a new user setup When the active user is changed private void reevaluateSystemTheme
  23. siddroid siddroid Theme Overlay Controller private final BroadcastReceiver mBroadcastReceiver =

    new BroadcastReceiver() { @Override public void onReceive(Context context, Intent intent) { if (Intent.ACTION_WALLPAPER_CHANGED.equals(intent.getAction())) { . .. mAcceptColorEvents = true; } } }; frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
  24. siddroid siddroid Theme Overlay Controller protected @Nullable FabricatedOverlay getOverlay(int color,

    int type) frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
  25. siddroid siddroid Monet Color Scheme // Build Monet ColorScheme mColorScheme

    = new ColorScheme(color, nightMode); List<Integer> colorShades = mColorScheme.getAllAccentColors(); // OR List<Integer> colorShades = mColorScheme.getAllNeutralColors(); frameworks/base/packages/SystemUI/monet/src/com/android/systemui/monet/ColorScheme.kt frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
  26. siddroid siddroid Monet Color Scheme String name = type ==

    ACCENT ? "accent" : "neutral"; for (int i = 0; i < colorShades.size(); i ++ ) { int luminosity = i % paletteSize; int paletteIndex = i / paletteSize + 1; String resourceName; switch (luminosity) { case 0: resourceName = "android:color/system_" + name + paletteIndex + "_10"; break; case 1: resourceName = "android:color/system_" + name + paletteIndex + "_50"; break; default: int l = luminosity - 1; resourceName = "android:color/system_" + name + paletteIndex + "_" + l + "00"; } } frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
  27. siddroid siddroid System color resources public static final int system_accent3_600

    = 17170521; public static final int system_accent3_700 = 17170522; public static final int system_accent3_800 = 17170523; public static final int system_accent3_900 = 17170524; public static final int system_neutral1_0 = 17170461; public static final int system_neutral1_10 = 17170462; public static final int system_neutral1_100 = 17170464; Android .R fi le
  28. siddroid siddroid Theme Overlay Applier /* * All theme customization

    categories used by the system, in order that they should be applied, * starts with launcher and grouped by target package. */ static final List<String> THEME_CATEGORIES = Lists.newArrayList( OVERLAY_CATEGORY_SYSTEM_PALETTE, OVERLAY_CATEGORY_ICON_LAUNCHER, OVERLAY_CATEGORY_SHAPE, OVERLAY_CATEGORY_FONT, OVERLAY_CATEGORY_ACCENT_COLOR, OVERLAY_CATEGORY_ICON_ANDROID, OVERLAY_CATEGORY_ICON_SYSUI, OVERLAY_CATEGORY_ICON_SETTINGS, OVERLAY_CATEGORY_ICON_THEME_PICKER); frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayApplier.java
  29. siddroid siddroid Tonal Palette internal class TonalPalette( // 13 shades

    of each val neutral: Color, val neutralVariant: Color, val primary: Color, val secondary: Color, val tertiary: Color, ) androidx.compose.material3.TonalPalette.kt
  30. siddroid siddroid Dynamic Tonal Palette internal fun dynamicTonalPalette(context: Context): TonalPalette

    = TonalPalette( neutral100 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_0), neutral99 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_10), neutral95 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_50), .. ) androidx.compose.material3.DynamicTonalPalette.kt
  31. siddroid siddroid Dynamic Tonal Palette @RequiresApi(23) private object ColorResourceHelper {

    @DoNotInline fun getColor(context: Context, @ColorRes id: Int): Color { return Color(context.resources.getColor(id, context.theme)) } } androidx.compose.material3.DynamicTonalPalette.kt
  32. siddroid siddroid Color Schemes fun dynamicLightColorScheme(context: Context): ColorScheme { val

    tonalPalette = dynamicTonalPalette(context) return lightColorScheme( primary = tonalPalette.primary40, onPrimary = tonalPalette.primary100, primaryContainer = tonalPalette.primary90, androidx.compose.material3.DynamicTonalPalette.kt
  33. siddroid siddroid Shout out to! The Android and Material design

    teams at Google! James O’leary for his work on color science behind Material You material.io & Guru Singh for his help with my questions! Android code search A video of Monet in Action from Material foundation