Droidcon Berlin - A ride through AOSP's new colors

Dynamic theming on the Pixel is an amazing feature, but we as Android engineers always care about how it works under the hood!

In this talk, we will understand “Monet”, the wallpaper-based dynamic theming system.

We will see:

What is Monet?
Dynamic behavior of system-wide theming
Wallpaper color extraction
K-means and Celebi quantizers
Color extraction strategy
Runtime dynamic color updation
Dynamic themes with M3 components for Jetpack Compose
In this session, we will aim to understand how the Android operating system works to enable dynamic theming (basics fundamentals of Material You). We will deep dive into AOSP to examine parts of the framework that form Monet and ultimately understand how dynamic system colors work.

Join me on the colourful ride!

Transcript

1. siddroid
   Droidcon Berlin, July 2022
   A ride through AOSP’s new colors
   Sid Patil
   

   View Slide

2. siddroid
   siddroid
   About Me
   
   
   Android Engineer at Delivery Hero
   based in Berlin
   
   
   Built Paytm, a mobile payments
   platform in India, ~ 100m users
   
   
   Organizes and hosts communities
   like Kotlin Mumbai & Android
   Worldwide
   
   
   Know more at siddroid.com
   A ride through AOSP’s new colors!
   

   View Slide

3. siddroid
   siddroid
   Questions?
   
   
   Do you recognise this logo?
   
   
   How many of you use a Google
   Pixel?
   
   
   Have you heard about Monet?
   
   
   Do you use Material 3 in your
   apps?
   M3 logo from material.io
   

   View Slide

4. siddroid
   siddroid
   Material You, the face of the
   Android OS
   A ride through AOSP’s new colors
   

   View Slide

5. siddroid
   siddroid
   

   View Slide

6. siddroid
   siddroid
   .. the current face of Android
   A ride through AOSP’s new colors!
   

   View Slide

7. siddroid
   siddroid
   A promo From material.io
   

   View Slide

8. siddroid
   siddroid
   What is Monet?
   
   
   The dynamic theme engine on
   Android
   
   
   Enables features like dynamic
   color and themed icons on
   Android 12 and 13
   
   
   First introduced with Android 12
   and Google Pixel devices
   
   
   Fully added in AOSP from Android
   12L
   A promo From material.io
   

   View Slide

9. siddroid
   siddroid
   How would you build it?
   
   
   Read and observe changes in the
   wallpaper
   
   
   Make sense of information from
   the device wallpaper
   
   
   Extract dominant colors of the
   wallpaper
   
   
   Map them to a design
   speci
   fi
   cation
   
   
   Propagate them to the apps
   Picture by Sagar Viradiya
   

   View Slide

10. siddroid
    siddroid
    Let’s step into the Android
    framework
    A ride through AOSP’s new colors!
    

    View Slide

11. siddroid
    siddroid
    Android framework components
    
    
    System UI
    

    ThemeOverlayController & ThemeOverlayApplier
    WallpaperManager & WallpaperManagerService
    Core
    

    WallpaperColors
    System UI
    

    Monet
    Graphics
    

    Palette
    Graphics
    

    Quantizers
    A ride through AOSP’s new colors!
    

    View Slide

12. siddroid
    siddroid
    Wallpaper Manager Service
    
    
    public class WallpaperManagerService extends IWallpaperManager.Stub
    
    
    implements IWallpaperManagerService
    
    
    class WallpaperObserver extends FileObserver
    
    
    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
    

    View Slide

13. siddroid
    siddroid
    Wallpaper Manager Service
    
    
    class WallpaperObserver extends FileObserver {
    
    
    mWallpaperFile = new File(mWallpaperDir, WALLPAPER);
    
    
    mWallpaperLockFile = new File(mWallpaperDir, WALLPAPER_LOCK_ORIG);
    
    
    final boolean sysWallpaperChanged =
    
    
    (mWallpaperFile.equals(changedFile));
    
    
    final boolean lockWallpaperChanged =
    
    
    (mWallpaperLockFile.equals(changedFile));
    
    
    }
    
    
    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
    

    View Slide

14. siddroid
    siddroid
    Wallpaper Manager Service
    
    
    class WallpaperObserver extends FileObserver {
    
    
    //
    If home wallpaper has changed
    
    
    notifyColorsWhich
    |=
    FLAG_SYSTEM;
    
    
    //
    If lockscreen wallpaper has changed
    
    
    notifyColorsWhich
    |=
    FLAG_LOCK;
    
    
    if (notifyColorsWhich
    !=
    0) {
    
    
    notifyWallpaperColorsChanged(wallpaper, notifyColorsWhich);
    
    
    }
    
    
    }
    
    
    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
    

    View Slide

15. siddroid
    siddroid
    Wallpaper Manager Service
    
    
    void notifyWallpaperColorsChanged(
    
    
    WallpaperData wallpaper,
    
    
    int which) {
    
    
    wallpaper.connection.forEachDisplayConnector(connector
    ->
    {
    
    
    notifyWallpaperColorsChangedOnDisplay(
    
    
    wallpaper,
    
    
    which,
    
    
    connector.mDisplayId
    
    
    );
    
    
    });
    
    
    }
    
    
    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
    

    View Slide

16. siddroid
    siddroid
    Monet & System-UI
    A ride through AOSP’s new colors!
    

    View Slide

17. siddroid
    siddroid
    System-UI & Feature flags
    
    
    //
    Flags.java
    
    
    public static final BooleanFlag MONET =
    
    
    new BooleanFlag(800, true, R.bool.flag_monet);
    
    
    //
    FeatureFlags.java
    
    
    public boolean isMonetEnabled() {
    
    
    return isEnabled(Flags.MONET);
    
    
    }
    
    
    //
    ThemeOverlayController.java
    
    
    mIsMonetEnabled = featureFlags.isMonetEnabled();
    
    
    frameworks/base/packages/SystemUI/src/com/android/systemui/
    fl
    ags/FeatureFlags.java & Flags.java
    
    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java

    

    View Slide

18. siddroid
    siddroid
    Theme Overlay Controller
    
    
    OnColorsChangedListener mOnColorsChangedListener
    
    
    = new OnColorsChangedListener() {
    
    
    @Override
    
    
    public void onColorsChanged(
    
    
    WallpaperColors wallpaperColors,
    
    
    int which,
    
    
    int userId) {
    
    
    //
    Post performing user Id specific logic
    
    
    handleWallpaperColors(wallpaperColors, which, userId);
    
    
    }
    
    
    };
    
    
    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
    

    View Slide

19. siddroid
    siddroid
    Wallpaper Manager Service
    
    
    private void notifyWallpaperColorsChangedOnDisplay(
    
    
    WallpaperData wallpaper,
    
    
    int which,
    
    
    int displayId) {
    
    
    boolean needsExtraction;
    
    
    needsExtraction = wallpaper.primaryColors
    ==
    null;
    
    
    if (needsExtraction) {
    
    
    extractColors(wallpaper);
    
    
    }
    
    
    notifyColorListeners(
    ...
    );
    
    
    }
    
    
    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
    

    View Slide

20. siddroid
    siddroid
    Wallpaper Manager Service
    
    
    private void extractColors(WallpaperData wallpaper) {
    
    
    WallpaperColors colors = null;
    
    
    if (cropFile
    !=
    null) {
    
    
    Bitmap bitmap = BitmapFactory.decodeFile(cropFile);
    
    
    if (bitmap
    !=
    null) {
    
    
    colors = WallpaperColors.fromBitmap(bitmap);
    
    
    }
    
    
    } else if (defaultImageWallpaper) {
    
    
    colors = extractDefaultImageWallpaperColors();
    
    
    }
    
    
    }
    
    
    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java
    

    View Slide

21. siddroid
    siddroid
    Extracting colors from the
    wallpaper
    A ride through AOSP’s new colors!
    

    View Slide

22. siddroid
    siddroid
    Wallpaper Colors
    
    
    public static WallpaperColors fromBitmap(@NonNull Bitmap bitmap) {
    
    
    final int bitmapArea = bitmap.getWidth() * bitmap.getHeight();
    
    
    if (bitmapArea > MAX_WALLPAPER_EXTRACTION_AREA) {
    
    
    Size optimalSize = calculateOptimalSize(
    
    
    bitmap.getWidth(),
    
    
    bitmap.getHeight());
    
    
    bitmap = Bitmap.createScaledBitmap(bitmap,
    
    
    optimalSize.getWidth(),
    
    
    optimalSize.getHeight(),
    
    
    false);
    
    
    }
    
    
    }
    
    
    frameworks/base/core/java/android/app/WallpaperColors.java
    

    View Slide

23. siddroid
    siddroid
    Wallpaper Colors
    
    
    if (ActivityManager.isLowRamDeviceStatic()) {
    
    
    palette = Palette
    
    
    .from(bitmap, new VariationalKMeansQuantizer())
    
    
    .maximumColorCount(5)
    
    
    .resizeBitmapArea(MAX_WALLPAPER_EXTRACTION_AREA)
    
    
    .generate();
    
    
    } else {
    
    
    palette = Palette
    
    
    .from(bitmap, new CelebiQuantizer())
    
    
    .maximumColorCount(128)
    
    
    .resizeBitmapArea(MAX_WALLPAPER_EXTRACTION_AREA)
    
    
    .generate();
    
    
    }
    
    
    frameworks/base/core/java/android/app/WallpaperColors.java
    

    View Slide

24. siddroid
    siddroid
    Dominant colors and Quantizers
    A ride through AOSP’s new colors!
    

    View Slide

25. siddroid
    siddroid
    Image quantization
    
    
    An image is a set of Pixels
    
    
    A quantization algorithm reduces
    the number of pixels
    
    
    Each pixel consists of a single
    color
    
    
    Android uses K-means clustering
    (Variational and Celebi)
    
    
    Device RAM determines which
    technique is used
    Pixelated image from cs.princeton.edu
    

    View Slide

26. siddroid
    siddroid
    Color science
    
    
    A color denotes a given value from
    the spectrum of light
    
    
    For example, RGB and CMYK are
    the most commonly used
    
    
    The HSL model uses hue,
    saturation and lightness of a color
    
    
    Android uses a custom HCT model
    for calculating colors
    

    View Slide

27. siddroid
    siddroid
    Material Design’s HCT model
    
    
    Stands for Hue, Chroma and Tone
    
    
    A blend of LCH and CAM16 color
    spaces
    
    
    Best of both color systems
    
    
    Better contrast ratio calculations
    with tone values
    
    
    Android uses this custom HCT
    model for calculating Material You
    theme colors via Monet
    From material.io “The science of color and design” by James O'Leary
    

    View Slide

28. siddroid
    siddroid
    Why Monet?
    
    
    Original Older Algorithm Celebi Quantizer
    From material.io “The science of color and design” by James O'Leary
    Claude Monet’s “Impression Sunrise”
    A ride through AOSP’s new colors!
    

    View Slide

29. siddroid
    siddroid
    Palette & Swatches
    
    
    Palette(List swatches) {
    
    
    mSwatches = swatches;
    
    
    mDominantSwatch = findDominantSwatch();
    
    
    }
    
    
    public Swatch(@ColorInt int colorInt, int population) {
    
    
    mColor = Color.valueOf(colorInt);
    
    
    mPopulation = population;
    
    
    }
    
    
    frameworks/base/core/java/com/android/internal/graphics/palette/Palette.java
    

    View Slide

30. siddroid
    siddroid
    Finding the seed color
    
    
    Swatches are
    fi
    rst sorted by
    population count
    
    
    Each color is assigned a “score”
    
    
    Colors are then sorted in
    descending order by the score
    
    
    Iterate through the scored colors
    and identify seed color
    
    
    Seed color is used to build the
    fi
    nal theme via Monet
    

    View Slide

31. siddroid
    siddroid
    Re-evaluating System Theme
    A ride through AOSP’s new colors!
    

    View Slide

32. siddroid
    siddroid
    When to re-evaluate?
    
    
    First boot of the device
    
    
    Via broadcast receiver when
    Intent.ACTION_WALLPAPER_CHANGED is
    received
    
    
    On a new user setup
    
    
    When the active user is changed
    private void reevaluateSystemTheme
    

    View Slide

33. siddroid
    siddroid
    Theme Overlay Controller
    
    
    private final BroadcastReceiver mBroadcastReceiver
    
    
    = new BroadcastReceiver() {
    
    
    @Override
    
    
    public void onReceive(Context context, Intent intent) {
    
    
    if (Intent.ACTION_WALLPAPER_CHANGED.equals(intent.getAction())) {
    
    
    .
    ..
    
    mAcceptColorEvents = true;
    
    
    }
    
    
    }
    
    
    };
    
    
    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
    

    View Slide

34. siddroid
    siddroid
    Runtime updates of System
    colors
    A ride through AOSP’s new colors!
    

    View Slide

35. siddroid
    siddroid
    Theme Overlay Controller
    
    
    protected @Nullable FabricatedOverlay getOverlay(int color, int type)
    
    
    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
    

    View Slide

36. siddroid
    siddroid
    Monet Color Scheme
    
    
    //
    Build Monet ColorScheme
    
    
    mColorScheme = new ColorScheme(color, nightMode);
    
    
    List colorShades = mColorScheme.getAllAccentColors();
    
    
    //
    OR
    
    
    List colorShades = mColorScheme.getAllNeutralColors();
    
    
    frameworks/base/packages/SystemUI/monet/src/com/android/systemui/monet/ColorScheme.kt
    
    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
    

    View Slide

37. siddroid
    siddroid
    Monet Color Scheme
    
    
    String name = type
    ==
    ACCENT ? "accent" : "neutral";
    
    
    for (int i = 0; i < colorShades.size(); i
    ++
    ) {
    
    
    int luminosity = i % paletteSize;
    
    
    int paletteIndex = i / paletteSize + 1;
    
    
    String resourceName;
    
    
    switch (luminosity) {
    
    
    case 0:
    
    
    resourceName = "android:color/system_" + name + paletteIndex + "_10";
    
    
    break;
    
    
    case 1:
    
    
    resourceName = "android:color/system_" + name + paletteIndex + "_50";
    
    
    break;
    
    
    default:
    
    
    int l = luminosity - 1;
    
    
    resourceName = "android:color/system_" + name + paletteIndex + "_" + l + "00";
    
    
    }
    
    
    }
    
    
    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java
    

    View Slide

38. siddroid
    siddroid
    System color resources
    
    
    public static final int system_accent3_600 = 17170521;
    
    
    public static final int system_accent3_700 = 17170522;
    
    
    public static final int system_accent3_800 = 17170523;
    
    
    public static final int system_accent3_900 = 17170524;
    
    
    public static final int system_neutral1_0 = 17170461;
    
    
    public static final int system_neutral1_10 = 17170462;
    
    
    public static final int system_neutral1_100 = 17170464;
    
    
    Android .R
    fi
    le
    

    View Slide

39. siddroid
    siddroid
    Theme Overlay Applier
    
    
    /*
    
    * All theme customization categories used by the system, in order that they should be applied,
    
    
    * starts with launcher and grouped by target package.
    
    
    */
    
    static final List THEME_CATEGORIES = Lists.newArrayList(
    
    
    OVERLAY_CATEGORY_SYSTEM_PALETTE,
    
    
    OVERLAY_CATEGORY_ICON_LAUNCHER,
    
    
    OVERLAY_CATEGORY_SHAPE,
    
    
    OVERLAY_CATEGORY_FONT,
    
    
    OVERLAY_CATEGORY_ACCENT_COLOR,
    
    
    OVERLAY_CATEGORY_ICON_ANDROID,
    
    
    OVERLAY_CATEGORY_ICON_SYSUI,
    
    
    OVERLAY_CATEGORY_ICON_SETTINGS,
    
    
    OVERLAY_CATEGORY_ICON_THEME_PICKER);
    
    
    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayApplier.java
    

    View Slide

40. siddroid
    siddroid
    Monet and Material 3 with
    compose
    A ride through AOSP’s new colors!
    

    View Slide

41. siddroid
    siddroid
    Tonal Palette
    
    
    internal class TonalPalette(
    
    
    //
    13 shades of each
    
    
    val neutral: Color,
    
    
    val neutralVariant: Color,
    
    
    val primary: Color,
    
    
    val secondary: Color,
    
    
    val tertiary: Color,
    
    
    )
    
    
    androidx.compose.material3.TonalPalette.kt
    

    View Slide

42. siddroid
    siddroid
    An image From material.io
    

    View Slide

43. siddroid
    siddroid
    Dynamic Tonal Palette
    
    
    internal fun dynamicTonalPalette(context: Context): TonalPalette = TonalPalette(
    
    
    neutral100 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_0),
    
    
    neutral99 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_10),
    
    
    neutral95 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_50),
    
    
    ..
    
    )
    
    
    androidx.compose.material3.DynamicTonalPalette.kt
    

    View Slide

44. siddroid
    siddroid
    Dynamic Tonal Palette
    
    
    @RequiresApi(23)
    
    
    private object ColorResourceHelper {
    
    
    @DoNotInline
    
    
    fun getColor(context: Context, @ColorRes id: Int): Color {
    
    
    return Color(context.resources.getColor(id, context.theme))
    
    
    }
    
    
    }
    
    
    androidx.compose.material3.DynamicTonalPalette.kt
    

    View Slide

45. siddroid
    siddroid
    Color Schemes
    
    
    fun dynamicLightColorScheme(context: Context): ColorScheme {
    
    
    val tonalPalette = dynamicTonalPalette(context)
    
    
    return lightColorScheme(
    
    
    primary = tonalPalette.primary40,
    
    
    onPrimary = tonalPalette.primary100,
    
    
    primaryContainer = tonalPalette.primary90,
    
    
    androidx.compose.material3.DynamicTonalPalette.kt
    

    View Slide

46. siddroid
    siddroid
    Material Color Utilities
    https://github.com/material-foundation/material-color-utilities
    

    View Slide

47. siddroid
    siddroid
    Material Color Utilities
    https://github.com/material-foundation/material-color-utilities
    

    View Slide

48. siddroid
    siddroid
    A promo From material.io
    

    View Slide

49. siddroid
    siddroid
    Shout out to!
    
    
    The Android and Material design
    teams at Google!
    
    
    James O’leary for his work on
    color science behind Material You
    
    
    material.io & Guru Singh for his
    help with my questions!
    
    
    Android code search
    A video of Monet in Action from Material foundation
    

    View Slide

50. siddroid
    siddroid
    Further reading
    
    
    

    View Slide

51. siddroid
    siddroid
    Danke Schone!
    Droidcon Berlin, July 2022
    

    View Slide