Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Droidcon Berlin - A ride through AOSP's new colors

Sid Patil
July 11, 2022

Droidcon Berlin - A ride through AOSP's new colors

Dynamic theming on the Pixel is an amazing feature, but we as Android engineers always care about how it works under the hood!

In this talk, we will understand “Monet”, the wallpaper-based dynamic theming system.

We will see:

What is Monet?
Dynamic behavior of system-wide theming
Wallpaper color extraction
K-means and Celebi quantizers
Color extraction strategy
Runtime dynamic color updation
Dynamic themes with M3 components for Jetpack Compose
In this session, we will aim to understand how the Android operating system works to enable dynamic theming (basics fundamentals of Material You). We will deep dive into AOSP to examine parts of the framework that form Monet and ultimately understand how dynamic system colors work.

Join me on the colourful ride!

Sid Patil

July 11, 2022
Tweet

Other Decks in Science

Transcript

  1. siddroid
    Droidcon Berlin, July 2022
    A ride through AOSP’s new colors
    Sid Patil

    View full-size slide

  2. siddroid
    siddroid
    About Me


    Android Engineer at Delivery Hero
    based in Berlin


    Built Paytm, a mobile payments
    platform in India, ~ 100m users


    Organizes and hosts communities
    like Kotlin Mumbai & Android
    Worldwide


    Know more at siddroid.com
    A ride through AOSP’s new colors!

    View full-size slide

  3. siddroid
    siddroid
    Questions?


    Do you recognise this logo?


    How many of you use a Google
    Pixel?


    Have you heard about Monet?


    Do you use Material 3 in your
    apps?
    M3 logo from material.io

    View full-size slide

  4. siddroid
    siddroid
    Material You, the face of the
    Android OS
    A ride through AOSP’s new colors

    View full-size slide

  5. siddroid
    siddroid

    View full-size slide

  6. siddroid
    siddroid
    .. the current face of Android
    A ride through AOSP’s new colors!

    View full-size slide

  7. siddroid
    siddroid
    A promo From material.io

    View full-size slide

  8. siddroid
    siddroid
    What is Monet?


    The dynamic theme engine on
    Android


    Enables features like dynamic
    color and themed icons on
    Android 12 and 13


    First introduced with Android 12
    and Google Pixel devices


    Fully added in AOSP from Android
    12L
    A promo From material.io

    View full-size slide

  9. siddroid
    siddroid
    How would you build it?


    Read and observe changes in the
    wallpaper


    Make sense of information from
    the device wallpaper


    Extract dominant colors of the
    wallpaper


    Map them to a design
    speci
    fi
    cation


    Propagate them to the apps
    Picture by Sagar Viradiya

    View full-size slide

  10. siddroid
    siddroid
    Let’s step into the Android
    framework
    A ride through AOSP’s new colors!

    View full-size slide

  11. siddroid
    siddroid
    Android framework components


    System UI

    ThemeOverlayController & ThemeOverlayApplier
    WallpaperManager & WallpaperManagerService
    Core

    WallpaperColors
    System UI

    Monet
    Graphics

    Palette
    Graphics

    Quantizers
    A ride through AOSP’s new colors!

    View full-size slide

  12. siddroid
    siddroid
    Wallpaper Manager Service


    public class WallpaperManagerService extends IWallpaperManager.Stub


    implements IWallpaperManagerService


    class WallpaperObserver extends FileObserver


    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java

    View full-size slide

  13. siddroid
    siddroid
    Wallpaper Manager Service


    class WallpaperObserver extends FileObserver {


    mWallpaperFile = new File(mWallpaperDir, WALLPAPER);


    mWallpaperLockFile = new File(mWallpaperDir, WALLPAPER_LOCK_ORIG);


    final boolean sysWallpaperChanged =


    (mWallpaperFile.equals(changedFile));


    final boolean lockWallpaperChanged =


    (mWallpaperLockFile.equals(changedFile));


    }


    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java

    View full-size slide

  14. siddroid
    siddroid
    Wallpaper Manager Service


    class WallpaperObserver extends FileObserver {


    //
    If home wallpaper has changed


    notifyColorsWhich
    |=
    FLAG_SYSTEM;


    //
    If lockscreen wallpaper has changed


    notifyColorsWhich
    |=
    FLAG_LOCK;


    if (notifyColorsWhich
    !=
    0) {


    notifyWallpaperColorsChanged(wallpaper, notifyColorsWhich);


    }


    }


    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java

    View full-size slide

  15. siddroid
    siddroid
    Wallpaper Manager Service


    void notifyWallpaperColorsChanged(


    WallpaperData wallpaper,


    int which) {


    wallpaper.connection.forEachDisplayConnector(connector
    ->
    {


    notifyWallpaperColorsChangedOnDisplay(


    wallpaper,


    which,


    connector.mDisplayId


    );


    });


    }


    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java

    View full-size slide

  16. siddroid
    siddroid
    Monet & System-UI
    A ride through AOSP’s new colors!

    View full-size slide

  17. siddroid
    siddroid
    System-UI & Feature flags


    //
    Flags.java


    public static final BooleanFlag MONET =


    new BooleanFlag(800, true, R.bool.flag_monet);


    //
    FeatureFlags.java


    public boolean isMonetEnabled() {


    return isEnabled(Flags.MONET);


    }


    //
    ThemeOverlayController.java


    mIsMonetEnabled = featureFlags.isMonetEnabled();


    frameworks/base/packages/SystemUI/src/com/android/systemui/
    fl
    ags/FeatureFlags.java & Flags.java

    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java


    View full-size slide

  18. siddroid
    siddroid
    Theme Overlay Controller


    OnColorsChangedListener mOnColorsChangedListener


    = new OnColorsChangedListener() {


    @Override


    public void onColorsChanged(


    WallpaperColors wallpaperColors,


    int which,


    int userId) {


    //
    Post performing user Id specific logic


    handleWallpaperColors(wallpaperColors, which, userId);


    }


    };


    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java

    View full-size slide

  19. siddroid
    siddroid
    Wallpaper Manager Service


    private void notifyWallpaperColorsChangedOnDisplay(


    WallpaperData wallpaper,


    int which,


    int displayId) {


    boolean needsExtraction;


    needsExtraction = wallpaper.primaryColors
    ==
    null;


    if (needsExtraction) {


    extractColors(wallpaper);


    }


    notifyColorListeners(
    ...
    );


    }


    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java

    View full-size slide

  20. siddroid
    siddroid
    Wallpaper Manager Service


    private void extractColors(WallpaperData wallpaper) {


    WallpaperColors colors = null;


    if (cropFile
    !=
    null) {


    Bitmap bitmap = BitmapFactory.decodeFile(cropFile);


    if (bitmap
    !=
    null) {


    colors = WallpaperColors.fromBitmap(bitmap);


    }


    } else if (defaultImageWallpaper) {


    colors = extractDefaultImageWallpaperColors();


    }


    }


    frameworks/base/services/core/java/com/android/server/wallpaper/WallpaperManagerService.java

    View full-size slide

  21. siddroid
    siddroid
    Extracting colors from the
    wallpaper
    A ride through AOSP’s new colors!

    View full-size slide

  22. siddroid
    siddroid
    Wallpaper Colors


    public static WallpaperColors fromBitmap(@NonNull Bitmap bitmap) {


    final int bitmapArea = bitmap.getWidth() * bitmap.getHeight();


    if (bitmapArea > MAX_WALLPAPER_EXTRACTION_AREA) {


    Size optimalSize = calculateOptimalSize(


    bitmap.getWidth(),


    bitmap.getHeight());


    bitmap = Bitmap.createScaledBitmap(bitmap,


    optimalSize.getWidth(),


    optimalSize.getHeight(),


    false);


    }


    }


    frameworks/base/core/java/android/app/WallpaperColors.java

    View full-size slide

  23. siddroid
    siddroid
    Wallpaper Colors


    if (ActivityManager.isLowRamDeviceStatic()) {


    palette = Palette


    .from(bitmap, new VariationalKMeansQuantizer())


    .maximumColorCount(5)


    .resizeBitmapArea(MAX_WALLPAPER_EXTRACTION_AREA)


    .generate();


    } else {


    palette = Palette


    .from(bitmap, new CelebiQuantizer())


    .maximumColorCount(128)


    .resizeBitmapArea(MAX_WALLPAPER_EXTRACTION_AREA)


    .generate();


    }


    frameworks/base/core/java/android/app/WallpaperColors.java

    View full-size slide

  24. siddroid
    siddroid
    Dominant colors and Quantizers
    A ride through AOSP’s new colors!

    View full-size slide

  25. siddroid
    siddroid
    Image quantization


    An image is a set of Pixels


    A quantization algorithm reduces
    the number of pixels


    Each pixel consists of a single
    color


    Android uses K-means clustering
    (Variational and Celebi)


    Device RAM determines which
    technique is used
    Pixelated image from cs.princeton.edu

    View full-size slide

  26. siddroid
    siddroid
    Color science


    A color denotes a given value from
    the spectrum of light


    For example, RGB and CMYK are
    the most commonly used


    The HSL model uses hue,
    saturation and lightness of a color


    Android uses a custom HCT model
    for calculating colors

    View full-size slide

  27. siddroid
    siddroid
    Material Design’s HCT model


    Stands for Hue, Chroma and Tone


    A blend of LCH and CAM16 color
    spaces


    Best of both color systems


    Better contrast ratio calculations
    with tone values


    Android uses this custom HCT
    model for calculating Material You
    theme colors via Monet
    From material.io “The science of color and design” by James O'Leary

    View full-size slide

  28. siddroid
    siddroid
    Why Monet?


    Original Older Algorithm Celebi Quantizer
    From material.io “The science of color and design” by James O'Leary
    Claude Monet’s “Impression Sunrise”
    A ride through AOSP’s new colors!

    View full-size slide

  29. siddroid
    siddroid
    Palette & Swatches


    Palette(List swatches) {


    mSwatches = swatches;


    mDominantSwatch = findDominantSwatch();


    }


    public Swatch(@ColorInt int colorInt, int population) {


    mColor = Color.valueOf(colorInt);


    mPopulation = population;


    }


    frameworks/base/core/java/com/android/internal/graphics/palette/Palette.java

    View full-size slide

  30. siddroid
    siddroid
    Finding the seed color


    Swatches are
    fi
    rst sorted by
    population count


    Each color is assigned a “score”


    Colors are then sorted in
    descending order by the score


    Iterate through the scored colors
    and identify seed color


    Seed color is used to build the
    fi
    nal theme via Monet

    View full-size slide

  31. siddroid
    siddroid
    Re-evaluating System Theme
    A ride through AOSP’s new colors!

    View full-size slide

  32. siddroid
    siddroid
    When to re-evaluate?


    First boot of the device


    Via broadcast receiver when
    Intent.ACTION_WALLPAPER_CHANGED is
    received


    On a new user setup


    When the active user is changed
    private void reevaluateSystemTheme

    View full-size slide

  33. siddroid
    siddroid
    Theme Overlay Controller


    private final BroadcastReceiver mBroadcastReceiver


    = new BroadcastReceiver() {


    @Override


    public void onReceive(Context context, Intent intent) {


    if (Intent.ACTION_WALLPAPER_CHANGED.equals(intent.getAction())) {


    .
    ..

    mAcceptColorEvents = true;


    }


    }


    };


    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java

    View full-size slide

  34. siddroid
    siddroid
    Runtime updates of System
    colors
    A ride through AOSP’s new colors!

    View full-size slide

  35. siddroid
    siddroid
    Theme Overlay Controller


    protected @Nullable FabricatedOverlay getOverlay(int color, int type)


    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java

    View full-size slide

  36. siddroid
    siddroid
    Monet Color Scheme


    //
    Build Monet ColorScheme


    mColorScheme = new ColorScheme(color, nightMode);


    List colorShades = mColorScheme.getAllAccentColors();


    //
    OR


    List colorShades = mColorScheme.getAllNeutralColors();


    frameworks/base/packages/SystemUI/monet/src/com/android/systemui/monet/ColorScheme.kt

    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java

    View full-size slide

  37. siddroid
    siddroid
    Monet Color Scheme


    String name = type
    ==
    ACCENT ? "accent" : "neutral";


    for (int i = 0; i < colorShades.size(); i
    ++
    ) {


    int luminosity = i % paletteSize;


    int paletteIndex = i / paletteSize + 1;


    String resourceName;


    switch (luminosity) {


    case 0:


    resourceName = "android:color/system_" + name + paletteIndex + "_10";


    break;


    case 1:


    resourceName = "android:color/system_" + name + paletteIndex + "_50";


    break;


    default:


    int l = luminosity - 1;


    resourceName = "android:color/system_" + name + paletteIndex + "_" + l + "00";


    }


    }


    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayController.java

    View full-size slide

  38. siddroid
    siddroid
    System color resources


    public static final int system_accent3_600 = 17170521;


    public static final int system_accent3_700 = 17170522;


    public static final int system_accent3_800 = 17170523;


    public static final int system_accent3_900 = 17170524;


    public static final int system_neutral1_0 = 17170461;


    public static final int system_neutral1_10 = 17170462;


    public static final int system_neutral1_100 = 17170464;


    Android .R
    fi
    le

    View full-size slide

  39. siddroid
    siddroid
    Theme Overlay Applier


    /*

    * All theme customization categories used by the system, in order that they should be applied,


    * starts with launcher and grouped by target package.


    */

    static final List THEME_CATEGORIES = Lists.newArrayList(


    OVERLAY_CATEGORY_SYSTEM_PALETTE,


    OVERLAY_CATEGORY_ICON_LAUNCHER,


    OVERLAY_CATEGORY_SHAPE,


    OVERLAY_CATEGORY_FONT,


    OVERLAY_CATEGORY_ACCENT_COLOR,


    OVERLAY_CATEGORY_ICON_ANDROID,


    OVERLAY_CATEGORY_ICON_SYSUI,


    OVERLAY_CATEGORY_ICON_SETTINGS,


    OVERLAY_CATEGORY_ICON_THEME_PICKER);


    frameworks/base/packages/SystemUI/src/com/android/systemui/theme/ThemeOverlayApplier.java

    View full-size slide

  40. siddroid
    siddroid
    Monet and Material 3 with
    compose
    A ride through AOSP’s new colors!

    View full-size slide

  41. siddroid
    siddroid
    Tonal Palette


    internal class TonalPalette(


    //
    13 shades of each


    val neutral: Color,


    val neutralVariant: Color,


    val primary: Color,


    val secondary: Color,


    val tertiary: Color,


    )


    androidx.compose.material3.TonalPalette.kt

    View full-size slide

  42. siddroid
    siddroid
    An image From material.io

    View full-size slide

  43. siddroid
    siddroid
    Dynamic Tonal Palette


    internal fun dynamicTonalPalette(context: Context): TonalPalette = TonalPalette(


    neutral100 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_0),


    neutral99 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_10),


    neutral95 = ColorResourceHelper.getColor(context, android.R.color.system_neutral1_50),


    ..

    )


    androidx.compose.material3.DynamicTonalPalette.kt

    View full-size slide

  44. siddroid
    siddroid
    Dynamic Tonal Palette


    @RequiresApi(23)


    private object ColorResourceHelper {


    @DoNotInline


    fun getColor(context: Context, @ColorRes id: Int): Color {


    return Color(context.resources.getColor(id, context.theme))


    }


    }


    androidx.compose.material3.DynamicTonalPalette.kt

    View full-size slide

  45. siddroid
    siddroid
    Color Schemes


    fun dynamicLightColorScheme(context: Context): ColorScheme {


    val tonalPalette = dynamicTonalPalette(context)


    return lightColorScheme(


    primary = tonalPalette.primary40,


    onPrimary = tonalPalette.primary100,


    primaryContainer = tonalPalette.primary90,


    androidx.compose.material3.DynamicTonalPalette.kt

    View full-size slide

  46. siddroid
    siddroid
    Material Color Utilities
    https://github.com/material-foundation/material-color-utilities

    View full-size slide

  47. siddroid
    siddroid
    Material Color Utilities
    https://github.com/material-foundation/material-color-utilities

    View full-size slide

  48. siddroid
    siddroid
    A promo From material.io

    View full-size slide

  49. siddroid
    siddroid
    Shout out to!


    The Android and Material design
    teams at Google!


    James O’leary for his work on
    color science behind Material You


    material.io & Guru Singh for his
    help with my questions!


    Android code search
    A video of Monet in Action from Material foundation

    View full-size slide

  50. siddroid
    siddroid
    Further reading


    View full-size slide

  51. siddroid
    siddroid
    Danke Schone!
    Droidcon Berlin, July 2022

    View full-size slide