Custom Codecs with KotlinX Serialization

Custom Codecs with KotlinX Serialization Marcello Galhardo https://marcellogalhardo.dev

Before we start • This is a niche talk. •
Targeted at library authors. • …or enthusiasts curious about how SavedState and Nav3 Serialization works.

Table of contents SavedState Serialization 01 Building a Codec 02
Nav3 Serialization 03 Problems, lessons, and takeaways 04

SavedState Serialization

Process Death

SavedState • AndroidX solution for saving and restoring state. •
Works with Activities, ViewModels, Compose, and more. • Uses a Bundle on Android and a Map on other KMP platforms.

// commonMain expect class SavedState // androidMain actual typealias SavedState
= android.os.Bundle // nonAndroidMain actual class SavedState(val map: MutableMap<String, Any?>)

// commonMain public inline fun <T> SavedState.read( block: SavedStateReader.() ->
T, ): T = block(SavedStateReader(source = this)) public inline fun <T> SavedState.write( block: SavedStateWriter.() -> T ): T = block(SavedStateWriter(source = this)

public expect value class SavedStateReader( val source: SavedState ) {
public fun getBoolean(key: String): Boolean public fun getChar(key: String): Char public fun getCharSequence(key: String): CharSequence // .. }

public expect value class SavedStateWriter( val source: SavedState ) {
public fun putBoolean(key: String, value: Boolean) public fun putChar(key: String, value: Char) public fun putCharSequence(key: String, value: CharSequence) // .. }

Multiplatform Serialization

KTX Serialization

Building a Codec

Goals • Build an Encoder and Decoder for SavedState.

Goals • Build an Encoder and Decoder for SavedState. •
Always restore the same state.

Goals • Build an Encoder and Decoder for SavedState. •
Always restore the same state. • Serializable data classes can be "ported" to any platform and/or format.

@Serializable data class User(name: String, age: Int)

Encoder • Ask the serializer to encode the value.

Encoder • Ask the serializer to encode the value. •
For objects, it calls: ◦ beginStructure(descriptor) → ◦ then field by field: encodeElement for index 0, 1, 2… → ◦ finish with endStructure().

Encoder • Ask the serializer to encode the value. •
For objects, it calls: ◦ beginStructure(descriptor) → ◦ then field by field: encodeElement for index 0, 1, 2… → ◦ finish with endStructure(). • For lists/maps, it uses the same idea, but indexes are 0..n-1 ◦ (and for maps, even/odd indexes are key/value).

internal class SavedStateEncoder

internal class SavedStateEncoder : Encoder, CompositeEncoder

internal class SavedStateEncoder : Encoder, CompositeEncoder { override val serializersModule:
SerializersModule override fun beginStructure(..): CompositeEncoder override fun endStructure(..) override fun encodeBooleanElement(..) override fun encodeByteElement(..) override fun encodeShortElement(..) override fun encodeCharElement(..) override fun encodeIntElement(..) override fun encodeLongElement(..) override fun encodeFloatElement(..) override fun encodeDoubleElement(..) override fun encodeStringElement(..) override fun encodeInlineElement(..) override fun <T> encodeSerializableElement(..) override fun <T> encodeNullableSerializableElement(..) override fun encodeNull() override fun encodeBoolean(..) override fun encodeByte(..) override fun encodeShort(..) override fun encodeChar(..) override fun encodeInt(..) override fun encodeLong(..) override fun encodeFloat(..) override fun encodeDouble(..) override fun encodeString(..) override fun encodeEnum(..) override fun encodeInline(..): Encoder }

internal class SavedStateEncoder : AbstractEncoder()

internal class SavedStateEncoder : AbstractEncoder() { override val serializersModule: SerializersModule
}

internal class SavedStateEncoder( internal val savedState: SavedState, private val configuration:
SavedStateConfiguration, ) : AbstractEncoder() { override val serializersModule get() = configuration.serializersModule }

SavedStateConfiguration, ) : AbstractEncoder() { override val serializersModule get() = configuration.serializersModule } public class SavedStateConfiguration( val serializersModule: SerializersModule = DEFAULT_SERIALIZERS_MODULE, val classDiscriminatorMode: Int = ClassDiscriminatorMode.POLYMORPHIC, val encodeDefaults: Boolean = false, )

SavedStateConfiguration, ) : AbstractEncoder() { override val serializersModule get() = configuration.serializersModule } val DEFAULT_SERIALIZERS_MODULE: SerializersModule = SerializersModule { contextual(SavedStateSerializer) contextual(MutableStateFlow::class) { elementSerializers -> MutableStateFlowSerializer(elementSerializers.first()) } contextual(SizeSerializer) contextual(SizeFSerializer) // .. }

override fun beginStructure(descriptor: SerialDescriptor): CompositeEncoder{ // We flatten single structured
object at root to prevent encoding to a // SavedState containing only one SavedState inside. // For example, a `Pair(3, 5)` would become `{"first" = 3, "second" = 5}`. return if (key == "") { this } else { val childState = savedState() // Link child to parent. savedState.write { putSavedState(key, childState) } SavedStateEncoder(configuration, childState) } }

var key: String = "" override fun encodeElement( descriptor: SerialDescriptor,
index: Int, ): Boolean { key = descriptor.getElementName(index) // .. return true }

override fun encodeBoolean(value: Boolean): Unit = savedState.write { putBoolean(key, value)
} override fun encodeInt(value: Int): Unit = savedState.write { putInt(key, value) } override fun encodeLong(value: Long): Unit = savedState.write { putLong(key, value) } override fun encodeFloat(value: Float): Unit = savedState.write { putFloat(key, value) } // ..

override fun <T> encodeSerializableValue(serializer: SerializationStrategy<T>, value: T) { // First,
try any platform-specific types val platformEncoded = encodeFormatSpecificTypesOnPlatform(serializer, value) // Platform encoder handled it, we're done. if (platformEncoded) return // If platform encoding didn't handle it, try our known fast-path types. when (serializer.descriptor) { Descriptors.IntList -> savedState.write { putIntList(key, value as List<Int>) } Descriptors.StringList -> savedState.write { putStringList(key, value as List<String>) } // .. else -> // This isn't a type we can specially handle. // Fall back to the default serialization behavior. super.encodeSerializableValue(serializer, value) } }

override fun <T> decodeSerializableValue(deserializer: DeserializationStrategy<T>): T { // First, try
any platform-specific types val platformDecoded = decodeFormatSpecificTypesOnPlatform(deserializer) // Platform decoder handled it, we're done. if (platformDecoded != null) return platformDecoded as T // If platform decoding didn't handle it, try our known fast-path types. return when (deserializer.descriptor) { Descriptors.IntList -> savedState.read { getIntList(key) } Descriptors.StringList -> savedState.read { getStringList(key) } // .. else -> // This isn't a type we can specially handle. // Fall back to the default deserialization behavior. super.decodeSerializableValue(deserializer) } as T? } internal fun <T> SavedStateEncoder.encodeFormatSpecificTypesOnPlatform( strategy: SerializationStrategy<T>, value: T, ): Boolean { when (strategy.descriptor) { Descriptors.Parcelable -> ParcelableSerializer().serialize(this, value as Parcelable) Descriptors.JavaSerializable -> JavaSerializableSerializer().serialize(this, value as JavaSerializable) // .. else -> return false } return true }

override fun <T> encodeSerializableValue(serializer: SerializationStrategy<T>, value: T) { // First,
try any platform-specific types val platformEncoded = encodeFormatSpecificTypesOnPlatform(serializer, value) // Platform encoder handled it, we're done. if (platformEncoded) return // If platform encoding didn't handle it, try our known fast-path types. when (serializer.descriptor) { Descriptors.IntList -> savedState.write { putIntList(key, value as List<Int>) } Descriptors.StringList -> savedState.write { putStringList(key, value as List<String>) } // .. else -> // This isn't a type we can specially handle. // Fall back to the default serialization behavior. super.encodeSerializableValue(serializer, value) } }

Decoder • Ask the serializer to decode the value.

Decoder • Ask the serializer to decode the value. •
For objects/lists/maps: ◦ beginStructure(descriptor) → ◦ loop calling decodeElementIndex(); for each index, call the right decodeXxx → ◦ finish with endStructure().

internal class SavedStateDecoder( internal val savedState: SavedState, private val configuration:
SavedStateConfiguration, ) : AbstractDecoder() { override val serializersModule get() = configuration.serializersModule }

override fun beginStructure(descriptor: SerialDescriptor): CompositeDecoder { // We flatten single
structured object at root to prevent encoding to a // SavedState containing only one SavedState inside. // For example, a `Pair(3, 5)` would become `{"first" = 3, "second" = 5}`. return if (key == "") { this } else { SavedStateDecoder( savedState = savedState.read { getSavedState(key) }, configuration = configuration, ) } }

override fun decodeElementIndex(descriptor: SerialDescriptor): Int { // Get iteration boundary.
For collections, it's the saved size. // For classes, it's all schema fields, as optional ones might be missing. val elementCount = if (descriptor.kind == StructureKind.LIST || descriptor.kind == StructureKind.MAP) { savedState.read { size() } } else { descriptor.elementsCount } // Find the next element present in the saved state, skipping // omitted optional fields. 'index' is a class property. while (index < elementCount) { val elementName = descriptor.getElementName(index) // Skip optional fields that aren't in the saved state // (they will use their default value). if (descriptor.isElementOptional(index) && savedState.read { elementName !in this }) { index++ continue } // This element is present or non-optional. // Set 'key' so subsequent decode* calls know what to read from the state. key = elementName // Return current index; increment 'index' for the next call. return index++ } // All elements processed. return CompositeDecoder.DECODE_DONE }

override fun decodeBoolean(): Boolean = savedState.read { getBoolean(key) } override
fun decodeInt(): Int = savedState.read { getInt(key) } override fun decodeLong(): Long = savedState.read { getLong(key) } override fun decodeFloat(): Float = savedState.read { getFloat(key) } // ..

any platform-specific types val platformDecoded = decodeFormatSpecificTypesOnPlatform(deserializer) // Platform decoder handled it, we're done. if (platformDecoded != null) return platformDecoded as T // If platform decoding didn't handle it, try our known fast-path types. return when (deserializer.descriptor) { Descriptors.IntList -> savedState.read { getIntList(key) } Descriptors.StringList -> savedState.read { getStringList(key) } // .. else -> // This isn't a type we can specially handle. // Fall back to the default deserialization behavior. super.decodeSerializableValue(deserializer) } as T? }

any platform-specific types val platformDecoded = decodeFormatSpecificTypesOnPlatform(deserializer) // Platform decoder handled it, we're done. if (platformDecoded != null) return platformDecoded as T // If platform decoding didn't handle it, try our known fast-path types. return when (deserializer.descriptor) { Descriptors.IntList -> savedState.read { getIntList(key) } Descriptors.StringList -> savedState.read { getStringList(key) } // .. else -> // This isn't a type we can specially handle. // Fall back to the default deserialization behavior. super.decodeSerializableValue(deserializer) } as T? } internal fun <T> SavedStateDecoder.decodeFormatSpecificTypesOnPlatform( strategy: DeserializationStrategy<T> ): T? { return when (strategy.descriptor) { Descriptors.Parcelable -> ParcelableSerializer().deserialize(this) Descriptors.JavaSerializable -> JavaSerializableSerializer().deserialize(this) // .. else -> null } as T? }

any platform-specific types val platformDecoded = decodeFormatSpecificTypesOnPlatform(deserializer) // Platform decoder handled it, we're done. if (platformDecoded != null) return platformDecoded as T // If platform decoding didn't handle it, try our known fast-path types. return when (deserializer.descriptor) { Descriptors.IntList -> savedState.read { getIntList(key) } Descriptors.StringList -> savedState.read { getStringList(key) } // .. else -> // This isn't a type we can specially handle. // Fall back to the default deserialization behavior. super.decodeSerializableValue(deserializer) } as T? }

Round Trip: encode → store → decode → same object

• Encode the object with the same serializers/module'

• Encode the object with the same serializers/module' • Store in a SavedState

• Encode the object with the same serializers/module' • Store in a SavedState • Decode back → equal to original

@Serializable data class User(name: String, age: Int)

@Serializable data class User(name: String, age: Int) val user =
User("John Doe", 37)

User("John Doe", 37) val encoded = encodeToSavedState(user)

User("John Doe", 37) val encoded = encodeToSavedState(user) val decoded = decodeFromSavedState(encoded) assertThat(user)isEqualTo(decoded) public fun <T : Any> encodeToSavedState( serializer: SerializationStrategy<T>, value: T, configuration: SavedStateConfiguration, ): SavedState { val result = savedState() SavedStateEncoder(result, configuration).encodeSerializableValue(serializer, value) return result }

Encoding flow • beginStructure(descriptor) • For name: String → encodeElement(descriptor,
index = 0) → encodeString("John Doe") • For age: Int → encodeElement(descriptor, index = 1) → encodeInt(37) • endStructure(descriptor)

User("John Doe", 37) val encoded = encodeToSavedState(user) val decoded = decodeFromSavedState(encoded)

User("John Doe", 37) val encoded = encodeToSavedState(user) val decoded = decodeFromSavedState(encoded) assertThat(user)isEqualTo(decoded) public fun <T : Any> decodeFromSavedState( deserializer: DeserializationStrategy<T>, savedState: SavedState, configuration: SavedStateConfiguration, ): T { return SavedStateDecoder(savedState, configuration).decodeSerializableValue(deserializer) }

Decoding flow • beginStructure(descriptor) • decodeElementIndex(descriptor) → returns 0 →
decodeString() → "John Doe" • decodeElementIndex(descriptor) → returns 1 → decodeInt() → 37 • decodeElementIndex(descriptor) → returns DECODE_DONE • endStructure(descriptor)

User("John Doe", 37) val encoded = encodeToSavedState(user) val decoded = decodeFromSavedState(encoded) check(user == decoded)

Should you build a codec?

Should you build a codec? • Probably not, unless you
need a custom format (CSV, binary, domain-specific).

Should you build a codec? • Probably not, unless you
need a custom format (CSV, binary, domain-specific). • In most cases, someone already built it → check the supported formats: ◦ github.com/Kotlin/kotlinx.serialization/blob/master/formats/README.md

Nav3 and Serialization

Nav3 • A BackStack is just a List<T : Any>.

• You own it and it is fully under your control.

• You own it and it is fully under your control. • You can choose how to implement and save it with custom types and codecs.

// Define routes and arguments data object Home data class
Product(val id: String) // BackStack with initial route val backStack = mutableStateListOf<Any>(Home)

// Define routes and arguments data object Home data class
Product(val id: String) // BackStack with initial route val backStack = remember { mutableStateListOf<Any>(Home) }

// Define routes and arguments @Serializable data object Home @Serializable
data class Product(val id: String) // BackStack with initial route val backStack = rememberSerializable { mutableStateListOf<Any>(Home) }

data class Product(val id: String) // BackStack with initial route val backStack = rememberSerializable { mutableStateListOf<Any>(Home) } @Composable public fun <T : Any> rememberSerializable( vararg inputs: Any?, serializer: KSerializer<T>, configuration: SavedStateConfiguration = DEFAULT, init: () -> T, ): T { val saver = serializableSaver(serializer, configuration) @Suppress("DEPRECATION") return rememberSaveable(*inputs, saver = saver, key = null, init = init) }

data class Product(val id: String) // BackStack with initial route val backStack = rememberSerializable { mutableStateListOf<Any>(Home) } internal fun <Serializable : Any> serializableSaver( serializer: KSerializer<Serializable>, configuration: SavedStateConfiguration = SavedStateConfiguration.DEFAULT, ): Saver<Serializable, SavedState> { return Saver( save = { original -> encodeToSavedState(serializer, original, configuration) }, restore = { savedState -> decodeFromSavedState(serializer, savedState, configuration) }, ) }

data class Product(val id: String) // BackStack with initial route val backStack = rememberSerializable { mutableStateListOf<Any>(Home) }

Open Polymorphic Serialization • You can mark a base class
and its subclasses with @Serializable.

and its subclasses with @Serializable. • KotlinX adds a type field so the decoder knows which subclass to restore.

and its subclasses with @Serializable. • KotlinX adds a type field so the decoder knows which subclass to restore. • You register ALL KNOWN subclasses in a SerializersModule. ◦ If you forget one, it will crash at runtime.

data class Product(val id: String) // BackStack with initial route val backStack = remember { mutableStateListOf<Any>(Home) }

data class Product(val id: String) // Define serializers module val configuration = SavedStateConfiguration { serializersModule = SerializersModule { polymorphic(Any::class) { subclass(Product::class) // ..other 100-ish classes. } } } // BackStack with initial route val backStack = rememberSerializable(configuration) { mutableStateListOf<Any>(Home) }

data class Product(val id: String) // Define serializers module val configuration = SavedStateConfiguration { serializersModule = SerializersModule { polymorphic(Any::class) { subclass(Product::class) // ..other 100-ish classes. } } } // BackStack with initial route val backStack = rememberSerializable(PolymorphicSerializer(Any::class), configuration) { mutableStateListOf<Any>(Home) }

NavKey

/** * Marker interface for keys. * * Objects and
classes that extend this class must be marked with the [Serializable] annotation in * order to be saved with by the [rememberNavBackStack] function. * * This class is required because [Serializable] is only an annotation and does not provide a way to * link classes marked with the annotation together and provide a serializable that works with all * of them, resulting it making it impossible to properly save and restore. */ public interface NavKey

data class Product(val id: String) // Define serializers module val configuration = SavedStateConfiguration { serializersModule = SerializersModule { polymorphic(Any::class) { subclass(Product::class) // ..other 100-ish classes. } } } // BackStack with initial route val backStack = rememberSerializable(PolymorphicSerializer(Any::class), configuration) { mutableStateListOf<Any>(Home) }

// Define routes and arguments @Serializable data object Home :
NavKey @Serializable data class Product(val id: String) : NavKey // Define serializers module val configuration = SavedStateConfiguration { serializersModule = SerializersModule { polymorphic(NavKey::class) { subclass(Product::class) // ..other 100-ish classes. } } } // BackStack with initial route val backStack = rememberSerializable(configuration){ mutableStateListOf<NavKey>(Home) }

NavKey @Serializable data class Product(val id: String) : NavKey // Define serializers module val configuration = SavedStateConfiguration { serializersModule = SerializersModule { polymorphic(NavKey::class) { subclass(Product::class) // ..other 100-ish classes. } } } // BackStack with initial route val backStack = rememberSerializable(configuration) { mutableStateListOf<NavKey>(Home) }

rememberNavBackStack

@Composable public fun rememberNavBackStack( configuration: SavedStateConfiguration, vararg elements: NavKey, ):
NavBackStack<NavKey> { return rememberSerializable( configuration = configuration, serializer = NavBackStackSerializer(PolymorphicSerializer(NavKey::class)), ) { NavBackStack(*elements) } }

NavKey @Serializable data class Product(val id: String) : NavKey // Define serializers module val configuration = SavedStateConfiguration { serializersModule = SerializersModule { polymorphic(NavKey::class) { subclass(Product::class) // ..other 100-ish classes. } } } // BackStack with initial route val backStack = rememberSerializable(configuration) { mutableStateListOf<NavKey>(Home) }

NavKey @Serializable data class Product(val id: String) : NavKey // Define serializers module val configuration = SavedStateConfiguration { serializersModule = SerializersModule { polymorphic(NavKey::class) { subclass(Product::class) // ..other 100-ish classes. } } } // BackStack with initial route val backStack = rememberNavBackStack(configuration, Home)

NavKeySerializer Android-only

public open class NavKeySerializer<T : NavKey> : KSerializer<T> { override
fun serialize(encoder: Encoder, value: T) { encoder.encodeStructure(descriptor) { val className = value::class.java.name encodeStringElement(descriptor, index = 0, className) val serializer = value::class.serializer() as KSerializer<T> encodeSerializableElement(descriptor, index = 1, serializer, value) } } override fun deserialize(decoder: Decoder): T { return decoder.decodeStructure(descriptor) { val className = decodeStringElement(descriptor, decodeElementIndex(descriptor)) val serializer = Class.forName(className).kotlin.serializer() decodeSerializableElement(descriptor, decodeElementIndex(descriptor), serializer) as T } } }

// Define routes and arguments @Serializable data class Product(val id:
String) : NavKey // Define serializers module val configuration = SavedStateConfiguration { serializersModule = SerializersModule { polymorphic(NavKey::class) { subclass(Product::class) // ..other 100-ish classes. } } } // BackStack with initial route val backStack = rememberNavBackStack(configuration, Home) @Composable public fun rememberNavBackStack(vararg elements: NavKey): NavBackStack<NavKey> { return rememberSerializable( serializer = NavBackStackSerializer(elementSerializer = NavKeySerializer()) ) { NavBackStack(*elements) } }

String) : NavKey // Define serializers module val configuration = SavedStateConfiguration { serializersModule = SerializersModule { polymorphic(NavKey::class) { subclass(Product::class) // ..other 100-ish classes. } } } // BackStack with initial route val backStack = rememberNavBackStack(configuration, Home) @Serializable(with = NavBackStackSerializer::class) public class NavBackStack<T : NavKey> (val base: SnapshotStateList<T>) : MutableList<T> by base, StateObject by base, RandomAccess by base

String) : NavKey // Define serializers module val configuration = SavedStateConfiguration { serializersModule = SerializersModule { polymorphic(NavKey::class) { subclass(Product::class) // ..other 100-ish classes. } } } // BackStack with initial route val backStack = rememberNavBackStack(configuration, Home) public class NavBackStackSerializer<T : NavKey>(val elementSerializer: KSerializer<T>) : KSerializer<NavBackStack<T>> { private val delegate = SnapshotStateListSerializer(elementSerializer) override val descriptor = SerialDescriptor("...NavBackStack", delegate.descriptor) override fun serialize(encoder: Encoder, value: NavBackStack<T>): Unit = encoder.encodeSerializableValue(serializer = delegate, value = value.base) override fun deserialize(decoder: Decoder): NavBackStack<T> = NavBackStack(base = decoder.decodeSerializableValue(deserializer = delegate)) }

String) : NavKey // Define serializers module val configuration = SavedStateConfiguration { serializersModule = SerializersModule { polymorphic(NavKey::class) { subclass(Product::class) // ..other 100-ish classes. } } } // BackStack with initial route val backStack = rememberNavBackStack(configuration, Home)

NavKey @Serializable data class Product(val id: String) : NavKey // BackStack with initial route val backStack = rememberNavBackStack(Home)

Why not just use sealed classes instead?

Open Polymorphism and Trade-Offs • Explicit registration is the KotlinX
way → correct and performant.

way → correct and performant. • But it can be verbose (registering every route) if you use distinct classes.

way → correct and performant. • But it can be verbose (registering every route) if you use distinct classes. • On Android we use a reflective serializer for ease of use.

way → correct and performant. • But it can be verbose (registering every route) if you use distinct classes. • On Android we use a reflective serializer for ease of use. • The KotlinX team is working on improvements (e.g. register sealed interfaces once). ◦ github.com/Kotlin/kotlinx.serialization/issues/2199

github.com/Kotlin/kotlinx.serialization/issues/2199

Custom Types and Helpers

NavKey @Serializable data class Product(val id: String) : NavKey // BackStack with initial route val backStack = rememberNavBackStack(Home)

// Define routes and arguments @Serializable(with = MyKeySerializer::class) interface MyKey
: NavKey private object MyKeySerializer : NavKeySerializer<MyKey>() // BackStack with initial route val backStack = rememberSerializable { NavBackStack<MyKey>() }

// Define routes and arguments @Serializable sealed class MyKey //
BackStack with initial route val backStack = rememberSerializable { NavBackStack<MyKey>() }

// Define routes and arguments @Serializable sealed class MyKey class
MyViewModel(handle: SavedStateHandle) : ViewModel() { val backStack = handle.saved { NavBackStack<MyKey>() } }

// Define routes and arguments @Serializable sealed class MyKey class
MyViewModel(handle: SavedStateHandle) : ViewModel() { val backStack = handle.saved { NavBackStack<MyKey>() } } public fun <T> SavedStateHandle.saved( serializer: KSerializer<T>, key: String? = null, configuration: SavedStateConfiguration = SavedStateConfiguration.DEFAULT, init: () -> T, ): ReadWriteProperty<Any?, T> { return SavedStateHandleDelegate(this, serializer, key, configuration, init) }

@dagger.Singleton class Navigator(val backStack: NavBackStack<NavKey>)

Conclusion • This work makes navigation state shareable across KMP
platforms. goo.gle/nav3

Conclusion • This work makes navigation state shareable across KMP
platforms in Nav3. • We’d love your feedback. goo.gle/nav3

Thank you

Custom Codecs with KotlinX Serialization

Custom Codecs with KotlinX Serialization

More Decks by Marcello Galhardo

Other Decks in Programming

Featured

Transcript