Droidcon London '21 - A Hitchhiker's Guide to Compose Compiler: Composers, Compiler Plugins, and Snapshots

If you are an Android Developer, chances are you are pretty excited about Jetpack Compose! But how does it work? In this talk, we will take you along the journey of a Composable function, from being written, going through the unknown lands of the Kotlin and Compose compilers to being executed and displaying UI!

From Composables all the way down into the compiler plugins, we expose the metaprogramming responsible for all the "magic". Whether your interest is Jetpack Compose, compilers, or code transformations, this talk takes an otherwise complicated topic and makes it digestible for everybody by diving into specific features offered by Compose.

By examining Compose's snapshot system, you'll follow the compiler phases down to intercepting code transformations with IR. By recognizing the patterns that make it hard for Compose to generate efficient code, and this can help us to recognize patterns to use for performance.

Transcript

1. Hitchhiker’s Guide to Compose Snapshots, Compiler Plugins and Appliers @mvndy_hd @jossiwolf #hg2compose
2. "Jetpack Compose is Android's modern toolkit for building native UI." #hg2compose - developer.android.com @mvndy_hd @jossiwolf
3. Compiler Runtime UI #hg2compose != @mvndy_hd @jossiwolf
4. #hg2compose TextView(text = "") FrameLayout FrameLayout @mvndy_hd @jossiwolf "Drawn out: How Android UI renders Google I/O '18"
5. Drawn out: How Android UI Renders (Google I/O '18) https://bit.ly/3pRnStc
6. #hg2compose setText("Bob") TextView(text = "") FrameLayout FrameLayout @mvndy_hd @jossiwolf "Drawn out: How Android UI renders Google I/O '18"
7. #hg2compose FrameLayout setText("Bob") FrameLayout TextView(text = "Bob") @mvndy_hd @jossiwolf "Drawn out: How Android UI renders Google I/O '18"
8. #hg2compose FrameLayout FrameLayout TextView(text = "Bob") getDisplayList() draw() @mvndy_hd @jossiwolf "Drawn out: How Android UI renders Google I/O '18"
9. #hg2compose FrameLayout FrameLayout TextView(text = "Bob") getDisplayList() getDisplayList() @mvndy_hd @jossiwolf "Drawn out: How Android UI renders Google I/O '18"
10. #hg2compose FrameLayout FrameLayout TextView(text = "Bob") getDisplayList() getDisplayList() getDisplayList() @mvndy_hd @jossiwolf "Drawn out: How Android UI renders Google I/O '18"
11. #hg2compose FrameLayout FrameLayout TextView(text = "Bob") draw() Bob @mvndy_hd @jossiwolf "Drawn out: How Android UI renders Google I/O '18"
12. #hg2compose Text(text = name) Layout Layout name by remember { mutableStateOf("") } @mvndy_hd @jossiwolf
13. A Composable is a restartable function that emits nodes into a tree. #hg2compose @mvndy_hd @jossiwolf Glossary
14. @Composable fun HitchhikerApp() { var name by remember { mutableStateOf("Bob") } HelloWorld(greeting = name) name = "Chet" } @Composable fun HelloWorld(greeting: String) { Text("Hello, $greeting!") } #hg2compose @mvndy_hd @jossiwolf
15. #hg2compose Text(text = name) Layout Layout name by remember { mutableStateOf("") } @mvndy_hd @jossiwolf
16. #hg2compose Text(text = name) Layout Layout name by remember { mutableStateOf("Bob") } @mvndy_hd @jossiwolf
17. #hg2compose Text(text = name) Layout Layout name by remember { mutableStateOf("Bob") } @mvndy_hd @jossiwolf
18. #hg2compose Text(text = name) Layout Layout name by remember { mutableStateOf("Bob") } Bob @mvndy_hd @jossiwolf
19. • State (Snapshots) • Memory (Slot Tables) • State Management (Recomposition) #hg2compose @mvndy_hd @jossiwolf
20. Snapshots 📸 #hg2compose @mvndy_hd @jossiwolf
21. "A Snapshot is a lot like a save point in a video game: it represents the state of your entire program at a single point in history." #hg2compose https://dev.to/zachklipp/introduction-to-the-compose-snapshot-system-19cn - "Introduction to the Compose Snapshot System" by Zach Klippenstein Snapshots 📸 (and dogs🐕) @mvndy_hd @jossiwolf
22. #hg2compose interface MutableState<T> : State<T> { override var value: T operator fun component1(): T operator fun component2(): (T) .> Unit } package androidx.compose.runtime @mvndy_hd @jossiwolf
23. #hg2compose https://dev.to/zachklipp/introduction-to-the-compose-snapshot-system-19cn fun main() { val dog = Dog() dog.name.value = "Spot" val snapshot = Snapshot.takeSnapshot() dog.name.value = "Fido" println(dog.name.value) ./ Fido snapshot.enter { println(dog.name.value) ./ Spot } println(dog.name.value) ./ Fido } Snapshots 📸 (and dogs🐕) @mvndy_hd @jossiwolf
24. #hg2compose Dog(name = "Spot") Snapshots 📸 (and dogs🐕) @mvndy_hd @jossiwolf Snapshot.takeMutableSnapshot()
25. #hg2compose Snapshot.takeMutableSnapshot() 1 2 Dog(name = "Spot") Snapshots 📸 (and dogs🐕) @mvndy_hd @jossiwolf
26. #hg2compose dog.name.value = "Fido" 1 2 Dog(name = "Spot") Dog(name = "Fido") Snapshots 📸 (and dogs🐕) @mvndy_hd @jossiwolf Snapshot.takeMutableSnapshot()
27. #hg2compose dog.name.value = "Fido" 1 2 Dog(name = "Spot") Dog(name = "Fido") snapshot.enter { ... } Snapshots 📸 (and dogs🐕) @mvndy_hd @jossiwolf Snapshot.takeMutableSnapshot()
28. #hg2compose @mvndy @jossiwolf Snapshots 📸 (and dogs🐕) 1 2 dog.name.value = "Fido" Dog(name = "Fido") snapshot.enter { dog.name.value = "Taco" } Dog(name = "Spot") Snapshot.takeMutableSnapshot()
29. #hg2compose @mvndy @jossiwolf Snapshots 📸 (and dogs🐕) 1 2 dog.name.value = "Fido" Dog(name = "Fido") snapshot.enter { dog.name.value = "Taco" } Dog(name = "Spot") Snapshot.takeMutableSnapshot()
30. #hg2compose @mvndy @jossiwolf Snapshots 📸 (and dogs🐕) 1 2 dog.name.value = "Fido" Dog(name = "Fido") snapshot.enter { dog.name.value = "Taco" } Dog(name = "Spot") Dog(name = "Taco") ✔ Snapshot.takeMutableSnapshot()
31. #hg2compose @mvndy @jossiwolf Snapshots 📸 (and dogs🐕) snapshot.apply() dog.name.value = "Fido" 1 2 snapshot.enter { dog.name.value = "Taco" } Dog(name = "Spot") Dog(name = "Taco") Dog(name = "Fido") Dog(name = "Taco") Snapshot.takeMutableSnapshot()
32. #hg2compose @mvndy @jossiwolf Snapshots 📸 (and dogs🐕) snapshot.apply() dog.name.value = "Fido" 1 2 snapshot.enter { dog.name.value = "Taco" } Dog(name = "Spot") Dog(name = "Taco") Dog(name = "Fido") Dog(name = "Taco") Snapshot.takeMutableSnapshot()
33. #hg2compose @mvndy @jossiwolf fun main() { val dog = Dog() dog.name.value = "Spot" val snapshot = Snapshot.takeMutableSnapshot( readObserver = { value .> println(value) ./ MutableState(value=Spot)@... }, writeObserver = { value .> ... } ) dog.name.value = "Fido" snapshot.enter { dog.name.value = "Taco" } } Snapshots 📸 (and dogs🐕)
34. • State (Snapshots) • Memory (Slot Tables) • State Management (Recomposition) #hg2compose @mvndy_hd @jossiwolf
35. The Slot Table is used to store the current state of the Composition. #hg2compose @mvndy_hd @jossiwolf Glossary
36. The Composition holds the Slot Table, invalidations and a Composer. #hg2compose @mvndy_hd @jossiwolf Glossary
37. A Composer connects Composables to Runtime, keeps track of groups that are used to build the Slot Table and positions it. #hg2compose Glossary @mvndy_hd @jossiwolf
38. #hg2compose @mvndy_hd @jossiwolf interface Applier<N> { val current: N fun onBeginChanges() {} fun onEndChanges() {} fun down(node: N) fun up() fun insertTopDown(index: Int, instance: N) fun insertBottomUp(index: Int, instance: N) fun remove(index: Int, count: Int) fun move(from: Int, to: Int, count: Int) fun clear() }
39. #hg2compose var name by remember { mutableStateOf("Bob") } @Composable fun Greeting(greeting: String) { Text("Hello, $greeting!") } Greeting(name) name = "Jossi" ./ Hello, Bob! ./ Hello, Jossi! Slot Table by Example @mvndy_hd @jossiwolf
40. #hg2compose EMPTY EMPTY Group(4) State(“Bob”) var name by remember { mutableStateOf("Bob") } Slot Table by Example EMPTY EMPTY EMPTY EMPTY EMPTY @mvndy_hd @jossiwolf
41. #hg2compose Group(4) State(“Bob”) Group(5) var name by remember { mutableStateOf("Bob") } Slot Table by Example EMPTY EMPTY EMPTY EMPTY EMPTY EMPTY @Composable fun Greeting(greeting: String) { } Greeting(name) @mvndy_hd @jossiwolf
42. #hg2compose EMPTY EMPTY Group(4) State(“Bob”) Group(5) Group(6) Group(7) Group(8) “Hello, Bob!” var name by remember { mutableStateOf("Bob") } Slot Table by Example @Composable fun Greeting(greeting: String) { Text("Hello, $greeting!”) } Greeting(name) @mvndy_hd @jossiwolf
43. #hg2compose EMPTY EMPTY Group(4) State(“Bob”) Group(5) Group(6) Group(7) Group(8) “Hello, Jossi!” var name by remember { mutableStateOf("Bob") } @Composable fun Greeting(greeting: String) { Text("Hello, $greeting!”) } Greeting(name) name = "Jossi" Slot Table by Example @mvndy_hd @jossiwolf
44. #hg2compose Text(text = name) Layout Layout name by remember { mutableStateOf("Bob") } Bob @mvndy_hd @jossiwolf
45. Recomposition is the process of re-executing a Composable function to procude an updated slot table #hg2compose @mvndy_hd @jossiwolf Glossary
46. #hg2compose suspend fun runApp() { val composer = Recomposer() GlobalSnapshotManager.ensureStarted() launch(DefaultChoreographerFrameClock) { composer.runRecomposeAndApplyChanges() } Composition(NodeApplier(RootNode()), composer).apply { setContent { Content() } } } @mvndy_hd @jossiwolf
47. #hg2compose suspend fun runApp() { val composer = Recomposer() GlobalSnapshotManager.ensureStarted() launch(DefaultChoreographerFrameClock) { composer.runRecomposeAndApplyChanges() } Composition(NodeApplier(RootNode()), composer).apply { setContent { Content() } } } medium.com/@takahirom/inside-jetpack-compose-2e971675e55e @mvndy_hd @jossiwolf
48. Recomposers manage the Snapshot and Compositions, i.e. composing initially and recomposing. #hg2compose @mvndy_hd @jossiwolf
49. #hg2compose suspend fun runApp() { val composer = Recomposer() GlobalSnapshotManager.ensureStarted() launch(DefaultChoreographerFrameClock) { composer.runRecomposeAndApplyChanges() } Composition(NodeApplier(RootNode()), composer).apply { setContent { Content() } } } medium.com/@takahirom/inside-jetpack-compose-2e971675e55e @mvndy_hd @jossiwolf
50. #hg2compose private inline fun <T> composing( composition: ControlledComposition, modifiedValues: IdentityArraySet<Any>?, block: () .> T ): T { val snapshot = Snapshot.takeMutableSnapshot( readObserverOf(composition), writeObserverOf(composition, modifiedValues) ) try { return snapshot.enter(block) } finally { applyAndCheck(snapshot) } } Recomposer @mvndy_hd @jossiwolf
51. #hg2compose private inline fun <T> composing( composition: ControlledComposition, modifiedValues: IdentityArraySet<Any>?, block: () .> T ): T { val snapshot = Snapshot.takeMutableSnapshot( readObserverOf(composition), writeObserverOf(composition, modifiedValues) ) try { return snapshot.enter(block) } finally { applyAndCheck(snapshot) } } Recomposer @mvndy_hd @jossiwolf
52. #hg2compose private inline fun <T> composing( composition: ControlledComposition, modifiedValues: IdentityArraySet<Any>?, block: () .> T ): T { val snapshot = Snapshot.takeMutableSnapshot( readObserverOf(composition), writeObserverOf(composition, modifiedValues) ) try { return snapshot.enter(block) } finally { applyAndCheck(snapshot) } } Recomposer @mvndy_hd @jossiwolf
53. @Composable fun HelloWorld(greeting: String) { Text("Hello, $greeting!") } #hg2compose @mvndy_hd @jossiwolf
54. @Composable fun HelloWorld(greeting: String, %composer: Composer?, %changed: Int) { %composer = %composer.startRestartGroup(.>) val %dirty = %changed if (%changed and 0b1110 ..= 0) { %dirty = %dirty or if (%composer.changed(greeting)) 0b0100 else 0b0010 } if (%dirty and 0b1011 xor 0b0010 ..= 0 .| !%composer.skipping) { Text("Hello, $greeting!") } else { %composer.skipToGroupEnd() } %composer.endRestartGroup() ..updateScope { %composer: Composer?, %force: Int .> HelloWorld(greeting, %composer, %changed or 0b0001) } } #hg2compose @mvndy_hd @jossiwolf
55. @Composable fun HelloWorld(greeting: String, %composer: Composer?, %changed: Int) { %composer = %composer.startRestartGroup(.>) val %dirty = %changed if (%changed and 0b1110 ..= 0) { %dirty = %dirty or if (%composer.changed(greeting)) 0b0100 else 0b0010 } if (%dirty and 0b1011 xor 0b0010 ..= 0 .| !%composer.skipping) { Text("Hello, $greeting!") } else { %composer.skipToGroupEnd() } %composer.endRestartGroup() ..updateScope { %composer: Composer?, %force: Int .> HelloWorld(greeting, %composer, %changed or 0b0001) } } #hg2compose @mvndy_hd @jossiwolf
56. @Composable fun HelloWorld(greeting: String, %composer: Composer?, %changed: Int) { %composer = %composer.startRestartGroup(.>) val %dirty = %changed if (%changed and 0b1110 ..= 0) { %dirty = %dirty or if (%composer.changed(greeting)) 0b0100 else 0b0010 } if (%dirty and 0b1011 xor 0b0010 ..= 0 .| !%composer.skipping) { Text("Hello, $greeting!") } else { %composer.skipToGroupEnd() } %composer.endRestartGroup() ..updateScope { %composer: Composer?, %force: Int .> HelloWorld(greeting, %composer, %changed or 0b0001) } } #hg2compose @mvndy_hd @jossiwolf
57. @Composable fun HelloWorld(greeting: String, %composer: Composer?, %changed: Int) { %composer = %composer.startRestartGroup(.>) val %dirty = %changed if (%changed and 0b1110 ..= 0) { %dirty = %dirty or if (%composer.changed(greeting)) 0b0100 else 0b0010 } if (%dirty and 0b1011 xor 0b0010 ..= 0 .| !%composer.skipping) { Text("Hello, $greeting!") } else { %composer.skipToGroupEnd() } %composer.endRestartGroup() ..updateScope { %composer: Composer?, %force: Int .> HelloWorld(greeting, %composer, %changed or 0b0001) } } #hg2compose @mvndy_hd @jossiwolf
58. @Composable fun HelloWorld(greeting: String, %composer: Composer?, %changed: Int) { %composer = %composer.startRestartGroup(.>) val %dirty = %changed if (%changed and 0b1110 ..= 0) { %dirty = %dirty or if (%composer.changed(greeting)) 0b0100 else 0b0010 } if (%dirty and 0b1011 xor 0b0010 ..= 0 .| !%composer.skipping) { Text("Hello, $greeting!") } else { %composer.skipToGroupEnd() } %composer.endRestartGroup() ..updateScope { %composer: Composer?, %force: Int .> HelloWorld(greeting, %composer, %changed or 0b0001) } } #hg2compose @mvndy_hd @jossiwolf
59. @Composable fun HelloWorld(greeting: String, %composer: Composer?, %changed: Int) { %composer = %composer.startRestartGroup(.>) val %dirty = %changed if (%changed and 0b1110 ..= 0) { %dirty = %dirty or if (%composer.changed(greeting)) 0b0100 else 0b0010 } if (%dirty and 0b1011 xor 0b0010 ..= 0 .| !%composer.skipping) { Text("Hello, $greeting!") } else { %composer.skipToGroupEnd() } %composer.endRestartGroup() ..updateScope { %composer: Composer?, %force: Int .> HelloWorld(greeting, %composer, %changed or 0b0001) } } #hg2compose @mvndy_hd @jossiwolf
60. @Composable fun HelloWorld(greeting: String, %composer: Composer?, %changed: Int) { %composer = %composer.startRestartGroup(.>) val %dirty = %changed if (%changed and 0b1110 ..= 0) { %dirty = %dirty or if (%composer.changed(greeting)) 0b0100 else 0b0010 } if (%dirty and 0b1011 xor 0b0010 ..= 0 .| !%composer.skipping) { Text("Hello, $greeting!") } else { %composer.skipToGroupEnd() } %composer.endRestartGroup() ..updateScope { %composer: Composer?, %force: Int .> HelloWorld(greeting, %composer, %changed or 0b0001) } } #hg2compose @mvndy_hd @jossiwolf
61. Restartable Groups @Composable fun HelloWorld(greeting: String) { Text("Hello, $greeting!") } @mvndy_hd @jossiwolf
62. @Composable fun HelloWorld(names: List<String>) { names.forEach { name .> key(name) { Text(name) } } } Moveable Groups @mvndy_hd @jossiwolf
63. @Composable fun HelloWorld(name: String) { if (name .= "Amanda") { Text("Hello!") else if (name .= "Jossi") { Text("Hallo!") } else { Text("Uhm.. hi?") } } Replaceable Groups @mvndy_hd @jossiwolf
64. #hg2compose .kt Lexer (Scanner) Syntax Analysis Parsing Phase Generates tokens - Generates tokens to create an AST - PSI lays over AST Resolution Codegen Resolution performed on AST tree - PSI gets enhanced with descriptors - Symbol table generated; associated node w/ descriptor via BindingTrace Frontend Backend - Takes optimized IR and performs more analysis, transformations + optimisations specific to target CPU architecture - Multithreading Machine-Dependent Optimisations IR Transform Compiler Analysis Middle End “Lower” - Performs optimisations on IR - Removes dead code - Refactors the code - Improves performance - Analyzes IR for data needed to create - Call graph - Control-flow graph < > Bytecode Target Program FUN BLOCK_BODY CALL VAR IR Unoptimized IR Codegen IR to Bytecode Optimized IR FUN BLOCK_BODY CALL VAR IR Backend The Kotlin compiler @mvndy_hd @jossiwolf
65. #hg2compose .kt Lexer (Scanner) Syntax Analysis Parsing Phase Generates tokens - Generates tokens to create an AST - PSI lays over AST Resolution Codegen Resolution performed on AST tree - PSI gets enhanced with descriptors - Symbol table generated; associated node w/ descriptor via BindingTrace Frontend Backend - Takes optimized IR and performs more analysis, transformations + optimisations specific to target CPU architecture - Multithreading Machine-Dependent Optimisations IR Transformations Compiler Analysis Middle End “Lower” - Performs optimisations on IR - Removes dead code - Refactors the code - Improves performance - Analyzes IR for data needed to create - Call graph - Control-flow graph < > Bytecode Target Program FUN BLOCK_BODY CALL VAR IR Unoptimized IR Codegen IR to Bytecode R t bytecod Optimized IR FUN BLOCK_BODY CALL VAR IR Backend Compose compiler plugin Backend - Takes optimized IR and performs more analysis, transformations + optimisations specific to target CPU architecture - Multithreading Machine-Dependent Optimisations IR Transform Compiler Analysis Middle End “Lower” - Performs optimisations on IR - Removes dead code - Refactors the code - Improves performance - Analyzes IR for data needed to create - Call graph - Control-flow graph < > Bytecode Target Program FUN BLOCK_BODY CALL VAR IR Unoptimized IR Codegen IR to Bytecode Optimized IR FUN BLOCK_BODY CALL VAR IR Backend @mvndy_hd @jossiwolf
66. @Composable fun HelloWorld() { Text("Hello World!") } #hg2compose @mvndy_hd @jossiwolf
67. #hg2compose .kt Lexer (Scanner) Syntax Analysis Parsing Phase Generates tokens - Generates tokens to create an AST - PSI lays over AST Resolution Codegen Semantic Analysis performed on AST - PSI gets enhanced with descriptors - Symbol table generated; associated node w/ descriptor via BindingTrace Kotlin Compiler Frontend @mvndy_hd @jossiwolf
68. Element.FUN Element.BLOCK Element.CALL_EXPRESSION Token.L_BRACE Token.R_BRACE Element.REFERENCE_EXPRESSION Element.VALUE_ARGUMENT_LIST Token.IDENTIFIER Token.LPAR Token.RPAR Element.STRING_TEMPLATE Element.LITERAL_STRING_TEMPLATE_ENTRY Token.OPEN_QUOTE Element.REGULAR_STRING_PART Token.OPEN_QUOTE #hg2compose @mvndy_hd @jossiwolf Text("Hello, World!") AST
69. KtBlockExpression KtCallExpression KtReferenceExpression KtValueArgument #hg2compose @mvndy_hd @jossiwolf Text("Hello, World!") KtValueArgument KtStringTemplateExpression PSI
70. The Shape of Kotlin https://bit.ly/3mmKJe8
71. #hg2compose .kt Lexer (Scanner) Syntax Analysis Parsing Phase Generates tokens - Generates tokens to create an AST - PSI lays over AST Resolution Codegen Semantic Analysis performed on AST - PSI gets enhanced with descriptors - Symbol table generated; associated node w/ descriptor via BindingTrace Kotlin Compiler Frontend @mvndy_hd @jossiwolf
72. #hg2compose IR Transform Compiler Analysis Middle End “Lower” - Performs optimisations on IR - Removes dead code - Refactors the code - Improves performance - Analyzes IR for data needed to create - Call graph - Control-flow graph Backend - Takes optimized IR and performs more analysis, transformations + optimisations specific to target CPU architecture - Multithreading Optimized IR < > Bytecode Target Program @mvndy_hd @jossiwolf FUNC VAL_PARAM VAL_PARAM VA:_PARAM FUNC VAL_PARAM VAL_PARAM VA:_PARAM Unoptimized IR
73. #hg2compose @mvndy_hd @jossiwolf - Performs optimisations on IR - Removes dead code - Refactors the code - Improves performance - Analyzes IR for data needed to create - Call graph - Control-flow graph CPU-related IR Transformations IR Transform Compiler Analysis Middle End “Lower” Backend - Takes optimized IR and performs more analysis, transformations + optimisations specific to target CPU architecture - Multithreading < > Bytecode Target Program - Performs optimisations on IR - Removes dead code - Refactors the code - Improves performance - Analyzes IR for data needed to create - Call graph - Control-flow graph CPU-related IR Transformations FUNC VAL_PARAM VAL_PARAM VA:_PARAM Unoptimized IR
75. 🏃Efficient Codegen Tips󰝋 #hg2compose The point of Compose transformations is to reduce as many groups + group executions as possible - but you can help too! @mvndy_hd @jossiwolf
76. #hg2compose Use key to make recomposition of lists cheaper! This helps Compose generate moveable groups, avoiding recomposition i.e. when the position of a list item changes. @mvndy_hd @jossiwolf
77. #hg2compose Make sure to read state values at the lowest node of the tree possible You only want to recompose where you actually need the state :) @mvndy_hd @jossiwolf
78. • Jorge Castillo, author of "Compose Internals" • Leland Richardson for his work and sitting down with us to interview him • Zach Klippenstein's 5-part series on Compose state explained #hg2compose Thank you!!! @mvndy_hd @jossiwolf
79. #hg2compose linktr.ee/hg2compose @mvndy_hd @jossiwolf