Introducing RemoteCompose: break your UI out of the app sandbox.

RemoteCompose is a new AndroidX framework that lets you project components with rich graphics from your app (or even your server!) onto diverse remote surfaces, including android widgets.

In this session we’ll unpack the end-to-end architecture and developer story (from creation to playback), show how layout, state, animation, and expressions work, and build rich interactive demos (particle effects, shaders...). We’ll also dig into how RemoteCompose can power widgets in Android 16, and what that unlocks for the future of multi-surface experiences and server-driven systems.

Transcript

1. Nicolas Roard John Hoford RemoteCompose Break your UI out of the app sandbox
2. User Interface UI Business Logic Application
3. User Interface UI Business Logic Application
4. User Interface UI Business Logic Application
5. User Interface UI Business Logic Application Noti fi cations Widgets Cast
6. RemoteCompose handling Remote Surfaces
7. RemoteCompose Handling remote surfaces
8. RemoteCompose Handling remote surfaces
9. User Interface UI User Interaction Animation Components
10. RemoteCompose Binary Document
11. RemoteCompose Create Binary Document Play
12. RemoteCompose Create Binary Document Play Creation Library (KMP) Compose Library Android View player Widgets (RemoteView) Compose player Compose desktop player Glance
13. Features Feedback w elcom e!
14. What’s different about it • Self-contained • Binary format • More than a display list, less than an application • Small !
15. What does it do • 2D Canvas operations • Components • Expression system • Animation (path morphing, shaders, particles…) • Accessibility support • Touch input • Actions • Theme support
16. Components • Compose model (modi fi ers, etc.) • Built-in layouts • Box, Row, Column • Adaptive layout: FitBox, CollapsibleRow, CollapsibleColumn • Custom layouts • Built-in Component animation
17. Components Touch events & animation
18. Components Touch events & animation
19. Expression system • Built-in variables • Time, Sensor data • E ffi cient evaluation • Math & array operators • Value animation
20. Walkthrough
21. @RemoteComposable @Composable fun GreetingRC(modifier: RemoteModifier = RemoteModifier) { RemoteBox(modifier = modifier) { RemoteText( text = "Hello world!", color = RemoteColor(Color.White) ) } }
22. Some drawing… @Composable fun Simple() { Canvas(modifier = Modifier.fillMaxSize() .background(Color.White)) { val w = size.width val h = size.height drawLine(Color.Blue, Offset(0f, 0f), Offset(w, h)) drawLine(Color.Blue, Offset(0f, h), Offset(w, 0f)) } }
23. Some drawing… @Composable fun SimpleRC() { RemoteCanvas(modifier = RemoteModifier.fillMaxSize() .background(Color.White)) { val w = remote.component.width val h = remote.component.height drawLine(Color.Blue, RemoteOffset(0f, 0f), RemoteOffset(w, h)) drawLine(Color.Blue, RemoteOffset(0f, h), RemoteOffset(w, 0f)) } }
24. Some drawing… @Composable fun SimpleRC() { RemoteCanvas(modifier = RemoteModifier.fillMaxSize() .background(Color.White)) { val w = remote.component.width val h = remote.component.height drawLine(Color.Blue, RemoteOffset(0f, 0f), RemoteOffset(w, h)) drawLine(Color.Blue, RemoteOffset(0f, h), RemoteOffset(w, 0f)) } } @Composable fun Simple() { Canvas(modifier = Modifier.fillMaxSize() .background(Color.White)) { val w = size.width val h = size.height drawLine(Color.Blue, Offset(0f, 0f), Offset(w, h)) drawLine(Color.Blue, Offset(0f, h), Offset(w, 0f)) } }
27. RemoteBox( RemoteModifier.fillMaxSize().clip(RoundedCornerShape(42.dp)), horizontalAlignment = Alignment.CenterHorizontally, ) { RemoteCanvas(modifier = RemoteModifier.fillMaxSize().background(Color(3, 130, 166))) { val w = remote.component.width val h = remote.component.height val cx = w / 2f val cy = h / 2f val rad = min(cx, cy) val timeHr = remote.time.Hour() val timeMin: RemoteFloat = remote.time.Minutes() val continuousSec: RemoteFloat = remote.time.ContinuousSec() rotate(timeHr * 30f, cx, cy) { drawLine( Color.LightGray, ROffset(cx, cy), ROffset(cx, cy - rad / 2f), strokeWidth = 64f, cap = StrokeCap.Round, ) } rotate(timeMin * 6f, cx, cy) { drawLine( Color.Gray,
28. RemoteBox( RemoteModifier.fillMaxSize().clip(RoundedCornerShape(42.dp)), horizontalAlignment = Alignment.CenterHorizontally, ) { RemoteCanvas(modifier = RemoteModifier.fillMaxSize().background(Color(3, 130, 166))) { val w = remote.component.width val h = remote.component.height val cx = w / 2f val cy = h / 2f val rad = min(cx, cy) val timeHr = remote.time.Hour() val timeMin: RemoteFloat = remote.time.Minutes() val continuousSec: RemoteFloat = remote.time.ContinuousSec() rotate(timeHr * 30f, cx, cy) { drawLine( Color.LightGray, ROffset(cx, cy), ROffset(cx, cy - rad / 2f), strokeWidth = 64f, cap = StrokeCap.Round, ) } rotate(timeMin * 6f, cx, cy) { drawLine( Color.Gray, Expressions Canvas
29. ROffset(cx, cy), ROffset(cx, cy - rad / 2f), strokeWidth = 64f, cap = StrokeCap.Round, ) } rotate(timeMin * 6f, cx, cy) { drawLine( Color.Gray, ROffset(cx, cy), ROffset(cx, cy - rad * 0.8f), strokeWidth = 32f, cap = StrokeCap.Round, ) } drawLine( Color.White, ROffset(cx, cy), ROffset( w / 2f + rad * sin(continuousSec * (2 * Math.PI.toFloat() / 60f)), h / 2f - rad * cos(continuousSec * (2 * Math.PI.toFloat() / 60f)), ), strokeWidth = 2f, cap = StrokeCap.Round, ) } }
30. ROffset(cx, cy), ROffset(cx, cy - rad / 2f), strokeWidth = 64f, cap = StrokeCap.Round, ) } rotate(timeMin * 6f, cx, cy) { drawLine( Color.Gray, ROffset(cx, cy), ROffset(cx, cy - rad * 0.8f), strokeWidth = 32f, cap = StrokeCap.Round, ) } drawLine( Color.White, ROffset(cx, cy), ROffset( w / 2f + rad * sin(continuousSec * (2 * Math.PI.toFloat() / 60f)), h / 2f - rad * cos(continuousSec * (2 * Math.PI.toFloat() / 60f)), ), strokeWidth = 2f, cap = StrokeCap.Round, ) } } Seconds Minutes
31. fun clockCreation(): RemoteComposeWriter { val rc = RemoteComposeContextAndroid( width = 500, height = 500, contentDescription = "Simple Timer", apiLevel = 6, profiles = 0, platform = AndroidxRcPlatformServices(), ) { root { box(RecordingModifier().fillMaxSize(), BoxLayout.START, BoxLayout.START) { canvas(RecordingModifier().fillMaxSize()) { val w = ComponentWidth() val h = ComponentHeight() val cx = w / 2f val cy = h / 2f val rad = min(cx, cy) painter.setColor(Color.BLUE).commit() drawRoundRect(0, 0, w, h, rad / 4f, rad / 4f) painter .setColor(Color.GRAY) .setStrokeWidth(32f) .setStrokeCap(Paint.Cap.ROUND) .commit()
32. fun clockCreation(): RemoteComposeWriter { val rc = RemoteComposeContextAndroid( width = 500, height = 500, contentDescription = "Simple Timer", apiLevel = 6, profiles = 0, platform = AndroidxRcPlatformServices(), ) { root { box(RecordingModifier().fillMaxSize(), BoxLayout.START, BoxLayout.START) { canvas(RecordingModifier().fillMaxSize()) { val w = ComponentWidth() val h = ComponentHeight() val cx = w / 2f val cy = h / 2f val rad = min(cx, cy) painter.setColor(Color.BLUE).commit() drawRoundRect(0, 0, w, h, rad / 4f, rad / 4f) painter .setColor(Color.GRAY) .setStrokeWidth(32f) .setStrokeCap(Paint.Cap.ROUND) .commit() Setup Components
33. box(RecordingModifier().fillMaxSize(), BoxLayout.START, BoxLayout.START) { canvas(RecordingModifier().fillMaxSize()) { val w = ComponentWidth() val h = ComponentHeight() val cx = w / 2f val cy = h / 2f val rad = min(cx, cy) painter.setColor(Color.BLUE).commit() drawRoundRect(0, 0, w, h, rad / 4f, rad / 4f) painter .setColor(Color.GRAY) .setStrokeWidth(32f) .setStrokeCap(Paint.Cap.ROUND) .commit() save() { rotate(Minutes() * 6f, cx, cy) drawLine(cx, cy, cx, cy - rad * 0.8f) } painter .setColor(Color.LTGRAY) .setStrokeWidth(16f) .setStrokeCap(Paint.Cap.ROUND) .commit() save() { rotate(Hour() * 30f, cx, cy)
34. box(RecordingModifier().fillMaxSize(), BoxLayout.START, BoxLayout.START) { canvas(RecordingModifier().fillMaxSize()) { val w = ComponentWidth() val h = ComponentHeight() val cx = w / 2f val cy = h / 2f val rad = min(cx, cy) painter.setColor(Color.BLUE).commit() drawRoundRect(0, 0, w, h, rad / 4f, rad / 4f) painter .setColor(Color.GRAY) .setStrokeWidth(32f) .setStrokeCap(Paint.Cap.ROUND) .commit() save() { rotate(Minutes() * 6f, cx, cy) drawLine(cx, cy, cx, cy - rad * 0.8f) } painter .setColor(Color.LTGRAY) .setStrokeWidth(16f) .setStrokeCap(Paint.Cap.ROUND) .commit() save() { rotate(Hour() * 30f, cx, cy) Paint. Draw calls Expressions
35. .commit() save() { rotate(Minutes() * 6f, cx, cy) drawLine(cx, cy, cx, cy - rad * 0.8f) } painter .setColor(Color.LTGRAY) .setStrokeWidth(16f) .setStrokeCap(Paint.Cap.ROUND) .commit() save() { rotate(Hour() * 30f, cx, cy) drawLine(cx, cy, cx, cy - rad / 2f) } painter.setColor(Color.WHITE).setStrokeWidth(4f).commit() drawLine( cx, cy, w / 2f + rad * sin(ContinuousSec() * (2 * Math.PI.toFloat() / 60f)), h / 2f - rad * cos(ContinuousSec() * (2 * Math.PI.toFloat() / 60f)), ) } } } } return rc.writer }
36. .commit() save() { rotate(Minutes() * 6f, cx, cy) drawLine(cx, cy, cx, cy - rad * 0.8f) } painter .setColor(Color.LTGRAY) .setStrokeWidth(16f) .setStrokeCap(Paint.Cap.ROUND) .commit() save() { rotate(Hour() * 30f, cx, cy) drawLine(cx, cy, cx, cy - rad / 2f) } painter.setColor(Color.WHITE).setStrokeWidth(4f).commit() drawLine( cx, cy, w / 2f + rad * sin(ContinuousSec() * (2 * Math.PI.toFloat() / 60f)), h / 2f - rad * cos(ContinuousSec() * (2 * Math.PI.toFloat() / 60f)), ) } } } } return rc.writer } Paint. Draw calls
37. Running in Launcher (Baklava+) Small Standalone Resizable Animating
38. Running in Launcher (Baklava+) Small Standalone Resizable Animating
39. Pick & Choose
40. Multiple use cases • Android Launcher Widgets • Server-driven UI • Cross-application UI • … and more
41. RemoteCompose : AndroidX libraries Create Play Creation Library (KMP) Compose Library Android View player Widgets (RemoteView) Compose player Compose desktop player Glance
42. RemoteCompose : Launcher Widgets Create Play Creation Library (KMP) Compose Library Android View player Widgets (RemoteView) Compose player Compose desktop player Glance Android 16+
43. RemoteCompose : Server-Driven UI Create Play Creation Library (KMP) Compose Library Android View player Widgets (RemoteView) Compose player Compose desktop player Glance
44. RemoteCompose : x-applications Create Play Creation Library (KMP) Compose Library Android View player Widgets (RemoteView) Compose player Compose desktop player Glance
45. Demos
46. Components
47. Components • Live Resize • Animation • Adaptive Layout • Accessibility support
48. Components • Live Resize • Animation • Adaptive Layout • Accessibility support
49. Desktop • Work in Progress
50. Desktop • Work in Progress
51. GraphicsLayer + scrollview
52. GraphicsLayer + scrollview
53. Shaders + native scrollview
54. Shaders + native scrollview
55. Graphics!
66. Outside the box…
67. • Create a RemoteDocument • Push it to an ESP32 • The ESP32 adds data • Send the result via bluetooth RemoteCompose Player Bluetooth Remote UI with… an ESP32 + RC Document Data ESP32
69. Ambient computing : NFC • RemoteDocuments stored on NFC cards • 1K NFC cards: ~700 bytes of payload • 4K, 8K cards exist RemoteDocument NFC Card
71. Ambient computing : QR code • … how much data can you fi t in a QR code ? • About 2Kb !
72. Where to fi nd it androidx.compose.remote • AndroidX libraries • In pre-alpha • Snapshots available on AndroidX.dev maven { url = uri("https://androidx.dev/snapshots/builds/14351402/artifacts/repository") } Feedback w elcom e!
73. Small is a feature! Thank you! Nicolas Roard John Hoford @camaelon.bsky.social @hoford.bsky.social
74. Videos