Slide 1

Slide 1 text

A2UI for Android Safely rendering AI-generated UI with Jetpack Compose Android 向け A2UI AI が生成した UI を Jetpack Compose で安全にレンダリングする Dreamwalker @jaichangpark 1

Slide 2

Slide 2 text

Dreamwalker Park Jai-Chang ● ● ● GDE (Google Developer Expert) Dart - Flutter GDG Golang Korea Flutter Seoul @jaichangpark 2

Slide 3

Slide 3 text

The problem: text is a slow way to ask The user wanted a calendar. They got a conversation. ユーザーが欲しかったのはカレンダー、返ってきたのは「会話」だった。 👤User: "Book a table for 2 tomorrow at 7pm" 🤖Agent: "Okay, for what day?" 👤User: "Tomorrow" 🤖Agent: "What time?" 👤User: "7pm" 🤖Agent: "For how many people?" … 3

Slide 4

Slide 4 text

A2UI A Protocol for Agent-Driven Interfaces A2UI (Agent to User Interface) is a declarative UI protocol for agent-driven interfaces. Agents generate rich, interactive UIs that render natively, without executing arbitrary code. A2UI は、エージェント駆動型インターフェース向けの宣言型 UI プロトコルです。 任意のコードを実行することなく、ネイティブにレンダリングされるリッチでインタラクティ ブな UI をエージェントが生成します。 4

Slide 5

Slide 5 text

Why not just send HTML? なぜ、単に HTML を送るだけではダメなのか? Agents often run somewhere else, another server, another company. They cannot touch your UI. They can only send messages. エージェントは多くの場合、外部サーバーや他社など、別の環境で動作しています。 クライアントの UI を直接操作することはできず、メッセージ(データ)を送信することしか できません。 * HTML(HyperText Markup Language) 5

Slide 6

Slide 6 text

Traditional approach Send HTML/JavaScript in iframes. ● ● ● Heavy, visually disjointed. Security complexity. Doesn't match app styling. iframe (WebView) で HTML / JavaScript を表示する場合の課題 ● ● ● パフォーマンス低下 & 視覚的な違和感 セキュリティリスクと実装の複雑化 ネイティブ UI スタイルとの不整合 MCP Apps: https://modelcontextprotocol.io/extensions/apps/overview 6

Slide 7

Slide 7 text

Safe like data, expressive like code A2UI: Send JSON (JavaScript Object Notation) that describes the UI. ● ● ● LLMs are good at structured output Travels over any transport → A2A, AG-UI, SSE, WebSockets, MCP Your client renders it → with your own components Result: the agent picks the shape. You own security and styling. — A2UI:UI の構成を記述した JSON を送信 ● ● ● LLM は構造化出力が得意 任意のトランスポート層に対応 → A2A、AG-UI、SSE、WebSocket、MCP など クライアント側で描画 → 自前のネイティブコンポーネントで安全にレンダリング 「UI の構造(形)」はエージェントが決定し、「セキュリティとスタイル」はクライアント側が完全にコント ロールする。 7

Slide 8

Slide 8 text

What A2UI Is NOT (Non-Goals) ● ● ● ● Not a framework: it is a protocol Not a replacement for HTML: for agent-generated UIs, not static sites Not a robust styling system: client controls styling with limited server-side styling support Not limited to web: works on mobile and desktop — ● ● ● ● フレームワークではない : あくまで UI データを受け渡すための「プロトコル(仕様)」 HTML の代替ではない : 静的な Web サイト向けではなく、AI エージェントが動的に生成する UI に特化 包括的なスタイリングシステムではない : 見た目(スタイル)の制御権はクライアント側にあり、 サーバー側のスタイル指定は最小限 Web 限定の技術ではない : モバイル(Android / iOS)やデスクトップでもネイティブに動作 8

Slide 9

Slide 9 text

A2UI vs Server-Driven UI (SDUI) 比較項目 従来の SDUI A2UI (Agent-to-UI) UI の決定主体 バックエンド / 開発者 (ビジネスロジック・ルールベース ) AI エージェント / LLM (ユーザーの意図を解釈して即座に判断 ) 生成メカニズム 決定論的 (Deterministic) 定義済みテンプレートの組み合わ せ 生成的 (Generative) 文脈に応じたリアルタイムな UI 構造の合成 主な目的 アプリ更新なしの UI 変更 (A/B テスト、パーソナライズ ) テキスト会話のボトルネック解消 (タップ・選択による直感的な操作性の提供 ) ユーザー操作の還 元先 通常のバックエンド API (DB 保存、画面遷移など ) エージェント推論ループ (操作結果を次のアクションへの入力として還 元) 9

Slide 10

Slide 10 text

Roadmap: Current Status ● ● ● ● V0.8 : 2025 Q4 (2025.11) ○ Initial public release ○ minimal support v0.9 : 2026 Q2 (2026.01) ○ Feature complete v0.9.1 : 2026.05 ○ Current stable release V1.0 : 2026 3Q ○ Candidate ○ Jetpack Compose Renderer (Android): ■ Native Compose UI components ■ Material Design 3 support ■ Android platform integration 10

Slide 11

Slide 11 text

Agent V0.9 : server → V1.0 : agent An interactive endpoint in a bidirectional loop. It receives user inputs, UI events, and catalog capabilities from the renderer, and emits structured UI payloads and data updates. 双方向ループで動作する対話型エンドポイント。 レンダラーからカタログ仕様・UI イベント・ユーザー入力を受信し、構造化 UI ペイロード とデータ更新を送信する。 11

Slide 12

Slide 12 text

Renderer (レンダラー ) V0.9 : client → V1.0 : renderer The client that actually draws. It tells the agent which catalogs it supports and how to use them. UI を実際に描画(レンダリング)するクライアント。 自身がサポートしているカタログと利用ルールをエージェントに提示する。 12

Slide 13

Slide 13 text

Capability negotiation ● Who decides what may be drawn, the renderer says it first. 「何を描画できるか」を決めるのはレンダラー。クライアントが最初に宣言する。 ● supportedCatalogIds the catalogs this client can render. このクライアントが描画可能なカタログ ID の一覧。 ● inlineCatalogs a whole custom catalog, sent inline. inlineCatalogs:カスタムカタログの定義全体をインラインで直接送信・同梱。 13

Slide 14

Slide 14 text

Surface : The Unit of Independent UI ● Isolated UI Canvas Top-level UI boundary that encapsulates its own component tree and isolated Data Model under a unique surfaceId. 一意の surfaceId のもとで、専用のコンポーネントツリーと独立したデータモデル(Data Model) をカプセル化するトップレベルの UI 境界。 ● ● ● Explicit Lifecycle: Created and initialized via createSurface, and completely unmounted/removed via deleteSurface. Canonical Root Container: Automatically instantiates the implicit Surface component (child: "root") where the "id": "root" component mounts. State Synchronization Boundary: When sendDataModel: true is enabled, synchronizes the entire surface-level Data Model with the owning agent upon user actions. 14

Slide 15

Slide 15 text

Catalog ● UI Design System Contract (Schema) JSON Schema-based specification of components and functions pre-agreed upon by the agent and renderer. エージェントとレンダラー間で事前合意された、JSON Schema ベースのコンポーネントお よび関数の技術仕様。 ● Baseline Scope (Basic Catalog) Ships with 18 standard components (Layout, Display, Input) and 15 built-in functions (Validation, Formatting, System context @index). 種の標準コンポーネント(Layout / Display / Input)と 15 種の組み込み関数(検証・ フォーマット・システムコンテキスト @index 等)を標準提供。 15

Slide 16

Slide 16 text

Basic Catalog : 18 Components ● ● ● Display / Media ○ Text, Image, Icon, Video, AudioPlayer Layout / Container ○ Row, Column, List, Card, Tabs, Divider Interactive / Input ○ Button: (primary, borderless) ○ Modal ○ TextField ○ CheckBox, ChoicePicker, Slider ○ DateTimeInput 16

Slide 17

Slide 17 text

Basic Catalog: Functions System Context: @index: 0-based index of current item during template rendering. Validation & Checks: ● ● ● ● ● required: Not null, undefined, or empty. email: Validates email format. regex: Matches regular expression pattern. length: String length constraints (min/max). numeric: Numeric range constraints (min/max). 17

Slide 18

Slide 18 text

Basic Catalog: Functions Formatting & String ● ● ● ● ● formatString: Performs string interpolation via data paths/functions (${...}). formatNumber: Applies grouping separators and precision. formatCurrency: Converts numeric values to currency strings. formatDate: Applies date/time patterns. pluralize: Selects localized strings based on numeric counts. Logical ● ● and/or: Performs logical AND/OR operations on boolean lists. not: Performs logical NOT on a boolean. Utility ● openUrl: Opens a URL (requires user activation). 18

Slide 19

Slide 19 text

Component ● UI Instances on Screen Individual UI nodes generated by the agent in conformance with catalog specifications. カタログの定義に従って、エージェントが動的に生成する個々のUI インスタンス ● ● ● ● Adjacency List Structure: Streamed as a flat array of components with unique ids rather than a rigid nested tree. Progressive Rendering: The renderer dynamically builds the tree around the top-level node (id: "root") as component definitions arrive. JSON Pointer-Based Data Binding: Synchronizes View-Model state via absolute paths (/path) and relative paths (path) within collection templates. Two-Way Binding & Interactions: Enables real-time local model updates for input components and dispatches user events (action). 19

Slide 20

Slide 20 text

20

Slide 21

Slide 21 text

Theme & styling: the agent never had colour ● The agent says what a thing is. The renderer decides how it looks. エージェントが「意味(役割)」を伝え、レンダラー が「見た目(スタイル)」を決める。 ● not supported: No version ever had color, fontSize or padding 非対応:どのバージョンでも color / fontSize / padding はサポート外 ● meaning only: Left to the agent: variant, weight, align, direction. On Android, colour is your Material 3 theme. Nothing else. On a component: Text.variant v0.8 usageHint: "h1"…"h5", "body", "caption" v0.9 the same 7, renamed → variant v1.0 caption | body only. "h1 variant" → valid: false. headings move to Markdown "#" On a surface: theme v0.9.1 createSurface.theme { primaryColor, iconUrl, agentDisplayName } "not wired to use the theme" v1.0 removed. "Decoupled Branding: defer visual styling, entirely to the target framework’s native theme." 21

Slide 22

Slide 22 text

The Data Model: structure and data are separate A component holds an address, not a value. コンポーネントは「値そのもの」ではなく「参照(パス)」を保持す る。 ● {"text": "Optional"} a literal. Draw exactly this. ● {"text": {"path": "/answers/comment"}} an address. Go read the data model at that spot. ● {"text": {"call": "formatString", ...}} 固定値: 指定された文字列をその まま描画する。 アドレス : データモデルの指定パ スから値を読み取って描画する 関数・式評価 : クライアント関数を 実行し、その結果を描画する。 a question. Run it, and draw the answer. 22

Slide 23

Slide 23 text

Where did the data class go? The Usual Way A server response has a fixed shape, so a data class parses it and the compiler checks it. サーバーのレスポンスは構造が固定されており、data class でパースしコンパイラが静的に検証できる。 Why It Does Not Fit The agent decides the shape at runtime. You cannot write a class for fields you have not seen. エージェントは実行時(Runtime)に構造を決定する。まだ 見ぬフィールドに対して、事前にクラスを定義することはで きない。 // the usual Android way @Serializable data class Order(val menu: List) val order = Json.decodeFromString(s) // A2UI — the same job, moved into the catalog A2uiComponentDefinition( name = "MenuItemRow", propertySchema = componentSchema( properties = mapOf( "name" to dynamicString("Dish name."), "quantity" to twoWay(/* ... */), "max" to dynamicNumber(/* ... */), ), required = setOf("name", "quantity"), ), ) // checked when a message arrives, not at build time The types did not vanish, they moved. The data model itself is one JsonElement reached by path; what each component may receive is declared here, and the renderer refuses a message that does not match, with a reason, at runtime. 23

Slide 24

Slide 24 text

One JsonElement, addressed by path Untyped Storage ● ● The whole data model of a surface is a single JsonElement in Compose state. サーフェスのデータモデル全体は、 Compose の State 内で単 一の JsonElement として保持・管理される。 Path, Not Field ● ● Reads and writes go through a JSON Pointer, so a path the app has never seen still works. 読み書きはすべて JSON Pointer を経由するため、アプリが事 前に知らないパスであってもそのまま動作する。 Recomposition Is the Redraw ● ● updateData replaces the state object, and Compose redraws whatever read it. updateData が State オブジェクトを置き換えることで、それを 参照していた UI のみを Compose が自動的に再描画する。 // SurfaceState.kt class SurfaceState(val surfaceId: String) { var dataModel: JsonElement by mutableStateOf( JsonObject(emptyMap())) private set fun updateData(path: String, value: JsonElement?) { dataModel = JsonPointer.set(dataModel, path, value) } fun read(path: String): JsonElement? = JsonPointer.get(dataModel, path) } // the folded lines are one line each in the source Consequence: The renderer never needs to know the shape in advance. This lets the app handle unknown questions at runtime which is why every read is a cast. 24

Slide 25

Slide 25 text

Defining actions: event or functionCall Agent Event Sent as an action message. No response message; the agent replies with UI. action メッセージとして送信。個別の応答メッセージ はなく、エージェントは UI で返信する。 functionCall Runs a catalog function. Returns a value, and starts no conversational turn. "action": { "event": { "name": "submit_form", "context": {"itemId": "123"}}} "action": { "functionCall": { "call": "openUrl", "args": {"url": "${/url}"}}} カタログ関数を実行(戻り値を返し、対話ターンは発 生しない) 25

Slide 26

Slide 26 text

Message Types Direction Message Renderer → Agent (4) ● ● action, error callAgentFunction, rendererFunctionResponse Agent → Renderer (6) ● createSurface, updateComponents, updateDataModel, deleteSurface callRendererFunction, agentFunctionResponse ● 26

Slide 27

Slide 27 text

The agent-to-renderer protocol defines 4 message types createSurface: Signals the renderer to create a new surface and begin rendering it. レンダラーに新規サーフェスの作成を通 知し、初期描画を開始。 { "version": "v1.0", "createSurface": { "surfaceId": "user_profile_card", "catalogId": "https://a2ui.org/specification/v1_0/catalogs/basic/catalog.json", "sendDataModel": true, "components": [ { "id": "root", "component": "Column", "children": ["user_name"] }, { "id": "user_name", "component": "Text", "text": {"path": "/name"} } ], "dataModel": { "name": "John Doe" } } } 27

Slide 28

Slide 28 text

The agent-to-renderer protocol defines 4 message types updateComponents { "version": "v1.0", "updateComponents": { "surfaceId": "user_profile_card", "components": [ { "id": "root", "component": "Column", "children": ["user_name", "user_title"] }, { "id": "user_name", "component": "Text", "text": "John Doe" }, { "id": "user_title", "component": "Text", "text": "Software Engineer" } ] } Provides a list of component definitions to be added to or updated in a specific surface. 対象サーフェスに配置するコンポーネント 構造の定義リストを送信。 } 28

Slide 29

Slide 29 text

The agent-to-renderer protocol defines 4 message types updateDataModel { "version": "v1.0", "updateDataModel": { "surfaceId": "user_profile_card", "path": "/user/name", "value": "Jane Doe" } Provides new data to be inserted into or to replace a surface's data model. サーフェスのデータモデルに反映(挿入・ 置換)する最新データを渡す。 ● ● } path (string, optional): A JSON Pointer to the location in the data model to update. Defaults to /. value (any, required): The new value for the specified path. To delete the key/value at path, set value explicitly to null. 29

Slide 30

Slide 30 text

The agent-to-renderer protocol defines 4 message types createSurface, updateComponents, updateDataModel, deleteSurface ● deleteSurface Explicitly removes a surface and its contents from the UI. サーフェスおよび保持していた全状態を UI から完全に削除・クリーンアップ。 30

Slide 31

Slide 31 text

When the renderer says no A refusal is an answer. Dropping the message silently would hang a conversation, not protect it. 拒否(エラー)を返すことも一つの答え。不正なメッセージを無言で破棄してもシステムは守られず、 対話ループがハングするだけである。 The codes the spec defines VALIDATION_FAILED a property does not match the schema UNALLOWED_PARENT placed under a container that forbids it UNALLOWED_CHILD a container was given a child it forbids INVALID_FUNCTION_CALL rendererOnly, or not registered at all UNKNOWN_FUNCTION the agent does not have that one // renderer -> agent { "version": "v1.0", "error": { "code": "VALIDATION_FAILED", "surfaceId": "csat", "path": "/components/0/text",

Slide 32

Slide 32 text

Why this is safe ● ● ● ● No Code Execution: JSON only. No script, no eval. Component Allowlist: A type the catalog does not declare is refused by name. Composition Constraints: allowedParents / allowedChildren → UNALLOWED_PARENT, UNALLOWED_CHILD. Declared by custom catalogs; the basic catalog uses neither. User Activation Required: requiresUserActivation confines a function to a real gesture, and forces allowedCallers: rendererOnly. ● ● ● ● コード実行の完全禁止 純粋な JSON のみ。スクリプト実行や eval は一 切排除。 コンポーネントのホワイトリスト制 カタログに宣言されていない型(Type)は名前で 即座に拒否。 階層・親子構造の制約 allowedParents / allowedChildren による不 正配置の検出(UNALLOWED_PARENT / UNALLOWED_CHILD)。 カスタムカタログで宣言可能(Basic Catalog で は未使用)。 ユーザー操作の必須化(副作用の防止) requiresUserActivation により関数の実行を 実際のタップ操作内に限定し、allowedCallers: rendererOnly を強制。 32

Slide 33

Slide 33 text

Transport Layer The Transport Contract (4 Core Requirements) ● Transport-Agnostic Decouples protocol semantics and JSON structure from the underlying network layer. ● ● ● Reliable & Ordered Delivery: Enforces strict sequential processing to prevent UI state corruption. Message Framing: Delimits discrete envelope messages via JSONL newlines, SSE events, or WebSocket frames. Metadata & Bidirectional Channel: Transports metadata for capability negotiation / sendDataModel sync, with a return channel for actions and RPCs. 33

Slide 34

Slide 34 text

Transport Layer Supported Bindings & Ecosystem ● AG-UI & A2A Standard bindings for real-time frontend integration and agent-to-agent UI delegation. リアルタイムなフロントエンド統合およびエージェント間(Agent-to-Agent)の UI 委譲を実現する標準バインディン グ。 ● MCP (Model Context Protocol Renders dynamic UI via LLM tool calls and resource subscriptions. LLM のツール呼び出し(Tool Calls)やリソース購読(Resource Subscriptions)を介して動的 UI を描画。 ● Web Standards Full compatibility with WebSockets (streaming), SSE + JSON-RPC, and HTTP REST. WebSockets(ストリーミング)、SSE + JSON-RPC、HTTP REST との完全な互換性。 34

Slide 35

Slide 35 text

Protocol overview & data flow 35 https://a2ui.org/specification/v1.0-a2ui/#protocol-overview-data-flow

Slide 36

Slide 36 text

Option 1: Embedded Implementation (In-App) Option 2: Server Based Android App External Server (Agents) modules modules modules LLM API A2UI Engine A2UI Utils LLM API Android App (Renderer) A2ui Renderer A2ui engine All logic executes within the client. (Includes all source code) 全ビジネスロジック・実行コードはクライア ント側に集約 A2UI Renderer Network (API etc) Module Only data and UI payloads are exchanged with the server. (Separated structure) サーバーとの通信は純粋なデータおよび宣言的 UI ペイロードに限定 36

Slide 37

Slide 37 text

Access Date: 2026-08-20 https://developer.android.com/jetpack/androidx/releases/a2ui 37

Slide 38

Slide 38 text

Inside androidx.a2ui ● ● 2026-08-21 a2ui spec v0.9.1 a2ui ○ a2ui-model ○ a2ui-engine ○ compose ■ compose-runtime ■ compose-ui ■ compose-ui-testing compose ○ material3 ■ material3-a2ui 38

Slide 39

Slide 39 text

Layer Module Responsibility Protocol a2ui-model Wire format, JSON Pointer parsing, and JSON Schema definitions 通信フォーマット(Wire format)、JSON Pointer 解析、JSON Schema 定義 Engine a2ui-engine Message Pipeline & Validation, Coroutine Actor queue, validation boundary, and dynamic expression evaluation Coroutine Actor キュー、検証境界(バリデーション)、動的式の評価 State compose-runtime JSON → Snapshot State mapping, progressive streaming, and data bindings JSON → Snapshot State マッピング、プログレッシブストリーミング、データバ インディング Contract compose-ui A2uiComponent interfaces, component scopes, and catalog contracts A2uiComponent インターフェース、コンポーネントスコープ、カタログ契約 Testing compose-ui-testing Test controller, message assertions, and component stubbing テストコントローラー、メッセージ検証(アサーション)、コンポーネントスタブ Renderer material3-a2ui Native Material 3 UI widgets (Button, Card, Slider, List, etc.) ネイティブ Material 3 UI ウィジェット(Button、Card、Slider、List など) 39

Slide 40

Slide 40 text

Let's build it Jetpack Compose と androidx.a2ui で実際に作る A support team wants to know how a ticket felt. The trouble is that "how did we do" depends entirely on what they came for. A parcel three days late and an account someone was locked out of have no question in common worth asking. So the form is generated per ticket. Four tickets, four different sets of five questions, and androidx.a2ui,the real one, renders them. チケットごとにフォームを生成する。 配達遅延と アカウント凍結に、共通の質問は無い。 40

Slide 41

Slide 41 text

The catalog is the allowlist 2026-09-03 androidx.a2ui spec v0.9.1 const val SUPPORT_CATALOG_ID = "app.support.androidx/csat/v1" val AndroidxSupportCatalog: A2uiCatalog = A2uiCatalog( catalogId = SUPPORT_CATALOG_ID, components = listOf( // From material3-a2ui, unmodified. MaterialA2uiBasicCatalogV1Defaults.card, MaterialA2uiBasicCatalogV1Defaults.column, MaterialA2uiBasicCatalogV1Defaults.text, MaterialA2uiBasicCatalogV1Defaults.button, MaterialA2uiBasicCatalogV1Defaults.checkBox, MaterialA2uiBasicCatalogV1Defaults.slider, // Written in this app. QuestionComponent, // AndroidxSurveyComponents.kt StarRatingComponent, // AndroidxSurveyComponents.kt ChoicePickerComponent, // AndroidxSurveyComponents.kt TextFieldComponent, // AndroidxSharedComponents.kt ), functions = listOf( A2uiRequiredFunction.INSTANCE, A2uiNumericFunction.INSTANCE, A2uiFormatStringFunction.INSTANCE, ), ) 41

Slide 42

Slide 42 text

Before: Material3 independently owned property keys (No cross-renderer contract) 以前:各レンダラー(Material 3 等)が個別にプロパティ名を保持し、プラットフォーム横断の共通仕様 (Contract)が欠如していた public object MaterialButtonComponent : A2uiComponent { private val childProp = A2uiProperty.componentId("child") // Ad-hoc property definition private val actionProp = A2uiProperty.action("action") override fun A2uiComponentScope.Content(properties: A2uiComponentProperties, modifier: Modifier) { /* Manual property parsing & direct Compose Button rendering */ } } 42

Slide 43

Slide 43 text

After: Clean separation between Protocol Contract (Interface) & Design System (Impl) 現在:プロトコル規約(Interface)とデザインシステム(Impl)の完全な分離 // [1] Contract Layer (compose-ui): Enforces spec properties & automates parsing public interface A2uiBasicCatalogV1.Button : A2uiComponent { override fun Content(properties, modifier) { // Automatically parses spec properties and delegates to a type-safe contract TypedContent(childId, variant, action, modifier) } fun TypedContent( childId: String, variant: ButtonVariant, action: Action, modifier: Modifier ) } // [2] Implementation Layer (material3-a2ui) Purely renders UI for the design system internal object MaterialA2uiBasicCatalogV1Button : A2uiBasicCatalogV1.Button { override fun TypedContent(childId: String, variant: ButtonVariant, action: Action, modifier: Modifier) { // Renders Material 3 Button (Easily swappable with your own custom Design System!) } } 43

Slide 44

Slide 44 text

object TextFieldComponent : A2uiComponent { @Composable override fun A2uiComponentScope.Content( properties: A2uiComponentProperties, modifier: Modifier, ) { Column(modifier) { OutlinedTextField( value = text, TextFieldComponent 2026-08-21 androidx.a2ui spec v0.9.1 onValueChange = { touched = true; onTextChange?.invoke(it) }, enabled = onTextChange != null, isError = showError, singleLine = true, label = label?.let { { Text(it) } }, placeholder = placeholder?.let { { Text(it) } }, modifier = Modifier.fillMaxWidth(), ) if (showError) { Text( text = error, style = MaterialTheme.typography.bodySmall, color = MaterialTheme.colorScheme.error, modifier = Modifier.padding(start = 16.dp, top = 4.dp), ) } } } } 44

Slide 45

Slide 45 text

2026-08-21 androidx.a2ui spec v0.9.1 val processor = remember { A2uiMessageProcessor(catalogs = listOf(AndroidxSupportCatalog)) } val parser = remember { A2uiMessageParser() } val surfaces by processor.activeSurfaces.collectAsStateWithLifecycle() LaunchedEffect(run) { // The engine loop. It stays on the composition // dispatcher deliberately. launch { processor.collectMessages() } launch { processor.outboundEvents.collect(::onEvent) } } surfaces.firstOrNull { it.id == CSAT_SURFACE_ID }?.let { A2uiSurface(surfaceModel = it, modifier = Modifier.fillMaxWidth()) } 45

Slide 46

Slide 46 text

System prompt structure 2026-08-21 androidx.a2ui spec v0.9.1 // CsatPrompt.kt — csatSystemPrompt() You write customer-satisfaction surveys … # How to emit UI surfaceId / catalogId / dataModel / components # What the app puts on the wire around it the three v0.9.1 messages it will be wrapped in # The shape of the form root -> form -> header, q+a pairs, more, submit # Catalog <- generated, not typed val schema = catalog.toJsonSchemaString() "$schema" every component and function, verbatim the catalog is both the allowlist the renderer enforces and the menu the model is given. カタログは、レンダラーが強制する「ホワ イトリスト」であり、モデルに与えられる 「メニュー」でもある。 # Style "no jargon, no on a scale of 1 to 5" 46

Slide 47

Slide 47 text

2026-08-21 androidx.a2ui spec v0.9.1 1 The tap one ticket becomes one prompt val agent = remember(settings.effectiveApiKey, settings.effectiveModel) { settings.newAgent() // Gemini | OpenAI | Anthropic } val systemPrompt = remember { csatSystemPrompt() } agent.streamUi( systemPrompt = systemPrompt, userPrompt = ticket.briefing(), surfaceId = CSAT_SURFACE_ID, applyUi = ::accept, ) // briefing() is just the ticket: // - Ticket: #4417 // - What happened: The courier missed the promised // Tuesday window, the tracking page kept showing // "out for delivery" for two days... // Ask about what happened on THIS ticket. 47

Slide 48

Slide 48 text

2 An empty surface, and every answer pre-filled 2026-08-21 androidx.a2ui spec v0.9.1 // ==== agent -> renderer ============================ { "version": "v0.9.1", "createSurface": { "surfaceId": "csat", "catalogId": "app.support.androidx/csat/v1", "sendDataModel": true } } { "version":"v0.9.1", "updateDataModel":{ "surfaceId":"csat", "path":"/", "value":{ "answers":{ "overall":0, "estimate":0, "condition":"", "wanted":[], "comment":"" }}}} <- StarRating <- StarRating <- ChoicePicker, one of <- ChoicePicker, several <- TextField 48

Slide 49

Slide 49 text

3 The form grows, one message at a time 2026-08-21 androidx.a2ui spec v0.9.1 // agent -> renderer : updateComponents.components[ ... ] {"id":"root","component":"Card","child":"form"}, {"id":"form","component":"Column", "children":["header","q1","q2","q3","q4","q5", "more","submit"]}, {"id":"header","component":"Text","text":"Ticket #4417 your order arrived three days after the window we promised."} form names eight children. None of them exist yet. eight updateComponents messages, in this order: 1 root, form, header 5 q4, a4 2 q1, a1 6 q5, a5 3 q2, a2 7 more 4 q3, a3 8 submit, submit_label from the prompt: "Nesting is by id only, never by embedding one component object inside another." 49

Slide 50

Slide 50 text

2026-08-21 androidx.a2ui spec v0.9.1 4 A question arrives and Compose draws it { "id": "q1", "component": "Question", "text": "Overall, how satisfied are you with the way we handled this delay?", "required": true, "children": [ "a1" ] } val children = props.bindChildReferences(childrenProp) children?.forEach { reference -> key(reference.id, reference.baseDataPath) { when (val s = observeA2uiComponentState(reference)) { is A2uiComponentState.Loading -> Text("...") is A2uiComponentState.Error -> Text(s.exception.message ?: "error") is A2uiComponentState.Success -> A2uiComponent(component = s.component) } } } 50

Slide 51

Slide 51 text

2026-08-21 androidx.a2ui spec v0.9.1 5 The customer answers, nothing leaves the device object StarRatingComponent : A2uiComponent { private val valueProp = A2uiProperty.dynamicNumber("value", required = true) private val maxProp = A2uiProperty.number("max") @Composable override fun A2uiComponentScope.Content( properties: A2uiComponentProperties, modifier: Modifier, ) { val value = properties.bind(valueProp)?.toInt() ?: return val onValueChange = properties.bindUpdater(valueProp) val max = properties[maxProp]?.toInt() ?: 5 Row(modifier, horizontalArrangement = Arrangement.spacedBy(4.dp)) { for (star in 1..max) { IconButton( onClick = { onValueChange?.invoke(star) }, // a literal binding leaves this null: read-only enabled = onValueChange != null, ) { Icon(/* code */) } } } } } 51

Slide 52

Slide 52 text

6 Submit : one event, carrying the whole subtree 2026-08-21 androidx.a2ui spec v0.9.1 // One message. A flat array, not a tree. The renderer assembles the tree by following child / children. // ==== agent -> renderer : ONE updateComponents ========= {"version":"v0.9.1", "updateComponents":{ "surfaceId":"csat", "components":[ {"id":"submit","component":"Button", "child":"submit_label", <- points by id "variant":"primary", "action":{"event":{ "name":"submitCsat", "context":{ "ticketId":"4417", "answers":{"path":"/answers"} <- the whole branch }}}}, {"id":"submit_label","component":"Text", "text":"Send feedback"} ]}} <- the child, a sibling 52

Slide 53

Slide 53 text

2026-08-21 androidx.a2ui spec v0.9.1 6 Submit : The tap // androidx.a2ui, A2uiActionHandler.handleAction suspend fun handleAction( action: A2uiUserAction, executionContext: A2uiCoreExecutionContext, ) { var currentAction: A2uiUserAction = action for (interceptor in actionInterceptors) { currentAction = interceptor.onInterceptAction(currentAction) } ?: return // <- cancelled, no event when (currentAction) { is A2uiEventAction -> { /* code */ } is A2uiFunctionCallAction -> { /* runs locally */ } } } 53

Slide 54

Slide 54 text

2026-08-21 androidx.a2ui spec v0.9.1 6 Submit : The tap // A2uiCoreDynamicEvaluator.tryProcessPathNode val path = mapNode[KEY_PATH] as? String if (path == null || mapNode.size != 1) return PathNodeProcessingResult.NOT_MATCHED // <- one key, named "path" val resolvedPath = dataPath / path // dataPath = "/" val result = executionContext.resolveValue(resolvedPath) ?: return PathNodeProcessingResult.FAILED resultStack.add(result) return PathNodeProcessingResult.PROCESSED // // // "ticketId":"4417" {"path":"/answers"} -> Literal -> not a map, passes through -> Path -> the /answers BRANCH {overall=4, wanted=[...], ...} 54

Slide 55

Slide 55 text

2026-08-21 androidx.a2ui spec v0.9.1 6 Submit : The event object is built // A2uiActionHandler, continued is A2uiEventAction -> { val resolvedContext = resolvePayloadMap(executionContext, action.context) val eventMessage = A2uiClientEventMessage( type = currentAction.eventName, // "submitCsat" surfaceId = currentAction.surfaceId, // "csat" componentId = currentAction.componentId, timestamp = currentAction.timestamp, context = resolvedContext, // VALUES now clientDataModel = clientDataModelProvider(), ) emitToServer(eventMessage) // an object } 55

Slide 56

Slide 56 text

What v1.0 changes Screen: A food delivery order Protocol: v1.0 for every message from this point on. Step What it exercises 1 Order Request createSurface carries the components and the dataModel v1.0 2 Quantity Change Two-way binding, committed locally. Compose draws the stepper Compose 3 Delivery Address checks run locally on every keystroke; the Button stays disabled 4 Delivery Check Inside the place_order handler: callAgentFunction, and it waits 5 Placing the Order Only if the check passed. The app writes the payment screen itself 6 Rider & Live Status updateDataModel only, the components were sent once 7 Review Survey Two star ratings, a comment, one Button. v1.0 · RPC v1.0 56

Slide 57

Slide 57 text

When the renderer needs an answer The Limitation In v0.9.1 the renderer could send two messages: action and error. Neither returns a value, and neither carries an id. // v0.9.1 tap -> action{"name":"place_order"} the agent must send a whole new screen back, basket and all v0.9.1 でレンダラーが送信可能なメッセージは action と error の 2 種のみ 。いずれも戻り値を返さず、 ID も保持し ない。 // v1.0 tap -> callAgentFunction <- {"deliverable":false,"note":"..."} What Changes the app draws the note itself. Nothing is re-sent; the basket is untouched. The app keeps the screen it already has, and decides for itself what to draw next. アプリは現在の画面を保持し、次に何を描画するかを 自律的に決定する。 * RPC : Remote Procedure Call Calling a function that runs on the other side, and waiting for its value. 57

Slide 58

Slide 58 text

What actually got better ● No Redraw The agent sends no screen back ● ● ● ● 画面の再描画なし(No Redraw) エージェントは画面(UI)を送り返さない。 ● 対話ターンの発生なし(No Conversational Turn) 不要な LLM 推論ループを回さず、軽量なデータ のみを送受信。 ● エラーの即時可視化(Failure Is Visible) 配送不可や検証失敗などの結果を、その場でダ イアログや警告として直ちに表示。 ● 判断はクライアント側で完結(The Decision Stays Here) 次のアクション(決済へ進むか、入力を促すか)の 制御権・決定権はアプリ自身が保持する。 58 No Conversational Turn Failure Is Visible The Decision Stays Here

Slide 59

Slide 59 text

Bidirectional RPC Function Calls Direction Request Message Response Message Practical Example Agent → Renderer callRendererFunction rendererFunctionResponse Query Device Info The agent requests and retrieves client-side data, such as screen resolution or local sensor readings (getScreenResolution). callAgentFunction agentFunctionResponse Server-Side Validation The client performs real-time verification against the server (verifyProvider), such as validating a business registration ID or coupon code entered in a form. (Agent invokes client function) Renderer → Agent (Client invokes agent function) 59

Slide 60

Slide 60 text

The order screen arrives, in six messages The screen on the left is what those six built. Total is 0 and the button is dead: nothing chosen. agent -> renderer, spec v1.0 1 createSurface root Card + the whole dataModel 2 updateComponents col Column, hd Text 3 updateComponents list List -> row MenuItemRow 4 updateComponents total OrderTotalRow 5 updateComponents address TextField 6 updateComponents order Button {"version":"v1.0", "createSurface":{ "surfaceId":"turn1", "catalogId":"app.dining.catalog/v1", "components":[{"id":"root","component":"Card", "children":["col"]}], "dataModel":{ "menu":[{"name":"Margherita","price":14000, "quantity":0}, ...four dishes], "delivery":{"address":""}}}} 60

Slide 61

Slide 61 text

"MenuItemRow" to { node, scope, _ -> // Agent-controlled numbers, so clamp before they bound anything. val max = scope.readFloat(node.props["max"], 20f).toInt().coerceIn(1, 99) val quantity = scope.readFloat(node.props["quantity"], 0f).toInt().coerceIn(0, max) fun setQuantity(next: Int) { scope.write(node.props["quantity"], JsonPrimitive(next.coerceIn(0, max))) } Row( modifier = Modifier.fillMaxWidth().padding(vertical = 6.dp), verticalAlignment = Alignment.CenterVertically, /* ... */ ) { Column(Modifier.weight(1f)) { Text( scope.readString(node.props["name"]), style = MaterialTheme.typography.bodyLarge, color = if (quantity > 0) MaterialTheme.colorScheme.primary else MaterialTheme.colorScheme.onSurface, ) /* ... note, priceLabel, the − button ... */ FilledTonalIconButton( onClick = { setQuantity(quantity + 1) }, enabled = quantity < max, 61

Slide 62

Slide 62 text

From onClick to the total, without the network Tap 62

Slide 63

Slide 63 text

From onClick to the total, without the network nothing leaves the device in this whole sequence 1. the tap: FilledTonalIconButton inside MenuItemRow onClick = { setQuantity(quantity + 1) }, 2. the write: local, into this row’s own path props["quantity"] is {"path":"quantity"} relative, so it lands on THIS row, not on every row scope.write(node.props["quantity"], JsonPrimitive(next.coerceIn(0, max))) 3. the data model changes /menu/0/quantity : 0 -> 1 63

Slide 64

Slide 64 text

From onClick to the total, without the network 4. recomposition. OrderTotalRow re-reads its binding: {"id":"total", "component":"OrderTotalRow", "value":{"call":"formatCurrency","args":{ "value":{"call":"calcOrderTotal", "args":{"items":{"path":"/menu"}}}, "currency":"KRW","decimals":0}}} 5. calcOrderTotal runs again, in App. The agent never learns of the tap. Messages sent: 0. 64

Slide 65

Slide 65 text

The address, and the button unlocks // agent -> renderer : updateComponents.components[ ... ] {"id":"address","component":"TextField", "label":"Delivery address", "text":{"path":"/delivery/address"}, "checks":[{ "condition":{"call":"length","args":{ "value":{"path":"/delivery/address"},"min":5}}, "message":"Add a little more detail to the address"}]} {"id":"order","component":"Button","label":"Place order", "checks":[{ "condition":{"call":"numeric","args":{ "value":{"call":"countOrderItems", "args":{"items":{"path":"/menu"}}},"min":1}}, "message":"Add at least one dish"}], "action":{"name":"place_order","context":{ "total":{"call":"calcOrderTotal", /* ... */ }, "count":{"call":"countOrderItems", /* ... */ }, "address":{"path":"/delivery/address"}}}} // checks run in the renderer, on every keystroke. 65

Slide 66

Slide 66 text

// BasicCatalog.kt "TextField", reports only its own failure isError = failures.any { /* one line in the source */ it.severity == A2uiCheckSeverity.ERROR }, // BasicCatalog.kt "Button" — asks the WHOLE surface enabled = scope.surfaceIsSubmittable(), // BindingScope.kt fun surfaceIsSubmittable(): Boolean = surfaceCheckFailures() /* one line in the source */ .none { it.severity == A2uiCheckSeverity.ERROR } // A short address disables the Button too. Nothing leaves. 66

Slide 67

Slide 67 text

Is this address in range? Delivery check, before the payment screen // renderer -> agent // inside the place_order handler {"version":"v1.0","callAgentFunction":{ "surfaceId":"turn1", "functionCallId":"fc-7", "callFunction":{"call":"check_delivery_area", "args":{"address":"Shibuya 2-21-1"}}}} // agent -> renderer : same id comes back {"version":"v1.0","agentFunctionResponse":{ "functionCallId":"fc-7", "value":{ "deliverable":true, "etaMinutes":35 } }} 67

Slide 68

Slide 68 text

// DiningFlow.kt — Apps waits. An event could not. val area = session.client.callAgentFunction( surfaceId = action.surfaceId, call = buildJsonObject { /* ... */ }, ).getOrNull() as? JsonObject // area IS that "value" object. Read its fields: val deliverable = (area?.get("deliverable") as? JsonPrimitive) ?.contentOrNull?.toBooleanStrictOrNull() /* ... val note, the same way ... */ if (deliverable == false) { say(note ?: "We do not deliver to that address.") return true // no payment screen } 68

Slide 69

Slide 69 text

Payments Live Tracking Delivered 69

Slide 70

Slide 70 text

Key Takeaways: The 3 Core Pillars of A2UI 1. Native Security & Client Control (クライアント主導の安全性) ● ● Pure declarative JSON, zero arbitrary code execution Visual styling 100% owned by Material 3 theme 2. Decoupled & Reactive State (構造とデータの完全分離) ● ● Separate UI structure and Data Model via JSON Pointer Instant local validation without network round-trips 3. An Open Protocol for Any AI Agent (エージェント接続の標準規約) ● ● A shared protocol, not a proprietary UI library Seamlessly connects Android to A2A, MCP, and WebSockets 70

Slide 71

Slide 71 text

AI Era: AI Models & Android Bench AndroidBench と AI モデルの進化 Modern AI models excel at Android development AndroidBench などのベンチマーク が示す通り、最新の AI モデルは Kotlin や Jetpack Compose の実装 能力が飛躍的に向上しています。 From manual coding to specification-guided synthesis 明確な技術仕様( Specification)を与 えることで、高品質な Android ネイ ティブモジュールを AI と共に自律構 築できる時代になりました。 https://developer.android.com/bench 71

Slide 72

Slide 72 text

Build from Spec with AI 仕様書から AI で自作する : androidx.a2ui を自作する A2UI is an open, declarative specification A2UI はフレームワーク非依存であり、ワイヤー フォーマット、 Build your own renderer today with AI AIを活用すれば、仕様書をもとにパーサー、エンジ ン、Compose レンダラーを今すぐ自作・検証できま す。 72

Slide 73

Slide 73 text

Thank you A2UI for Android Safely rendering AI-generated UI with Jetpack Compose @jaichangpark, Dream Walker GDE Dart-Flutter 73