user wanted a calendar. They got a conversation. ユーザーが欲しかったのはカレンダー、返ってきたのは「会話」だった。 👤User: "Book a table for 2 tomorrow at 7pm" 🤖Agent: "Okay, for what day?" 👤User: "Tomorrow" 🤖Agent: "What time?" 👤User: "7pm" 🤖Agent: "For how many people?" … 3
run somewhere else, another server, another company. They cannot touch your UI. They can only send messages. エージェントは多くの場合、外部サーバーや他社など、別の環境で動作しています。 クライアントの UI を直接操作することはできず、メッセージ（データ）を送信することしか できません。 * HTML(HyperText Markup Language) 5
Object Notation) that describes the UI. • • • LLMs are good at structured output Travels over any transport → A2A, AG-UI, SSE, WebSockets, MCP Your client renders it → with your own components Result: the agent picks the shape. You own security and styling. — A2UI：UI の構成を記述した JSON を送信 • • • LLM は構造化出力が得意 任意のトランスポート層に対応 → A2A、AG-UI、SSE、WebSocket、MCP など クライアント側で描画 → 自前のネイティブコンポーネントで安全にレンダリング 「UI の構造（形）」はエージェントが決定し、「セキュリティとスタイル」はクライアント側が完全にコント ロールする。 7
a framework: it is a protocol Not a replacement for HTML: for agent-generated UIs, not static sites Not a robust styling system: client controls styling with limited server-side styling support Not limited to web: works on mobile and desktop — • • • • フレームワークではない : あくまで UI データを受け渡すための「プロトコル（仕様）」 HTML の代替ではない : 静的な Web サイト向けではなく、AI エージェントが動的に生成する UI に特化 包括的なスタイリングシステムではない : 見た目（スタイル）の制御権はクライアント側にあり、 サーバー側のスタイル指定は最小限 Web 限定の技術ではない : モバイル（Android / iOS）やデスクトップでもネイティブに動作 8
endpoint in a bidirectional loop. It receives user inputs, UI events, and catalog capabilities from the renderer, and emits structured UI payloads and data updates. 双方向ループで動作する対話型エンドポイント。 レンダラーからカタログ仕様・UI イベント・ユーザー入力を受信し、構造化 UI ペイロード とデータ更新を送信する。 11
The client that actually draws. It tells the agent which catalogs it supports and how to use them. UI を実際に描画（レンダリング）するクライアント。 自身がサポートしているカタログと利用ルールをエージェントに提示する。 12
renderer says it first. 「何を描画できるか」を決めるのはレンダラー。クライアントが最初に宣言する。 • supportedCatalogIds the catalogs this client can render. このクライアントが描画可能なカタログ ID の一覧。 • inlineCatalogs a whole custom catalog, sent inline. inlineCatalogs：カスタムカタログの定義全体をインラインで直接送信・同梱。 13
Canvas Top-level UI boundary that encapsulates its own component tree and isolated Data Model under a unique surfaceId. 一意の surfaceId のもとで、専用のコンポーネントツリーと独立したデータモデル（Data Model） をカプセル化するトップレベルの UI 境界。 • • • Explicit Lifecycle: Created and initialized via createSurface, and completely unmounted/removed via deleteSurface. Canonical Root Container: Automatically instantiates the implicit Surface component (child: "root") where the "id": "root" component mounts. State Synchronization Boundary: When sendDataModel: true is enabled, synchronizes the entire surface-level Data Model with the owning agent upon user actions. 14
of components and functions pre-agreed upon by the agent and renderer. エージェントとレンダラー間で事前合意された、JSON Schema ベースのコンポーネントお よび関数の技術仕様。 • Baseline Scope (Basic Catalog) Ships with 18 standard components (Layout, Display, Input) and 15 built-in functions (Validation, Formatting, System context @index). 種の標準コンポーネント（Layout / Display / Input）と 15 種の組み込み関数（検証・ フォーマット・システムコンテキスト @index 等）を標準提供。 15
by the agent in conformance with catalog specifications. カタログの定義に従って、エージェントが動的に生成する個々のUI インスタンス • • • • Adjacency List Structure: Streamed as a flat array of components with unique ids rather than a rigid nested tree. Progressive Rendering: The renderer dynamically builds the tree around the top-level node (id: "root") as component definitions arrive. JSON Pointer-Based Data Binding: Synchronizes View-Model state via absolute paths (/path) and relative paths (path) within collection templates. Two-Way Binding & Interactions: Enables real-time local model updates for input components and dispatches user events (action). 19
agent says what a thing is. The renderer decides how it looks. エージェントが「意味（役割）」を伝え、レンダラー が「見た目（スタイル）」を決める。 • not supported: No version ever had color, fontSize or padding 非対応：どのバージョンでも color / fontSize / padding はサポート外 • meaning only: Left to the agent: variant, weight, align, direction. On Android, colour is your Material 3 theme. Nothing else. On a component: Text.variant v0.8 usageHint: "h1"…"h5", "body", "caption" v0.9 the same 7, renamed → variant v1.0 caption | body only. "h1 variant" → valid: false. headings move to Markdown "#" On a surface: theme v0.9.1 createSurface.theme { primaryColor, iconUrl, agentDisplayName } "not wired to use the theme" v1.0 removed. "Decoupled Branding: defer visual styling, entirely to the target framework’s native theme." 21
holds an address, not a value. コンポーネントは「値そのもの」ではなく「参照（パス）」を保持す る。 • {"text": "Optional"} a literal. Draw exactly this. • {"text": {"path": "/answers/comment"}} an address. Go read the data model at that spot. • {"text": {"call": "formatString", ...}} 固定値： 指定された文字列をその まま描画する。 アドレス ： データモデルの指定パ スから値を読み取って描画する 関数・式評価 ： クライアント関数を 実行し、その結果を描画する。 a question. Run it, and draw the answer. 22
server response has a fixed shape, so a data class parses it and the compiler checks it. サーバーのレスポンスは構造が固定されており、data class でパースしコンパイラが静的に検証できる。 Why It Does Not Fit The agent decides the shape at runtime. You cannot write a class for fields you have not seen. エージェントは実行時（Runtime）に構造を決定する。まだ 見ぬフィールドに対して、事前にクラスを定義することはで きない。 // the usual Android way @Serializable data class Order(val menu: List<Item>) val order = Json.decodeFromString<Order>(s) // A2UI — the same job, moved into the catalog A2uiComponentDefinition( name = "MenuItemRow", propertySchema = componentSchema( properties = mapOf( "name" to dynamicString("Dish name."), "quantity" to twoWay(/* ... */), "max" to dynamicNumber(/* ... */), ), required = setOf("name", "quantity"), ), ) // checked when a message arrives, not at build time The types did not vanish, they moved. The data model itself is one JsonElement reached by path; what each component may receive is declared here, and the renderer refuses a message that does not match, with a reason, at runtime. 23
whole data model of a surface is a single JsonElement in Compose state. サーフェスのデータモデル全体は、 Compose の State 内で単 一の JsonElement として保持・管理される。 Path, Not Field • • Reads and writes go through a JSON Pointer, so a path the app has never seen still works. 読み書きはすべて JSON Pointer を経由するため、アプリが事 前に知らないパスであってもそのまま動作する。 Recomposition Is the Redraw • • updateData replaces the state object, and Compose redraws whatever read it. updateData が State オブジェクトを置き換えることで、それを 参照していた UI のみを Compose が自動的に再描画する。 // SurfaceState.kt class SurfaceState(val surfaceId: String) { var dataModel: JsonElement by mutableStateOf<JsonElement>( JsonObject(emptyMap())) private set fun updateData(path: String, value: JsonElement?) { dataModel = JsonPointer.set(dataModel, path, value) } fun read(path: String): JsonElement? = JsonPointer.get(dataModel, path) } // the folded lines are one line each in the source Consequence: The renderer never needs to know the shape in advance. This lets the app handle unknown questions at runtime which is why every read is a cast. 24
"v1.0", "updateDataModel": { "surfaceId": "user_profile_card", "path": "/user/name", "value": "Jane Doe" } Provides new data to be inserted into or to replace a surface's data model. サーフェスのデータモデルに反映（挿入・ 置換）する最新データを渡す。 • • } path (string, optional): A JSON Pointer to the location in the data model to update. Defaults to /. value (any, required): The new value for the specified path. To delete the key/value at path, set value explicitly to null. 29
Dropping the message silently would hang a conversation, not protect it. 拒否（エラー）を返すことも一つの答え。不正なメッセージを無言で破棄してもシステムは守られず、 対話ループがハングするだけである。 The codes the spec defines VALIDATION_FAILED a property does not match the schema UNALLOWED_PARENT placed under a container that forbids it UNALLOWED_CHILD a container was given a child it forbids INVALID_FUNCTION_CALL rendererOnly, or not registered at all UNKNOWN_FUNCTION the agent does not have that one // renderer -> agent { "version": "v1.0", "error": { "code": "VALIDATION_FAILED", "surfaceId": "csat", "path": "/components/0/text", <WHERE it went wrong "message": "Expected a string, but got: 42" } } 31
Execution: JSON only. No script, no eval. Component Allowlist: A type the catalog does not declare is refused by name. Composition Constraints: allowedParents / allowedChildren → UNALLOWED_PARENT, UNALLOWED_CHILD. Declared by custom catalogs; the basic catalog uses neither. User Activation Required: requiresUserActivation confines a function to a real gesture, and forces allowedCallers: rendererOnly. • • • • コード実行の完全禁止 純粋な JSON のみ。スクリプト実行や eval は一 切排除。 コンポーネントのホワイトリスト制 カタログに宣言されていない型（Type）は名前で 即座に拒否。 階層・親子構造の制約 allowedParents / allowedChildren による不 正配置の検出（UNALLOWED_PARENT / UNALLOWED_CHILD）。 カスタムカタログで宣言可能（Basic Catalog で は未使用）。 ユーザー操作の必須化（副作用の防止） requiresUserActivation により関数の実行を 実際のタップ操作内に限定し、allowedCallers: rendererOnly を強制。 32
App External Server (Agents) modules modules modules LLM API A2UI Engine A2UI Utils LLM API Android App (Renderer) A2ui Renderer A2ui engine All logic executes within the client. (Includes all source code) 全ビジネスロジック・実行コードはクライア ント側に集約 A2UI Renderer Network (API etc) Module Only data and UI payloads are exchanged with the server. (Separated structure) サーバーとの通信は純粋なデータおよび宣言的 UI ペイロードに限定 36
team wants to know how a ticket felt. The trouble is that "how did we do" depends entirely on what they came for. A parcel three days late and an account someone was locked out of have no question in common worth asking. So the form is generated per ticket. Four tickets, four different sets of five questions, and androidx.a2ui,the real one, renders them. チケットごとにフォームを生成する。 配達遅延と アカウント凍結に、共通の質問は無い。 40
csatSystemPrompt() You write customer-satisfaction surveys … # How to emit UI surfaceId / catalogId / dataModel / components # What the app puts on the wire around it the three v0.9.1 messages it will be wrapped in # The shape of the form root -> form -> header, q+a pairs, more, submit # Catalog <- generated, not typed val schema = catalog.toJsonSchemaString() "$schema" every component and function, verbatim the catalog is both the allowlist the renderer enforces and the menu the model is given. カタログは、レンダラーが強制する「ホワ イトリスト」であり、モデルに与えられる 「メニュー」でもある。 # Style "no jargon, no on a scale of 1 to 5" 46
one prompt val agent = remember(settings.effectiveApiKey, settings.effectiveModel) { settings.newAgent() // Gemini | OpenAI | Anthropic } val systemPrompt = remember { csatSystemPrompt() } agent.streamUi( systemPrompt = systemPrompt, userPrompt = ticket.briefing(), surfaceId = CSAT_SURFACE_ID, applyUi = ::accept, ) // briefing() is just the ticket: // - Ticket: #4417 // - What happened: The courier missed the promised // Tuesday window, the tracking page kept showing // "out for delivery" for two days... // Ask about what happened on THIS ticket. 47
androidx.a2ui spec v0.9.1 // agent -> renderer : updateComponents.components[ ... ] {"id":"root","component":"Card","child":"form"}, {"id":"form","component":"Column", "children":["header","q1","q2","q3","q4","q5", "more","submit"]}, {"id":"header","component":"Text","text":"Ticket #4417 your order arrived three days after the window we promised."} form names eight children. None of them exist yet. eight updateComponents messages, in this order: 1 root, form, header 5 q4, a4 2 q1, a1 6 q5, a5 3 q2, a2 7 more 4 q3, a3 8 submit, submit_label from the prompt: "Nesting is by id only, never by embedding one component object inside another." 49
draws it { "id": "q1", "component": "Question", "text": "Overall, how satisfied are you with the way we handled this delay?", "required": true, "children": [ "a1" ] } val children = props.bindChildReferences(childrenProp) children?.forEach { reference -> key(reference.id, reference.baseDataPath) { when (val s = observeA2uiComponentState(reference)) { is A2uiComponentState.Loading -> Text("...") is A2uiComponentState.Error -> Text(s.exception.message ?: "error") is A2uiComponentState.Success -> A2uiComponent(component = s.component) } } } 50
androidx.a2ui spec v0.9.1 // One message. A flat array, not a tree. The renderer assembles the tree by following child / children. // ==== agent -> renderer : ONE updateComponents ========= {"version":"v0.9.1", "updateComponents":{ "surfaceId":"csat", "components":[ {"id":"submit","component":"Button", "child":"submit_label", <- points by id "variant":"primary", "action":{"event":{ "name":"submitCsat", "context":{ "ticketId":"4417", "answers":{"path":"/answers"} <- the whole branch }}}}, {"id":"submit_label","component":"Text", "text":"Send feedback"} ]}} <- the child, a sibling 52
for every message from this point on. Step What it exercises 1 Order Request createSurface carries the components and the dataModel v1.0 2 Quantity Change Two-way binding, committed locally. Compose draws the stepper Compose 3 Delivery Address checks run locally on every keystroke; the Button stays disabled 4 Delivery Check Inside the place_order handler: callAgentFunction, and it waits 5 Placing the Order Only if the check passed. The app writes the payment screen itself 6 Rider & Live Status updateDataModel only, the components were sent once 7 Review Survey Two star ratings, a comment, one Button. v1.0 · RPC v1.0 56
the renderer could send two messages: action and error. Neither returns a value, and neither carries an id. // v0.9.1 tap -> action{"name":"place_order"} the agent must send a whole new screen back, basket and all v0.9.1 でレンダラーが送信可能なメッセージは action と error の 2 種のみ 。いずれも戻り値を返さず、 ID も保持し ない。 // v1.0 tap -> callAgentFunction <- {"deliverable":false,"note":"..."} What Changes the app draws the note itself. Nothing is re-sent; the basket is untouched. The app keeps the screen it already has, and decides for itself what to draw next. アプリは現在の画面を保持し、次に何を描画するかを 自律的に決定する。 * RPC : Remote Procedure Call Calling a function that runs on the other side, and waiting for its value. 57
Example Agent → Renderer callRendererFunction rendererFunctionResponse Query Device Info The agent requests and retrieves client-side data, such as screen resolution or local sensor readings (getScreenResolution). callAgentFunction agentFunctionResponse Server-Side Validation The client performs real-time verification against the server (verifyProvider), such as validating a business registration ID or coupon code entered in a form. (Agent invokes client function) Renderer → Agent (Client invokes agent function) 59
the left is what those six built. Total is 0 and the button is dead: nothing chosen. agent -> renderer, spec v1.0 1 createSurface root Card + the whole dataModel 2 updateComponents col Column, hd Text 3 updateComponents list List -> row MenuItemRow 4 updateComponents total OrderTotalRow 5 updateComponents address TextField 6 updateComponents order Button {"version":"v1.0", "createSurface":{ "surfaceId":"turn1", "catalogId":"app.dining.catalog/v1", "components":[{"id":"root","component":"Card", "children":["col"]}], "dataModel":{ "menu":[{"name":"Margherita","price":14000, "quantity":0}, ...four dishes], "delivery":{"address":""}}}} 60
the device in this whole sequence 1. the tap: FilledTonalIconButton inside MenuItemRow onClick = { setQuantity(quantity + 1) }, 2. the write: local, into this row’s own path props["quantity"] is {"path":"quantity"} relative, so it lands on THIS row, not on every row scope.write(node.props["quantity"], JsonPrimitive(next.coerceIn(0, max))) 3. the data model changes /menu/0/quantity : 0 -> 1 63
OrderTotalRow re-reads its binding: {"id":"total", "component":"OrderTotalRow", "value":{"call":"formatCurrency","args":{ "value":{"call":"calcOrderTotal", "args":{"items":{"path":"/menu"}}}, "currency":"KRW","decimals":0}}} 5. calcOrderTotal runs again, in App. The agent never learns of the tap. Messages sent: 0. 64
: updateComponents.components[ ... ] {"id":"address","component":"TextField", "label":"Delivery address", "text":{"path":"/delivery/address"}, "checks":[{ "condition":{"call":"length","args":{ "value":{"path":"/delivery/address"},"min":5}}, "message":"Add a little more detail to the address"}]} {"id":"order","component":"Button","label":"Place order", "checks":[{ "condition":{"call":"numeric","args":{ "value":{"call":"countOrderItems", "args":{"items":{"path":"/menu"}}},"min":1}}, "message":"Add at least one dish"}], "action":{"name":"place_order","context":{ "total":{"call":"calcOrderTotal", /* ... */ }, "count":{"call":"countOrderItems", /* ... */ }, "address":{"path":"/delivery/address"}}}} // checks run in the renderer, on every keystroke. 65
failures.any { /* one line in the source */ it.severity == A2uiCheckSeverity.ERROR }, // BasicCatalog.kt "Button" — asks the WHOLE surface enabled = scope.surfaceIsSubmittable(), // BindingScope.kt fun surfaceIsSubmittable(): Boolean = surfaceCheckFailures() /* one line in the source */ .none { it.severity == A2uiCheckSeverity.ERROR } // A short address disables the Button too. Nothing leaves. 66
area = session.client.callAgentFunction( surfaceId = action.surfaceId, call = buildJsonObject { /* ... */ }, ).getOrNull() as? JsonObject // area IS that "value" object. Read its fields: val deliverable = (area?.get("deliverable") as? JsonPrimitive) ?.contentOrNull?.toBooleanStrictOrNull() /* ... val note, the same way ... */ if (deliverable == false) { say(note ?: "We do not deliver to that address.") return true // no payment screen } 68
Security & Client Control (クライアント主導の安全性) • • Pure declarative JSON, zero arbitrary code execution Visual styling 100% owned by Material 3 theme 2. Decoupled & Reactive State (構造とデータの完全分離) • • Separate UI structure and Data Model via JSON Pointer Instant local validation without network round-trips 3. An Open Protocol for Any AI Agent (エージェント接続の標準規約) • • A shared protocol, not a proprietary UI library Seamlessly connects Android to A2A, MCP, and WebSockets 70
モデルの進化 Modern AI models excel at Android development AndroidBench などのベンチマーク が示す通り、最新の AI モデルは Kotlin や Jetpack Compose の実装 能力が飛躍的に向上しています。 From manual coding to specification-guided synthesis 明確な技術仕様（ Specification）を与 えることで、高品質な Android ネイ ティブモジュールを AI と共に自律構 築できる時代になりました。 https://developer.android.com/bench 71
を自作する A2UI is an open, declarative specification A2UI はフレームワーク非依存であり、ワイヤー フォーマット、 Build your own renderer today with AI AIを活用すれば、仕様書をもとにパーサー、エンジ ン、Compose レンダラーを今すぐ自作・検証できま す。 72
itsmedreamwalker
shubox
noguhiro2002
kota_yata
izumin5210
satoshi256kbyte
kiakiraki
nogu66
kishida
musan
crysta1221
starfish719
contour_gara
kneath
moore
mongodb
caitiem20
marktimemedia
andyhume
schlessera
ryanjones
mfonobong
aleyda
katarinadahlin
garrettdimon
