Operating an AI Infrastructure Through MCP Apps on Agents

Showcase for MCP Apps, You can manage complex tasks with prompts. Learning MCP Apps how it works, architecture and how to code it

Transcript

1. Operating an AI Infrastructure Through MCP Apps on Agents HyounKyoung (Jimmy) Moon AI Agentic Lead, Lablup
2. Speaker Introduction • 문현경 (Jimmy Moon) • 래블업(Lablup) AI Agentic Team Lead • 전) 토스(Toss) Frontend Platform Team • 전) Riiid Frontend Tech Leader • 전) Google Developer Experts for WebTech • Open Source Maintainer and Contributor (Yeoman, Polymer, Nextjs and more)
3. MCP Apps
4. Necessity Sometime, in case of wall of text problem, a picture is worth a thousand words
5. Generative UI UI whose content and composition are determined by an AI agent at runtime • Declarative: json-render, A2UI, agent returns a structured JSON schema - host renders its own native components • Open-ended: MCP Apps, raw HTML shipped as a resource / max expressiveness,
6. MCP Extension MCP Apps let servers return interactive HTML interfaces that render directly in the chat • Official MCP Extension, io.modelcontextprotocol/ui • Built on MCP-UI and the OpenAI Apps SDK • Authored by MCP core maintainers at OpenAI and Anthropic, with the MCP-UI creators
7. Use Cases • Data exploration and comparison — tables, charts, maps • Multi-option configuration — forms, panels • Real-time monitoring — dashboards, status cards • What-if analysis and simulation — interactive graphs • Multi-step workflows — steppers, approval panels • https://developers.openai.com/plugins/build/chatgpt-ui Interactive explanation — interactive diagrams, quiz cards
8. Features • Visual responses instead of text-only answers • Interactive UI instead of text-only output • Follow-up actions without leaving the chat • Tools available directly in the conversation
9. Showcases for declarative UI
10. Showcases for declarative UI
11. Showcases for open-ended UI
12. Showcases for open-ended UI
13. MCP Apps Architecture
14. Architecture: Tool, Resource, Host, and View • Tool: Declares UI Resource and returns Tool call result • Resource: Provides HTML, CSS, and JavaScript • Host: Calls Tool, loads Resource, and delivers the result • View: Runs Resource in a sandboxed iframe
16. Architecture: Tool • Declares UI Resource in _meta.ui.resourceUri • Returns one CallToolResult per call • Puts text for the LLM in content • Puts JSON for the View in structuredContent
18. Architecture: Resource • Bundles markup, styles, and script into one document • Serves it under ui://, ui://my-mcp-app • Declares MIME text/html;profile=mcp-app • Reaches no network: deny-all default CSP • Whitelist domains in _meta.ui.csp • Inlines every asset at build time
20. Architecture: Host • Calls Tool • Loads Resource • Creates the sandboxed iframe • Waits for the View, then delivers the result • Relays every message between View and server
21. ui/initialize to ui/resource-teardown
22. The ui/ methods, as the spec defines them
23. Architecture: View • Runs browser JavaScript, not a model • Reaches the Host through one app object • Exchanges MCP's JSON-RPC over window.postMessage • Receives the Tool call result • Calls Tools itself
24. Architecture: View
25. app calls: the server handlers they reach
26. Build the first MCP App for Backend.AI
27. Backend.AI -AI infrastructure, One unified OS • The most powerful containerized workload orchestrator for AI infrastructure, Sokovan • Precisely control GPU resources, 400% GPU utilization increase, 75% Infrastructure cost reduction • Manages 12+ AI accelerator types, including NVIDIA, Intel, AMD, Rebellions and FuriosaAI
28. Scaffolding the project
29. Wiring up the server
30. Wiring up the server
31. Wiring up the server
32. Wiring up the server
33. Wiring up the server
34. Wiring up the server
35. Calling the Backend.AI API
36. Calling the Backend.AI API
37. Calling the Backend.AI API
39. Registering the Tool, and its two payloads
40. Registering the Tool, and its two payloads
41. Registering the Tool, and its two payloads
42. Registering the Tool, and its two payloads
43. Registering the Tool, and its two payloads
44. Registering the Tool, and its two payloads
45. Registering the Tool, and its two payloads
46. Registering the Tool, and its two payloads
48. Writing the View
49. Writing the View
50. Writing the View
51. Writing the View
52. Writing the View
54. Bundling and registering the Resource
55. Bundling and registering the Resource
56. Bundling and registering the Resource
57. Bundling and registering the Resource
59. Running the MCP app in basic-host
62. Demo Backend.AI MCP Apps on Agents
63. Backend-AI GO – Run models on your own machine • Desktop AI app for people • One place for models, agents, and the work around them • Turn your office into a quiet little AI cluster • Powerful agents Workflows • MCP