Data Agents Need an Agent-facing Data Plane

https://opendatacircle.org/events/data-agent-meetup-1

Transcript

1. Data Agents Need an Agent-facing Data Plane A reference map for evaluating and evolving data platforms for AI agents Xiao Zhiyan – Open Data Circle May 26th, 2026
2. Chongqing, China 8D Magic City Hot Pot The Chinese University of Hong Kong Mathematics Information Engineering LY Corporation Software Engineer Streaming Data Pipeline Spark Iceberg Open Data Circle Data x AI x Rust AI-native Data Systems Chongqing Hong Kong Tokyo Xiao Zhiyan @xiaozhiyan
3. What This Talk Is (and Is Not) What this talk is What this talk is not • A reference map • An evolution path • A starting point for implementation • AI Agent 101 • Vendor ranking • Final standard
4. AI agents are moving from demos to real systems The Demo Phase Demos can work with static prompts and toy data. Real Systems Requirement Real systems need changing data, permissions, operational constraints, and action paths.
5. Agents are not just another dashboard user Human-facing analytics Agent-facing workflows • Query • Visualize • Interpret • Decide manually • Retrieve • Reason • Decide • Act through tools The access pattern changes — so the data platform requirements change.
6. The bottleneck shifts from model capability to data readiness  Model Capability Data Readiness Freshness Semantics Multimodality Governance Actionability Observability Better models are not enough if the data is stale, ambiguous, fragmented, ungoverned, hard to act on, or impossible to trace.
7. • Dashboards • BI • Analysts • Manual decisions • APIs • Services • Scheduled jobs • Pipelines Agent-facing • Context • Policy • Tool access • Trace • Action loop Existing data platforms are powerful — but not agent-facing by design Human-facing Application-facing The problem is not replacement. The problem is re-organization into an agent-facing data plane.
8. Freshness Current state, events, and recent changes Semantics Meaning, schema, ownership, and relationships Multimodality Tables, documents, vectors, logs, images, and events Governance Permissions, lineage, provenance, and policy control Actionability Safe workflows, write-back, and feedback loops Observability Trace from data to retrieval, reasoning, and action Six dimensions of Data Agent readiness These dimensions define a comprehensive framework for evaluating if data is truly ready for autonomous agents.
9. Reference Map: Agent-facing Data Plane DATA AGENTS AGENT-FACING DATA PLANE EXISTING DATA SYSTEMS FEEDBACK LOOP Retrieve Reason Act Freshness Semantics Multimodality Governance Actionability Observability Lakehouse / DWH Streaming / CDC Vector DB / Search Catalog / Metadata Governance / IAM Workflow / Apps Observe Feedback Improve
10. How existing systems fit into the map Capability Implementation Pattern Examples Freshness Streaming / CDC / incremental updates Flink, Kafka, Fluss, Debezium Semantics Catalog / metadata / data products Iceberg, DataHub, OpenLineage Multimodality Tables / documents / vectors / search Lakehouse, Lance, Milvus, Elasticsearch Governance IAM / policy / lineage / provenance RBAC, ABAC, audit logs Actionability Workflow / tools / write-back Airflow, Temporal, APIs, apps Observability Tracing / monitoring / evaluation OpenTelemetry, logs, feedback signals
11. Evolution Path: From data platform to agent-facing data plane 0. Existing Data Platform Humans, applications, and pipelines consume data. 1. Accessible Agent-accessible Data Agents can query selected data through tools and APIs. 2. Aware Agent-aware Data Plane Freshness, semantics, lineage, policy, and context are exposed. 3. Operable Agent-operable Data Plane Agents can trigger workflows, write back results, and leave traces. 4. Self-improving Self-improving Data Loop Agent behavior and failures improve metadata, indexing, quality, and routing. You do not need to rebuild the platform. You need a staged path to reorganize it.
12. A Minimal Starting Point for Implementation Infrastructure & Knowledge Execution & Governance Insight Start with one agent, one domain, one workflow — and make the loop traceable. Continuous Feedback Loop Data Sources Tables / documents / logs / events Context & Index Vector index / metadata / freshness / quality signals Agent Tools SQL query / retrieval / metadata lookup / workflow trigger Control Plane Permission check / policy guardrail / audit log / human approval Observability Prompt / tool call / retrieved data / action / outcome
13. Avoid building a fragile agent on top of invisible data Data & Context Risks Control & Operation Risks Stale context Agents answer from outdated snapshots or cached retrieval results. Ambiguous semantics Agents see columns, chunks, or documents without business meaning. Over-permissi oned tools Agents can access or trigger more than they should. No action boundary Write-back and workflow triggers happen without approval or constraints. No traceability Teams cannot inspect what data was used, why, and what happened. A working demo is not the same as a reliable data agent system.
14. Use the map to evaluate, position, prototype, and evolve Evaluate your platform Which capabilities are already covered? Where are the weakest links? Position technologies Where do lakehouse, streaming, vector DB, metadata, governance, and AI platforms fit? Prototype safely Start with one agent, one data domain, one workflow, and full traceability. Evolve step by step Move from agent-accessible data to agent-aware and agent-operable data planes. The map is useful only if it helps teams make better platform decisions.
15. Data Agents need more than data access — they need a data plane Reference Map Evaluate where your platform stands. Evolution Path Move from agent-accessible to agent-aware and agent-operable. Minimal Starting Point Start with one agent, one domain, one workflow, and full traceability. Today’s meetup continues the conversation: Architecture Agent-facing data plane AI Platform Alibaba Cloud, Qwen, Wan Data Plane Infrastructure Fluss + Lance Find gaps, start small, and evolve your data platform for real-world agents.
16. Thank You!