AI Agentにおける評価指標とAgent GPA

第61回 MLOps 勉強会の発表です。AI Agentの評価指標であるAgent GPAの説明です。

Transcript

1. © 2026 Snowflake Inc. All Rights Reserved AI Agent における評価手法と Agent GPA Sho Tanaka Feb 2026
2. © 2026 Snowflake Inc. All Rights Reserved AI/ML, Dataの登壇やデモ開発を担当 - ex-Google gTech Ads, ML/Data - MLOps community 運営 (2020~) - Google Developer Expert, AI/ML tsho / 田中 翔 (Sho Tanaka) Linkedin.com/in/tsho Lead Developer Advocate @ Snowflake
3. © 2026 Snowflake Inc. All Rights Reserved AI Agentとは?
4. © 2026 Snowflake Inc. All Rights Reserved
5. © 2026 Snowflake Inc. All Rights Reserved AI Agent の活用事例 メルカリにおけるデータアナリティクス AI エージェント「Socrates」と ADK 活用 事例 - Speaker Deck コクヨ、ジンズなどがAIエージェント自社 開発 「Snowflake Intelligence」日本提 供
6. © 2026 Snowflake Inc. All Rights Reserved
7. © 2026 Snowflake Inc. All Rights Reserved AIOpsは2016年ごろにガー トナーが定義したものもあ るので注意
8. © 2026 Snowflake Inc. All Rights Reserved
9. © 2026 Snowflake Inc. All Rights Reserved 評価手法
10. © 2026 Snowflake Inc. All Rights Reserved AI Agent / LLM による代表的な評価指標
11. © 2026 Snowflake Inc. All Rights Reserved 例：ADK の評価指標 Why Evaluate Agents - Agent Development Kit (ADK) LLM-as-a-judge Final_response_match_v2, rubric_based_final_response_qual ity_v1 etc. Code-based / Deterministic コード・ルールベース/一致 tool_trajectory_avg_score Traditional NLP Metrics 従来の自然言語処理指標 response_match_score Human Evaluation 人間による評価 (機能として明示的な「指標」はない が、Web UI (Trace View) で支援)
12. © 2026 Snowflake Inc. All Rights Reserved Agent GPA と TruLens
13. © 2026 Snowflake Inc. All Rights Reserved
14. © 2026 Snowflake Inc. All Rights Reserved
15. © 2026 Snowflake Inc. All Rights Reserved
16. © 2026 Snowflake Inc. All Rights Reserved Agent GPA の論文 What Is Your Agent's GPA? A Framework for Evaluating Agent Goal-Plan-Action Alignment
17. © 2026 Snowflake Inc. All Rights Reserved OSS としても提供中 https://github.com/truera/trulens https://www.trulens.org/
18. © 2026 Snowflake Inc. All Rights Reserved https://www.trulens.org/getting_started/quickstarts/web-search-agent-evaluation/#10-add- evaluations
19. © 2026 Snowflake Inc. All Rights Reserved
20. © 2026 Snowflake Inc. All Rights Reserved さいごに
21. © 2026 Snowflake Inc. All Rights Reserved Snowflake 上で Private Preview として提供中 What’s Your Agent’s GPA? A Framework for Evaluating AI Agent Reliability
22. © 2026 Snowflake Inc. All Rights Reserved 参考
23. © 2026 Snowflake Inc. All Rights Reserved CS 329T: Trustworthy Machine Learning
24. © 2026 Snowflake Inc. All Rights Reserved https://learn.deeplearning.ai/
25. © 2026 Snowflake Inc. All Rights Reserved THANK YOU