Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AI Agentにおける評価指標とAgent GPA

Avatar for tsho tsho
February 26, 2026
Technology
440
1

AI Agentにおける評価指標とAgent GPA

第61回 MLOps 勉強会の発表です。AI Agentの評価指標であるAgent GPAの説明です。

Avatar for tsho

tsho

February 26, 2026

More Decks by tsho

See All by tsho
20260321_エンベディングってなに？RAGってなに？エンベディングの説明とGemini Embedding 2 の紹介
Avatar for tsho tsho
0
200
Scale SciPy with jax.shard_map - jax.shard_mapで SciPy をスケール
Avatar for tsho tsho
0
120
Developer Advocate / Community Managerなるには？
Avatar for tsho tsho
0
730
25/04/12 - Build with AI Hands-on Appendix
Avatar for tsho tsho
1
81
Unit testしてますか？
Avatar for tsho tsho
1
660

Other Decks in Technology

See All in Technology
Rapid Start: Faster Internet Connections, with Ruby's Help
Avatar for kazuho kazuho
2
820
AIコーディング時代における、ソフトウェアサプライチェーン攻撃に対する防衛術（簡易版）
Avatar for D soysoysoyb
0
160
Chasing Real-Time Observability for CRuby
Avatar for White-Green whitegreen
0
280
はじめての MagicPod生成AI機能 機能紹介から活用方法まで
Avatar for MagicPod magicpod
0
120
AIが書いたコードを信じられない問題 〜レビュー負荷を下げるために変えたこと〜 / The AI Code Trust Gap: Reducing the Review Burden
Avatar for 株式会社ビットキー / Bitkey Inc. bitkey
PRO
8
1.4k
GKE Agent SandboxでAIが生成したコードを 安全に実行してみた
Avatar for Lamaglama39 lamaglama39
0
110
[最強DB講義]推薦システム | 評価編
Avatar for RecSysLab recsyslab
PRO
0
110
20年前の「OSS革命」に学ぶ AI時代の生存戦略
Avatar for Sam Akada samakada
0
500
ハーネスエンジニアリングをやりすぎた話 ~そのハーネスは解体された~
Avatar for Gota gotalab555
5
1.9k
AI와 협업하는 조직으로의 여정
Avatar for Arawn Park arawn
0
530
データ定義の混乱と戦う 〜 管理会計と財務会計 〜
Avatar for wonohe wonohe
0
150
UIライブラリに依存しすぎないReact Native設計を目指して
Avatar for Takahiro Kato grandbig
0
150

Featured

See All Featured
Jamie Indigo - Trashchat’s Guide to Black Boxes: Technical SEO Tactics for LLMs
Avatar for Tech SEO Connect techseoconnect
PRO
0
110
Designing Dashboards & Data Visualisations in Web Apps
Avatar for Des Traynor destraynor
231
54k
Leveraging Curiosity to Care for An Aging Population
Avatar for Cassini Nazir cassininazir
1
230
New Earth Scene 8
Avatar for Sydney Stone popppiees
3
2.1k
Ruling the World: When Life Gets Gamed
Avatar for Sebastian Deterding codingconduct
0
210
Designing for Performance
Avatar for Lara Hogan lara
611
70k
Bootstrapping a Software Product
Avatar for Garrett Dimon garrettdimon
PRO
307
120k
The Curious Case for Waylosing
Avatar for Cassini Nazir cassininazir
0
320
Sharpening the Axe: The Primacy of Toolmaking
Avatar for Bryan Cantrill bcantrill
46
2.8k
How to train your dragon (web standard)
Avatar for Monica Dinculescu notwaldorf
97
6.6k
YesSQL, Process and Tooling at Scale
Avatar for Rocio Delgado rocio
174
15k
Navigating the moral maze — ethical principles for Al-driven product design
Avatar for Skipper Chong Warson skipperchong
2
340

Transcript

  1. © 2026 Snowflake Inc. All Rights Reserved AI Agent における評価手法と

    Agent GPA Sho Tanaka Feb 2026

  2. © 2026 Snowflake Inc. All Rights Reserved AI/ML, Dataの登壇やデモ開発を担当 -

    ex-Google gTech Ads, ML/Data - MLOps community 運営 (2020~) - Google Developer Expert, AI/ML tsho / 田中 翔 (Sho Tanaka) Linkedin.com/in/tsho Lead Developer Advocate @ Snowflake

  3. © 2026 Snowflake Inc. All Rights Reserved AI Agentとは?

  4. © 2026 Snowflake Inc. All Rights Reserved

  5. © 2026 Snowflake Inc. All Rights Reserved AI Agent の活用事例

    メルカリにおけるデータアナリティクス AI エージェント「Socrates」と ADK 活用 事例 - Speaker Deck コクヨ、ジンズなどがAIエージェント自社 開発 「Snowflake Intelligence」日本提 供

  6. © 2026 Snowflake Inc. All Rights Reserved

  7. © 2026 Snowflake Inc. All Rights Reserved AIOpsは2016年ごろにガー トナーが定義したものもあ るので注意

  8. © 2026 Snowflake Inc. All Rights Reserved

  9. © 2026 Snowflake Inc. All Rights Reserved 評価手法

  10. © 2026 Snowflake Inc. All Rights Reserved AI Agent /

    LLM による代表的な評価指標

  11. © 2026 Snowflake Inc. All Rights Reserved 例：ADK の評価指標 Why

    Evaluate Agents - Agent Development Kit (ADK) LLM-as-a-judge Final_response_match_v2, rubric_based_final_response_qual ity_v1 etc. Code-based / Deterministic コード・ルールベース/一致 tool_trajectory_avg_score Traditional NLP Metrics 従来の自然言語処理指標 response_match_score Human Evaluation 人間による評価 (機能として明示的な「指標」はない が、Web UI (Trace View) で支援)

  12. © 2026 Snowflake Inc. All Rights Reserved Agent GPA と

    TruLens

  13. © 2026 Snowflake Inc. All Rights Reserved

  14. © 2026 Snowflake Inc. All Rights Reserved

  15. © 2026 Snowflake Inc. All Rights Reserved

  16. © 2026 Snowflake Inc. All Rights Reserved Agent GPA の論文

    What Is Your Agent's GPA? A Framework for Evaluating Agent Goal-Plan-Action Alignment

  17. © 2026 Snowflake Inc. All Rights Reserved OSS としても提供中 https://github.com/truera/trulens

    https://www.trulens.org/

  18. © 2026 Snowflake Inc. All Rights Reserved https://www.trulens.org/getting_started/quickstarts/web-search-agent-evaluation/#10-add- evaluations

  19. © 2026 Snowflake Inc. All Rights Reserved

  20. © 2026 Snowflake Inc. All Rights Reserved さいごに

  21. © 2026 Snowflake Inc. All Rights Reserved Snowflake 上で Private

    Preview として提供中 What’s Your Agent’s GPA? A Framework for Evaluating AI Agent Reliability

  22. © 2026 Snowflake Inc. All Rights Reserved 参考

  23. © 2026 Snowflake Inc. All Rights Reserved CS 329T: Trustworthy

    Machine Learning

  24. © 2026 Snowflake Inc. All Rights Reserved https://learn.deeplearning.ai/

  25. © 2026 Snowflake Inc. All Rights Reserved THANK YOU