Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari mercari PRO
November 14, 2025
Technology
0
57

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari

mercari PRO

November 14, 2025
Tweet

More Decks by mercari

See All by mercari
[DevDojo] Getting Started with BI: Looker Essentials - 2025
Avatar for mercari mercari
PRO
0
9
[DevDojo] Introduction to LLMs & AI Agents - 2025
Avatar for mercari mercari
PRO
0
3
[mercari GEARS 2025] Techniques for Reliable Code Generation Using AI Agents
Avatar for mercari mercari
PRO
0
45
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Avatar for mercari mercari
PRO
0
64
[mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion
Avatar for mercari mercari
PRO
1
180
[mercari GEARS 2025] The Past, Present, and Future of Anti-Phishing Measures at Mercari
Avatar for mercari mercari
PRO
0
43
[mercari GEARS 2025] Backend Standardization with MCP
Avatar for mercari mercari
PRO
0
60
[mercari GEARS 2025] Transforming customer engagement with Google Customer Engagement Suite
Avatar for mercari mercari
PRO
0
57
[mercari GEARS 2025] PJ Aurora’s Vision and Automated UI Quality Evaluation Agents
Avatar for mercari mercari
PRO
0
42

Other Decks in Technology

See All in Technology
変わるもの、変わらないもの ：OSSアーキテクチャで実現する持続可能なシステム
Avatar for gree_tech gree_tech
PRO
0
1.2k
"なるべくスケジューリングしない" を実現する "PreferNoSchedule" taint
Avatar for Kazuki Suda superbrothers
0
120
雲勉LT_Amazon Bedrock AgentCoreを知りAIエージェントに入門しよう！
Avatar for y-mae ymae
2
230
Flutter Thread Merge - Flutter Tokyo #11
Avatar for JaiChangPark itsmedreamwalker
1
110
Datadog LLM Observabilityで実現するLLMOps実践事例 / practical-llm-observability-with-datadog
Avatar for 逆井（さかさい） k6s4i53rx
0
180
Active Directory 勉強会 第 6 回目 Active Directory セキュリティについて学ぶ回
Avatar for Eureka eurekaberry
7
2.6k
機械学習を「社会実装」するということ 2025年冬版 / Social Implementation of Machine Learning November 2025 Version
Avatar for Moe Uchiike（内池 もえ） moepy_stats
4
690
Eight Engineering Unit 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
5.6k
AI時代のインシデント対応 〜時代を切り抜ける、組織アーキテクチャ〜
Avatar for Kazuto Kusama jacopen
4
170
段階的に進める、 挫折しない自宅サーバ入門
Avatar for yut yu_kod
3
830
組織の“見えない壁”を越えよ！エンタープライズシフトに必須な3つのPMの「在り方」変革 #pmconf2025
Avatar for Masakazu Inaba masakazu178
1
1k
Codeer.LowCode.Blazor　紹介と成長録
Avatar for wada-wada wadawada
0
100

Featured

See All Featured
YesSQL, Process and Tooling at Scale
Avatar for Rocio Delgado rocio
174
15k
ReactJS: Keep Simple. Everything can be a component!
Avatar for Pedro Nauck pedronauck
666
130k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
Avatar for Joe Casabona jcasabona
34
2.5k
Side Projects
Avatar for Sacha Greif sachag
455
43k
Git: the NoSQL Database
Avatar for Brandon Keepers bkeepers
PRO
432
66k
Rails Girls Zürich Keynote
Avatar for Gregor Martynus gr2m
95
14k
The Web Performance Landscape in 2024 [PerfNow 2024]
Avatar for Tammy Everts tammyeverts
11
940
Distributed Sagas: A Protocol for Coordinating Microservices
Avatar for Caitie McCaffrey caitiem20
333
22k
The Cult of Friendly URLs
Avatar for Andy Hume andyhume
79
6.7k
How to Think Like a Performance Engineer
Avatar for Harry Roberts csswizardry
28
2.3k
Documentation Writing (for coders)
Avatar for Carmen Chung carmenintech
76
5.1k
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
65
8k

Transcript

  1. Evals for LLMApps/Agents
 Jehandad Kamal
 Mercari / Engineer @ AI/LLM

    Team


  2. Mercari AI Listing Support https://about.mercari.com/en/press/news/articles/20240910_aisupport/

  3. Agents


  4. Agent Development Loop


  5. First SDLC for Agents
 ai.mercari.com

  6. Types of Evals ( by technique )
 ai.mercari.com

  7. Types of Evals (by perspective )


  8. Improved SDLC for Agents


  9. Thank You!