Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[mercari GEARS 2025] Evals for LLMApps/Agents

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for mercari mercari PRO
November 14, 2025
Technology
110
0

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari

mercari PRO

November 14, 2025

More Decks by mercari

See All by mercari
[DevDojo] Getting Started with BI: Looker Essentials - 2025
Avatar for mercari mercari
PRO
0
120
[DevDojo] Introduction to LLMs & AI Agents - 2025
Avatar for mercari mercari
PRO
0
150
[mercari GEARS 2025] Techniques for Reliable Code Generation Using AI Agents
Avatar for mercari mercari
PRO
0
270
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Avatar for mercari mercari
PRO
0
250
[mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion
Avatar for mercari mercari
PRO
1
430
[mercari GEARS 2025] The Past, Present, and Future of Anti-Phishing Measures at Mercari
Avatar for mercari mercari
PRO
0
130
[mercari GEARS 2025] Backend Standardization with MCP
Avatar for mercari mercari
PRO
0
200
[mercari GEARS 2025] Transforming customer engagement with Google Customer Engagement Suite
Avatar for mercari mercari
PRO
0
340
[mercari GEARS 2025] PJ Aurora’s Vision and Automated UI Quality Evaluation Agents
Avatar for mercari mercari
PRO
0
330

Other Decks in Technology

See All in Technology
20260619 私の日常業務での生成 AI 活用
Avatar for Masaru Ogura masaruogura
1
130
RSA暗号を手計算したくなること、ありますよね？？ (20260615_orestudy6_rsa)
Avatar for Koki Senda thousanda
0
260
Oracle AI Database@Azure：サービス概要のご紹介
Avatar for oracle4engineer oracle4engineer
PRO
6
1.9k
FinOps × AIエージェントで実現する コストインシデントの自動調査
Avatar for T.REX oasis1994liveforever
0
130
失敗を経て、Harness Engineering で 大切にしたいことを考える / Learning from Failure: What Matters in Harness Engineering
Avatar for 株式会社ビットキー / Bitkey Inc. bitkey
PRO
1
320
Socrates × Looker　〜セマンティックレイヤーで進化するデータ分析エージェント〜
Avatar for Uchide Hiroki(ucchi-) hanon52_
3
2.1k
2026TECHFRESH畢業分享會 - Lightning Talk - 資料也要 CI/CD? 用 Airbyte 自動化資料同步
Avatar for LINE Developers Taiwan line_developers_tw
PRO
0
810
日本 Fintech 未来予測レポート 2027〜2028年（オリジナル版）
Avatar for 8maki 8maki
0
1.9k
Snowflakeと仲良くなる第一歩
Avatar for あべ coco_se
4
430
フロンティアAIのゲート化と地政学リスク
Avatar for 長津孝輔 nagatsu
0
130
タクシーアプリ『GO』の実践的データ活用
Avatar for GO Inc. dev mot_techtalk
3
190
2026TECHFRESH畢業分享會 - AI 時代的人生存檔點
Avatar for LINE Developers Taiwan line_developers_tw
PRO
0
830

Featured

See All Featured
Navigating Weather and Climate Data
Avatar for Ryan Abernathey rabernat
0
220
How to Think Like a Performance Engineer
Avatar for Harry Roberts csswizardry
28
2.6k
[RailsConf 2023] Rails as a piece of cake
Avatar for Vladimir Dementyev palkan
59
6.7k
Ethics towards AI in product and experience design
Avatar for Skipper Chong Warson skipperchong
2
310
Self-Hosted WebAssembly Runtime for Runtime-Neutral Checkpoint/Restore in Edge–Cloud Continuum
Avatar for Yuki Nakata chikuwait chikuwait
0
580
The Impact of AI in SEO - AI Overviews June 2024 Edition
Avatar for Aleyda Solis aleyda
5
1.1k
Amusing Abliteration
Avatar for ianozsvald ianozsvald
1
200
Ten Tips & Tricks for a 🌱 transition
Avatar for Manu Carrasco Molina stuffmc
0
130
The MySQL Ecosystem @ GitHub 2015
Avatar for Sam Lambert samlambert
251
13k
Scaling GitHub
Avatar for Zach Holman holman
464
140k
Measuring Dark Social's Impact On Conversion and Attribution
Avatar for Stephen Akadiri stephenakadiri
2
220
The Pragmatic Product Professional
Avatar for Laura Van Doore lauravandoore
37
7.3k

Transcript

  1. Evals for LLMApps/Agents
 Jehandad Kamal
 Mercari / Engineer @ AI/LLM

    Team


  2. Mercari AI Listing Support https://about.mercari.com/en/press/news/articles/20240910_aisupport/

  3. Agents


  4. Agent Development Loop


  5. First SDLC for Agents
 ai.mercari.com

  6. Types of Evals ( by technique )
 ai.mercari.com

  7. Types of Evals (by perspective )


  8. Improved SDLC for Agents


  9. Thank You!