Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[mercari GEARS 2025] Evals for LLMApps/Agents

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for mercari mercari PRO
November 14, 2025
Technology
120
0

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari

mercari PRO

November 14, 2025

More Decks by mercari

See All by mercari
[DevDojo] Getting Started with BI: Looker Essentials - 2025
Avatar for mercari mercari
PRO
0
140
[DevDojo] Introduction to LLMs & AI Agents - 2025
Avatar for mercari mercari
PRO
0
170
[mercari GEARS 2025] Techniques for Reliable Code Generation Using AI Agents
Avatar for mercari mercari
PRO
0
290
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Avatar for mercari mercari
PRO
0
280
[mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion
Avatar for mercari mercari
PRO
1
470
[mercari GEARS 2025] The Past, Present, and Future of Anti-Phishing Measures at Mercari
Avatar for mercari mercari
PRO
0
150
[mercari GEARS 2025] Backend Standardization with MCP
Avatar for mercari mercari
PRO
0
220
[mercari GEARS 2025] Transforming customer engagement with Google Customer Engagement Suite
Avatar for mercari mercari
PRO
0
390
[mercari GEARS 2025] PJ Aurora’s Vision and Automated UI Quality Evaluation Agents
Avatar for mercari mercari
PRO
0
370

Other Decks in Technology

See All in Technology
データ活用研修 データマネジメント【MIXI 26新卒技術研修】
Avatar for MIXI ENGINEERS mixi_engineers
PRO
4
770
コンポーネント名には何を含めるべきなのか？ / what-should-be-included-in-component-names
Avatar for airRnot airrnot1106
0
180
SRENEXT_2026_Chairs__Talks_in_Tamachi.sre.pdf
Avatar for srenext srenext
1
170
人依存からAIネイティブの体制へ：バックエンド開発の裏側【SORACOM Discovery 2026】
Avatar for SORACOM（ソラコム） soracom
PRO
0
130
全社でのソフトウェアサプライチェーン攻撃対策をやってみた with Takumi Guard
Avatar for kaita z63d
0
320
AIQAのナレッジ構築について
Avatar for tonchan qatonchan
1
120
AI時代におけるテストの基礎の再定義 / Rethinking the Fundamentals of Testing in the AI Era
Avatar for Matsu mineo_matsuya
15
6.1k
Retriever と Reranker、結局どうする？
Avatar for hkazuakey kazuaki
1
400
Claude Code並行開発環境の ムダ‧ムラ‧ムリを見直した話
Avatar for Muranaka muranakaaa
0
300
NetBoxを利用した作業効率化の試み_NetDevNight4
Avatar for Takumi Nohara tnoha
0
270
ダッシュボード"開発"について 〜使われるダッシュボードのつくりかた〜
Avatar for きみちゃん kimichan
0
240
最高のシステムプロンプトを作るためにフィードバック機能を導入した話
Avatar for Ryotaro Harada alchemy1115
0
220

Featured

See All Featured
16th Malabo Montpellier Forum Presentation
Avatar for AKADEMIYA2063 akademiya2063
PRO
0
300
WENDY [Excerpt]
Avatar for Tessa Abrams tessaabrams
11
39k
CSS Pre-Processors: Stylus, Less & Sass
Avatar for Bermon Painter bermonpainter
360
30k
The MySQL Ecosystem @ GitHub 2015
Avatar for Sam Lambert samlambert
251
13k
Producing Creativity
Avatar for Steve Smith orderedlist
PRO
348
40k
The Power of CSS Pseudo Elements
Avatar for Geoffrey Crofte geoffreycrofte
82
6.5k
職位にかかわらず全員がリーダーシップを発揮するチーム作り / Building a team where everyone can demonstrate leadership regardless of position
Avatar for MADOX madoxten
64
56k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
Avatar for Michelle Schulp Hunt marktimemedia
31
2.8k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
Avatar for Joe Casabona jcasabona
34
2.8k
Are puppies a ranking factor?
Avatar for Jono Alderson jonoalderson
1
3.7k
JAMstack: Web Apps at Ludicrous Speed - All Things Open 2022
Avatar for David Neal reverentgeek
1
510
Let's Do A Bunch of Simple Stuff to Make Websites Faster
Avatar for Chris Coyier chriscoyier
508
140k

Transcript

  1. Evals for LLMApps/Agents
 Jehandad Kamal
 Mercari / Engineer @ AI/LLM

    Team


  2. Mercari AI Listing Support https://about.mercari.com/en/press/news/articles/20240910_aisupport/

  3. Agents


  4. Agent Development Loop


  5. First SDLC for Agents
 ai.mercari.com

  6. Types of Evals ( by technique )
 ai.mercari.com

  7. Types of Evals (by perspective )


  8. Improved SDLC for Agents


  9. Thank You!