Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari mercari PRO
November 14, 2025
Technology
110
0

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari

mercari PRO

November 14, 2025

More Decks by mercari

See All by mercari
[DevDojo] Getting Started with BI: Looker Essentials - 2025
Avatar for mercari mercari
PRO
0
110
[DevDojo] Introduction to LLMs & AI Agents - 2025
Avatar for mercari mercari
PRO
0
140
[mercari GEARS 2025] Techniques for Reliable Code Generation Using AI Agents
Avatar for mercari mercari
PRO
0
250
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Avatar for mercari mercari
PRO
0
240
[mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion
Avatar for mercari mercari
PRO
1
410
[mercari GEARS 2025] The Past, Present, and Future of Anti-Phishing Measures at Mercari
Avatar for mercari mercari
PRO
0
130
[mercari GEARS 2025] Backend Standardization with MCP
Avatar for mercari mercari
PRO
0
190
[mercari GEARS 2025] Transforming customer engagement with Google Customer Engagement Suite
Avatar for mercari mercari
PRO
0
310
[mercari GEARS 2025] PJ Aurora’s Vision and Automated UI Quality Evaluation Agents
Avatar for mercari mercari
PRO
0
290

Other Decks in Technology

See All in Technology
形式手法特論：公平性制約の位相的特徴づけ #kernelvm / Kernel VM Study Kansai 12th
Avatar for y_taka_23 ytaka23
1
280
Anthropic AIネイティブ・スタートアップ構築のプレイブック を理解する
Avatar for 長津孝輔 nagatsu
0
190
【ハノーバーメッセ振り返りイベントat名古屋】データは集約からAI起点の収集に ～組織内・組織間でのデータ連携～
Avatar for 田中 聖也 tanakaseiya
0
120
管理アカウント単一運用からAWS Organizationsに移行するの大変で滅
Avatar for 平目 hiramax
0
250
TSKaigi 2026 - enumよ、さようなら
Avatar for teamLab teamlab
PRO
3
550
論文紹介：Pixal3D (SIGGRAPH 2026)
Avatar for TakatoYoshikawa tenten0727
0
740
大規模環境でどのように監視を実現する？
Avatar for Yuto Obayashi yuobayashi
1
150
はじめてのAI-DLC
Avatar for 吉田真吾 yoshidashingo
2
550
ビジュアルプログラミングIoTLT vol.23
Avatar for 1ft-seabass 1ftseabass
PRO
0
130
まだ道半ば、AI-DLCを歩み始めている話
Avatar for 荒木 ARK news_it_enj
2
180
20260528_生成AIを専属DSに_Howの次にすべきことを考える
Avatar for NobuakiOshiro doradora09
PRO
0
200
oracle-to-databricks-migration-with-llm-and-dbt
Avatar for case-k-git casek
0
160

Featured

See All Featured
Mind Mapping
Avatar for Hélio Medeiros helmedeiros
PRO
1
200
Un-Boring Meetings
Avatar for Sebastian Deterding codingconduct
0
300
Primal Persuasion: How to Engage the Brain for Learning That Lasts
Avatar for Mike Taylor tmiket
0
340
Building an army of robots
Avatar for Kyle Neath kneath
306
46k
Leo the Paperboy
Avatar for Maya Tellez mayatellez
7
1.8k
Building a Modern Day 
E-commerce SEO Strategy
Avatar for Aleyda Solis aleyda
45
9k
No one is an island. Learnings from fostering a developers community.
Avatar for Antonio thoeni
21
3.7k
Rails Girls Zürich Keynote
Avatar for Gregor Martynus gr2m
96
14k
Winning Ecommerce Organic Search in an AI Era - #searchnstuff2025
Avatar for Aleyda Solis aleyda
1
2k
The Cost Of JavaScript in 2023
Avatar for Addy Osmani addyosmani
55
9.9k
Kristin Tynski - Automating Marketing Tasks With AI
Avatar for Tech SEO Connect techseoconnect
PRO
0
250
The SEO Collaboration Effect
Avatar for Kristina Bergwall kristinabergwall1
1
460

Transcript

  1. Evals for LLMApps/Agents
 Jehandad Kamal
 Mercari / Engineer @ AI/LLM

    Team


  2. Mercari AI Listing Support https://about.mercari.com/en/press/news/articles/20240910_aisupport/

  3. Agents


  4. Agent Development Loop


  5. First SDLC for Agents
 ai.mercari.com

  6. Types of Evals ( by technique )
 ai.mercari.com

  7. Types of Evals (by perspective )


  8. Improved SDLC for Agents


  9. Thank You!