Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari mercari PRO
November 14, 2025
Technology
130
0

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari

mercari PRO

November 14, 2025

More Decks by mercari

See All by mercari
[DevDojo] Getting Started with BI: Looker Essentials - 2025
Avatar for mercari mercari
PRO
0
160
[DevDojo] Introduction to LLMs & AI Agents - 2025
Avatar for mercari mercari
PRO
0
200
[mercari GEARS 2025] Techniques for Reliable Code Generation Using AI Agents
Avatar for mercari mercari
PRO
0
310
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Avatar for mercari mercari
PRO
0
300
[mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion
Avatar for mercari mercari
PRO
1
500
[mercari GEARS 2025] The Past, Present, and Future of Anti-Phishing Measures at Mercari
Avatar for mercari mercari
PRO
0
160
[mercari GEARS 2025] Backend Standardization with MCP
Avatar for mercari mercari
PRO
0
240
[mercari GEARS 2025] Transforming customer engagement with Google Customer Engagement Suite
Avatar for mercari mercari
PRO
0
420
[mercari GEARS 2025] PJ Aurora’s Vision and Automated UI Quality Evaluation Agents
Avatar for mercari mercari
PRO
0
400

Other Decks in Technology

See All in Technology
[RSJ26] AnoleVLA: Lightweight Vision-Language-Action Model with Deep State Space Models for Mobile Manipulation
Avatar for Semantic Machine Intelligence Lab., Keio Univ. keio_smilab
PRO
0
120
Redmine 7.0で私が開発した新機能の狙いと背景
Avatar for MAEDA Go vividtone
1
110
なぜSRE・セキュリティは評価されないのか？守りの組織を事業成長エンジンに変えた実践
Avatar for サイバーセキュリティクラウド cscengineer
PRO
3
2.2k
Amazon S3 Tablesに全部任せてみた結果——コンパクション/スナップショット管理は本当に手放せるか
Avatar for ShigeruOda shigeruoda
0
380
深夜のクラウド懺悔室 1:29:300 or 1:0:0
Avatar for kazzpapa3 kazzpapa3
0
160
Backstageでつくるセルフサービスな社内開発基盤
Avatar for Sakazaki Kikunosuke kikunosuke75
1
290
多層防御と最⼩権限で実現する、安全なAIエージェント設計パターン
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
180
AIエージェントを雇う前に決める5つのこと
Avatar for 西岡 賢一郎 (Kenichiro Nishioka) knishioka
1
160
Beyond the Hype: Practical AI for Your Oracle Database with MCP
Avatar for thatjeffsmith thatjeffsmith
1
260
AI-DLCって実際どう？ 〜聞きたいこと全部聞いてみる〜
Avatar for 荒木 ARK news_it_enj
0
220
Tab5をRubyで動くパソコンにする
Avatar for Katsuhiko Kageyama kishima
1
180
コスト最適化の「めんどくさい」を AWS FinOps Agent でチョット楽にする
Avatar for kaz classmethod_kaz
0
170

Featured

See All Featured
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
128
56k
Ecommerce SEO: The Keys for Success Now & Beyond - #SERPConf2024
Avatar for Aleyda Solis aleyda
1
2.1k
The Psychology of Web Performance [Beyond Tellerrand 2023]
Avatar for Tammy Everts tammyeverts
49
3.5k
Distributed Sagas: A Protocol for Coordinating Microservices
Avatar for Caitie McCaffrey caitiem20
333
23k
Typedesign – Prime Four
Avatar for Hannes Fritz hannesfritz
42
3.2k
A Soul's Torment
Avatar for Nat Ma seathinner
7
3.5k
Design of three-dimensional binary manipulators for pick-and-place task avoiding obstacles (IECON2024)
Avatar for konakalab konakalab
0
570
Balancing Empowerment & Direction
Avatar for Lara Hogan lara
6
1.3k
State of Search Keynote: SEO is Dead Long Live SEO
Avatar for Ryan Jones ryanjones
0
260
Skip the Path - Find Your Career Trail
Avatar for Mark Kilby mkilby
1
220
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
Avatar for UX Y'all uxyall
1
390
Leading Effective Engineering Teams in the AI Era
Avatar for Addy Osmani addyosmani
9
2.5k

Transcript

  1. Evals for LLMApps/Agents
 Jehandad Kamal
 Mercari / Engineer @ AI/LLM

    Team


  2. Mercari AI Listing Support https://about.mercari.com/en/press/news/articles/20240910_aisupport/

  3. Agents


  4. Agent Development Loop


  5. First SDLC for Agents
 ai.mercari.com

  6. Types of Evals ( by technique )
 ai.mercari.com

  7. Types of Evals (by perspective )


  8. Improved SDLC for Agents


  9. Thank You!