Enhanced EC Recommendations: Trustworthy Validation with Large Language Models for Two-Tower Model

Event: iThome Hello World Dev Conference
Speaker: Dan Chen

Transcript

2. Enhanced EC Recommendations: Trustworthy Validation with Large Language Models for Two-Tower Model EC Data Dev / Data Scientists Dan Chen
3. Dan LINE Taiwan EC Dev - Data Scientis Work Experience Side Project
4. 01 02 03 04 Evaluation Framework Offline & Online Evaluation LLM on Recommendation What is Trustworthy 05 Q&A CONTENT
5. Why it’s so important 01 What is Trustworthy
6. Element of trustworthy 特點項目文字 特點項目 Trustworthy 特點項目文字 特點項目 特點項目文字 特點項目
7. Four Perspective 特點項目文字 特點項目 Trustworthy Recommendation 特點項目文字 特點項目 特點項目文字 特點項目 Data Preparation Data Representation Recommendation Generation Performance Evaluation
8. How to Correctly Evaluate AI 02 Evaluation Framework
9. Two - Stage Recommendation system Brickmaster Scalable Scenario-wise KPI - Oriented Trustworthy
10. How to truly comprehensive understand performance Evaluation Framework (1/2)
11. How to truly comprehensive understand performance Evaluation Framework (1/2)
12. How to Correctly Evaluate AI 03 Offline & Online Evaluation
13. Key point to show how your algorithms can contribute to your business Offline Evaluation
14. Key point to show how your algorithms can contribute to your business Online Evaluation
15. Avoid pitfalls In Practice If experiment isn’t’ significant ?? Sample ratio mismatch ?? Novelty effect ?? Key point to show how your algorithms can contribute to your business A/B test
16. Case – EC Shop recommendation
17. 04 LLM On Recommendation
18. Recommendation with LLM - Feature Engineering: Text embedding generation - How to evaluate embedding (probing): RankMe / α-ReQ Metrincs
19. Recommendation with LLM - Feature Engineering: Text embedding generation - How to evaluate embedding (probing): RankMe / α-ReQ Metrincs
20. Evaluate & Challenge 05 Conclusion
21. Conclusion Business Value OpenAI, Claude, Gemini XGBoost or OpenSource 來源：https://zh.wikipedia.org/zh- tw/%E7%BE%8E%E5%9C%8B%E9%9A%8A%E9%95%B72%EF%BC%9A%E9%85%B7%E5%AF%9 2%E6%88%B0%E5%A3%AB 來源：https://images.app.goo.gl/HCygtJVtoPaU2KgX6
22. Conclusion & Challenge 1. Data Quality 2. Multiple – Metrics evaluation 3. Conduct A/B test Experiment 4. Human Perception Evaluation Challenge
23. Q&A 聯絡資訊 (Linkedin – Dan Chen)