Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Web界隈LT会_爆速改善_Interleaving.pdf
Search
Yuya Matsumura
July 06, 2018
Science
870
1
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Web界隈LT会_爆速改善_Interleaving.pdf
A/Bテストより10-100倍効率的と言われるランキング評価手法、Interleavingについて、ザックリと概要をお話ししました。
Yuya Matsumura
July 06, 2018
Transcript
17 slides
©2018 Wantedly, Inc. രվળʂInterleaving ABςετΑΓޮతͳϥϯΩϯάධՁΛٻΊͯ Webք۾LTձ 6.Jul.2018 - Yuya Matsumura
- @yu-ya4
©2018 Wantedly, Inc. •Yuya Matsumuraʢদଜ ༏ʣ •Wantedly, Inc. (ࣾձਓ4ϲ݄) •From
Kyoto •Interested in Information Retrieval & Recommendation Self Introduction @yu-ya4
©2018 Wantedly, Inc. ABςετͯ͠·͔͢ʁ
©2018 Wantedly, Inc. ͋Δಛఆͷظؒʹ2ύλʔϯͷࢪࡦΛ༻ҙ͠ɼͲͪΒͷ ࢪࡦͷํ͕ΑΓߴ͍ޮՌ͕ಘΒΕΔ͔Λݕূ͢Δ͜ͱ ABςετͱ WantedlyͰɼجຊతʹͯ͢ͷมߋʹରͯ͠ABςετΛߦ͔ͬͯΒ ͷਖ਼ࣜϦϦʔεΛߦ͍ͬͯ·͢ʂ
©2018 Wantedly, Inc. Wantedlyʹ͓͚ΔϥϯΩϯάͷABςετ A/B Testing ̍ ̎ ̍
̎ Group A Ranking A Control Variation Group B Ranking B
©2018 Wantedly, Inc. ༗ҙͳ͕ࠩग़Δ·Ͱʹ ABςετͷ՝ →PDCAαΠΫϧΛճ͢ͷ͕͘ͳΔʂ Ex. *ݱࡏCVR 1%ͷͷΛ10%վળ(CVR 1.1%)͢Δʹ134,379ͷαϯϓϧ͕ඞཁ
*༗ҙਫ४Ћ=5%, ݕग़ྗ1-β=95%(https://keisan.casio.jp/exec/user/1489063696) େྔͷαϯϓϧ(impression, PV)͕ඞཁ
©2018 Wantedly, Inc.
©2018 Wantedly, Inc. ͦ͜ͰInterleaving
©2018 Wantedly, Inc. 2ͭͷϥϯΩϯά͔Βަޓʹݕࡧ݁ՌΛ࠷্Ґ͔Βॱʹ બ͠1ͭͷϥϯΩϯάΛੜͯ͠Ϣʔβʹఏࣔ͢Δ Interleavingͱ ̍ ̎ ̍
̎ Ranking A Ranking B ̍ ̎ ̍ ̎ Ranking C ᶃ ᶄ ᶅ ᶆ
©2018 Wantedly, Inc. Interleavingͷੑೳ →രͰPDCAαΠΫϧ͕ճͤΔʂ *ਖ਼֬ʹʮΠϯλʔϦʔϏϯάA/BςετΑΓ100ഒޮతͳ߹͕͋Γɺ͓͓Αͦͷ߹ɺ 10~100ഒޮతͰɺA/BςετΑΓޮతͰͳ͍߹গͳ͍ʯ(Chapelle et al. 2012)
Interleaving ABςετΑΓ10ʙ100ഒޮత*
©2018 Wantedly, Inc.
©2018 Wantedly, Inc. Interleaving͕ABςετΑΓޮతͳཧ༝ ̍ ̎ ̍ ̎
Ranking A Ranking B A/B Testing ̍ Ranking A ͱ Ranking Bͷੑೳ͕ࣅ͍ͯΔ߹ ͕ࠩग़ʹ͍͘ → ̍ ̍ ̍ ̍ ̍
©2018 Wantedly, Inc. Interleaving͕ABςετΑΓޮతͳཧ༝ Interleaving ̍ Ranking A ͱ Ranking
B͕҉ʹൺֱ͞ΕΔͷͰ ͕ࠩग़͍͢ʂʂ → ̍ ̍ ̍ ̍ ̍ ̎ ̍ ̎ ̍
©2018 Wantedly, Inc. ࣮ࡍͷABςετͱInterleavingͷp-valueͷऩଋͷൺֱ (Wantedly VisitͷϥϯΩϯάվળʹͯܭଌ)
©2018 Wantedly, Inc. ͦͷ݁Ռ…
©2018 Wantedly, Inc. ৽ଔೖࣾ1ϲ݄ͰWantedly Visitͷ ৽͍͠ϥϯΩϯάࢪࡦΛຊ൪ϦϦʔε (ࠓͰܧଓͯ͠ྑ͍݁ՌΛग़͠ଓ͚͍ͯ·͢)
©2018 Wantedly, Inc. InterleavingΛ׆༻ͯ͠രϥϯΩϯάվળΛ
More Decks by Yuya Matsumura
See All by Yuya Matsumura
事業会社における 機械学習・推薦システム技術の活用事例と必要な能力 / ml-recsys-in-layerx-wantedly-2026
yuya4
0
270
「気づいたら仕事が終わっている」バクラクAIエージェント本番運用の裏側 / layerx-bakuraku-aie2026
yuya4
22
17k
「気づいたら仕事が終わっている」体験を目指す Ambient Agent と Document AI の実践事例 / layerx-ai-deim2026
yuya4
0
1.7k
バクラクにおける Document Understanding の挑戦:書類の「読取」から「意思決定」へ / document-understanding-in-bakuraku-2026
yuya4
1
360
AIの誤りが許されない業務システムにおいて“信頼されるAI” を目指す / building-trusted-ai-systems
yuya4
6
5.2k
生成AI時代におけるAI・機械学習技術を用いたプロダクト開発の深化と進化 /evolving-ai-ml-products-in-genai-era
yuya4
7
2.8k
バックオフィス向け toB SaaS バクラクにおけるレコメンド技術活用 / recommender-systems-in-layerx-bakuraku
yuya4
5
1.2k
LayerX バクラクの目指す『業務の完全自動運転』/ layerx-bakuraku-fully-auto-driving-202504
yuya4
3
1.8k
LayerXにおけるAI活用事例とその裏側(2025年2月) バクラクの目指す “業務の自動運転” の例 / layerx-ai-deim2025
yuya4
5
7.2k
Other Decks in Science
See All in Science
AI(人工知能)の過去・現在・未来 ~AIは人類を越えるのか~
tagtag
PRO
0
190
Biohub - Cell Tracking During Development - 3rd Place Solution
yu4u
3
690
O(log n)-Approximation Algorithms for Bipartiteness Ratio
tasusu
0
320
ゲームと人工知能
miyayou
0
300
JPA2026_NetworkSymposium_Shimotsukasa
junkashihara
0
120
勾配流から乖離した学習ダイナミクス
hanbao
1
1.2k
科学で迫る勝敗の法則-スポーツデータ分析の最前線 (刈谷市連携講座.2026年7月) / The principle of victory discovered by science. at Kariya City, 2027.07
konakalab
0
220
SAT ソルバーの仕組みと制約ソルバーへの応用
tsoh
3
790
明治薬科大学講義_ビッグデータ解析を支えるデータベース技術とクラウドコンピューティング
ktatsuya
1
220
データベース03: 関係データモデル
trycycle
PRO
1
960
[Relay webinaire eBIS actu n°16] Sélection pour la vitalité des jeunes bovins allaitants et laitiers
institutdelelevage
PRO
0
370
ssmonline #51 ヤマサキ春のサメ祭り 2026 / ssmjp Yamasaki Spring JAWS Festival 2026
naospon
1
180
Featured
See All Featured
The SEO Collaboration Effect
kristinabergwall1
1
620
Git: the NoSQL Database
bkeepers
PRO
433
67k
Balancing Empowerment & Direction
lara
6
1.4k
Self-Hosted WebAssembly Runtime for Runtime-Neutral Checkpoint/Restore in Edge–Cloud Continuum
chikuwait
0
950
HU Berlin: Industrial-Strength Natural Language Processing with spaCy and Prodigy
inesmontani
PRO
0
810
The AI Search Optimization Roadmap by Aleyda Solis
aleyda
2
6.4k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
6.2k
Avoiding the “Bad Training, Faster” Trap in the Age of AI
tmiket
1
310
Paper Plane (Part 1)
katiecoart
PRO
3
12k
Odyssey Design
rkendrick25
PRO
3
950
Redefining SEO in the New Era of Traffic Generation
szymonslowik
1
530
XXLCSS - How to scale CSS and keep your sanity
sugarenia
250
1.3M