Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Web界隈LT会_爆速改善_Interleaving.pdf
Yuya Matsumura
July 06, 2018
Science
0
570
Web界隈LT会_爆速改善_Interleaving.pdf
A/Bテストより10-100倍効率的と言われるランキング評価手法、Interleavingについて、ザックリと概要をお話ししました。
Yuya Matsumura
July 06, 2018
Tweet
Share
More Decks by Yuya Matsumura
See All by Yuya Matsumura
事業会社における推薦システム開発事例 / recsys-in-wantedly-2022
yuya4
1
990
会社訪問アプリ「Wantedly Visit」のデータで見る相互推薦システム / deim2022-rrs-wantedly-visit
yuya4
0
730
ジョブマッチングサービスにおける推薦システムの活用 / dbsj-students-6th-wantedly
yuya4
0
110
ウォンテッドリーにおける推薦システム開発事例 / recsys-in-wantedly-2021
yuya4
4
1.3k
会社訪問アプリ「Wantedly Visit」のデータで見る相互推薦システム / deim2021-rrs-wantedly-visit
yuya4
3
1.6k
【論文紹介】ImRec: Learning Reciprocal Preferences Using Images / recsys2020-imrec
yuya4
3
710
RecSys Challenge 2020 Workshop: A Stacking Ensemble Model for Prediction of Multi-type Tweet Engagements
yuya4
0
140
WWW2020 論文紹介 / www2020-papers
yuya4
4
1.8k
会社訪問アプリ「Wantedly Visit」における相互推薦システムの応用事例 / deim2020-rrs-wantedly-visit
yuya4
1
2.9k
Other Decks in Science
See All in Science
非ホロノミック拘束を受ける連続体の微分形式による運動の定式化
deepflow
0
340
DMLDiD
masa_asa
0
120
機械学習における評価指標~AUC&C-index~
taka88
0
240
Kaggle Feedback Prizeコンペ 反省会
shimacos
3
1.1k
主成分分析及びテンソル分解を用いた教師無し学習による変数選択法とそのバイオインフォマティクスへの応用
tagtag
1
170
計算量理論
hn410
0
330
The likelihood function and its discontents
dwhgg
0
860
20220121_バスケットボール周りの流れ
kamakiri1225
0
240
Back to the Future: Bringing Original Hypermedia and Cross-Media Concepts to Modern Desktop Environments
signer
PRO
0
420
Bayesian statistics Tokyo.R#94
kilometer
5
1.8k
Information law about COVID-19
yoshimine77
0
130
My research talk for CCA promotion
dfm
1
530
Featured
See All Featured
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
498
130k
A designer walks into a library…
pauljervisheath
196
16k
Reflections from 52 weeks, 52 projects
jeffersonlam
337
17k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
349
27k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
38
12k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
7
1k
How GitHub Uses GitHub to Build GitHub
holman
465
280k
It's Worth the Effort
3n
172
25k
Building Your Own Lightsaber
phodgson
94
4.6k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
181
15k
How To Stay Up To Date on Web Technology
chriscoyier
780
250k
Building an army of robots
kneath
299
40k
Transcript
©2018 Wantedly, Inc. രվળʂInterleaving ABςετΑΓޮతͳϥϯΩϯάධՁΛٻΊͯ Webք۾LTձ 6.Jul.2018 - Yuya Matsumura
- @yu-ya4
©2018 Wantedly, Inc. •Yuya Matsumuraʢদଜ ༏ʣ •Wantedly, Inc. (ࣾձਓ4ϲ݄) •From
Kyoto •Interested in Information Retrieval & Recommendation Self Introduction @yu-ya4
©2018 Wantedly, Inc. ABςετͯ͠·͔͢ʁ
©2018 Wantedly, Inc. ͋Δಛఆͷظؒʹ2ύλʔϯͷࢪࡦΛ༻ҙ͠ɼͲͪΒͷ ࢪࡦͷํ͕ΑΓߴ͍ޮՌ͕ಘΒΕΔ͔Λݕূ͢Δ͜ͱ ABςετͱ WantedlyͰɼجຊతʹͯ͢ͷมߋʹରͯ͠ABςετΛߦ͔ͬͯΒ ͷਖ਼ࣜϦϦʔεΛߦ͍ͬͯ·͢ʂ
©2018 Wantedly, Inc. Wantedlyʹ͓͚ΔϥϯΩϯάͷABςετ A/B Testing ̍ ̎ ̍
̎ Group A Ranking A Control Variation Group B Ranking B
©2018 Wantedly, Inc. ༗ҙͳ͕ࠩग़Δ·Ͱʹ ABςετͷ՝ →PDCAαΠΫϧΛճ͢ͷ͕͘ͳΔʂ Ex. *ݱࡏCVR 1%ͷͷΛ10%վળ(CVR 1.1%)͢Δʹ134,379ͷαϯϓϧ͕ඞཁ
*༗ҙਫ४Ћ=5%, ݕग़ྗ1-β=95%(https://keisan.casio.jp/exec/user/1489063696) େྔͷαϯϓϧ(impression, PV)͕ඞཁ
©2018 Wantedly, Inc.
©2018 Wantedly, Inc. ͦ͜ͰInterleaving
©2018 Wantedly, Inc. 2ͭͷϥϯΩϯά͔Βަޓʹݕࡧ݁ՌΛ࠷্Ґ͔Βॱʹ બ͠1ͭͷϥϯΩϯάΛੜͯ͠Ϣʔβʹఏࣔ͢Δ Interleavingͱ ̍ ̎ ̍
̎ Ranking A Ranking B ̍ ̎ ̍ ̎ Ranking C ᶃ ᶄ ᶅ ᶆ
©2018 Wantedly, Inc. Interleavingͷੑೳ →രͰPDCAαΠΫϧ͕ճͤΔʂ *ਖ਼֬ʹʮΠϯλʔϦʔϏϯάA/BςετΑΓ100ഒޮతͳ߹͕͋Γɺ͓͓Αͦͷ߹ɺ 10~100ഒޮతͰɺA/BςετΑΓޮతͰͳ͍߹গͳ͍ʯ(Chapelle et al. 2012)
Interleaving ABςετΑΓ10ʙ100ഒޮత*
©2018 Wantedly, Inc.
©2018 Wantedly, Inc. Interleaving͕ABςετΑΓޮతͳཧ༝ ̍ ̎ ̍ ̎
Ranking A Ranking B A/B Testing ̍ Ranking A ͱ Ranking Bͷੑೳ͕ࣅ͍ͯΔ߹ ͕ࠩग़ʹ͍͘ → ̍ ̍ ̍ ̍ ̍
©2018 Wantedly, Inc. Interleaving͕ABςετΑΓޮతͳཧ༝ Interleaving ̍ Ranking A ͱ Ranking
B͕҉ʹൺֱ͞ΕΔͷͰ ͕ࠩग़͍͢ʂʂ → ̍ ̍ ̍ ̍ ̍ ̎ ̍ ̎ ̍
©2018 Wantedly, Inc. ࣮ࡍͷABςετͱInterleavingͷp-valueͷऩଋͷൺֱ (Wantedly VisitͷϥϯΩϯάվળʹͯܭଌ)
©2018 Wantedly, Inc. ͦͷ݁Ռ…
©2018 Wantedly, Inc. ৽ଔೖࣾ1ϲ݄ͰWantedly Visitͷ ৽͍͠ϥϯΩϯάࢪࡦΛຊ൪ϦϦʔε (ࠓͰܧଓͯ͠ྑ͍݁ՌΛग़͠ଓ͚͍ͯ·͢)
©2018 Wantedly, Inc. InterleavingΛ׆༻ͯ͠രϥϯΩϯάվળΛ