Gunosy分析チームとA/Bテスト運用 #MLCT /ab-testing-in-gusnoy
by
ysekky
Link
Embed
Share
Beginning
This slide
Copy link URL
Copy link URL
Copy iframe embed code
Copy iframe embed code
Copy javascript embed code
Copy javascript embed code
Share
Tweet
Share
Tweet
Slide 1
Slide 1 text
About Gunosy Data Analysis Team? @ysekky_ 2015/12/15 Machine Learning Casual Talks #4
Slide 2
Slide 2 text
about • Yoshifumi Seki • גࣜձࣾGunosy։ൃຊ෦σʔλੳ෦ • Ϣʔβߦಈੳ, هࣄ৴ϩδοΫߏங • ౦ژେֶେֶӃത࢜ޙظ՝ఔࡏֶத • Ϣʔβߦಈੳ, ίϯςϯπධՁ • ΞΠυϧϮλΫ/কع/ຊञ
Slide 3
Slide 3 text
ࠓͷ • σʔλੳ෦νʔϜ͕ͲΜͳ͜ͱΛ͍ͬͯΔ͔ • σʔλੳνʔϜʹؔ͢Δϊϋ͕গͳ͍ • ͱ͜ΖͯΜ͞Μͷσϒαϛळͷൃද • Θ͔Δ • ͜͏͍͏ͷͬͱΦʔϓϯʹͯ͠ڞ༗͍ͯٞͨ͠͠ • ͔ͩΒΒެ։͠·͢ • ࣌ؒͳ͗͢͞ΔͷͰ֓ཁͱA/Bςετཧͷํ๏ʹ͍ͭͯ
Slide 4
Slide 4 text
σʔλੳ෦ʹ͍ͭͯ • ߏ • 5໊+ΞϧόΠτ2໊ • ओͳۀ • هࣄ৴ϩδοΫͷ։ൃ • KPIͷઃܭɾੳ • KPIμογϡϘʔυͷ࡞ • ࢪࡦͷੳ४උͱ݁Ռੳ • Ϣʔβߦಈੳ
Slide 5
Slide 5 text
ओʹͬͯΔαʔϏεπʔϧ • Github • Django+HighChartsʹΑΔࣾDashBoard • Մೳͳ͔͗ΓࡉԽՄೳʹͯ͠༻ҙ • csvμϯϩʔυՄೳʹͯ͠෦ॺ֎ͷਓ͕ੳͰ͖ΔΑ͏ʹ͢Δ • ipython notebook • Trello • redash • ఆৗूܭ෦ॺ֎ͷਓʹSQLͯͬͯ͠Β͏ • ͍͍ͪͪόονUI༻ҙ͢ΔͷΊΜͲ͍͘͞
Slide 6
Slide 6 text
ҎલͷਐΊํ MLCT #2ΑΓ
Slide 7
Slide 7 text
՝ • ৫ͷن͕େ͖͘ͳͬͨ • Ͳ͜ͰͳΜͷࢪࡦΛ͍ͬͯΔ͔ݟ͑ͮΒ͘ͳͬͨ • ͳʹ͕ͲͷʹӨڹ͍ͯ͠Δ͔ෳࡶԽ͍ͯ͠Δ • αʔϏεن͕େ͖͘ͳͬͯͪΐͬͱͷϛε͕Өڹ͢ΔΑ͏ ʹͳͬͨ • վળεϐʔυԼ͛ͨ͘ͳ͍
Slide 8
Slide 8 text
Github issueʹΑΔA/Bςετཧ issueͰA/BςετͷਐḿΛཧͯ͠Δ
Slide 9
Slide 9 text
Github issueʹΑΔA/Bςετཧ • త • ͳʹΛతͱͨ͠ͲΜͳςετͳͷ͔ • ܭը • ͲͷϢʔβʹఏڙͯ͠ͲͷΛ͏͔ • ܦաϩά • ͍ͭͲͷΑ͏ͳ݁ՌʹΑͬͯͲͷΑ͏ͳҙࢥܾఆΛ͔ͨ͠
Slide 10
Slide 10 text
Github issueʹΑΔA/Bςετཧ
Slide 11
Slide 11 text
Github issueʹΑΔA/Bςετཧ
Slide 12
Slide 12 text
Github issueʹΑΔA/Bςετཧ
Slide 13
Slide 13 text
ػೳͷϦϦʔεϑϩʔ 1. 5%ެ։ɺத2Ҏ্ (มߋͨ͠ϩδοΫʹΫϦςΟΧϧͳόά͕ͳ͍͔ͷνΣοΫ) 2. 10%ެ։ɺத2Ҏ্ (มߋͨ͠ϩδοΫʹΫϦςΟΧϧͳѱӨڹ͕ͳ͍͔ͷνΣοΫ) 3. 20%ެ։ɺத3Ҏ্ (มߋͨ͠ϩδοΫ͕తKPIΛվળ͍ͯ͠Δ͔ͷνΣοΫ) 4. 40%ެ։ɺத5Ҏ্ (มߋͨ͠ϩδοΫ͕తKPIΛվળ͍ͯ͠Δ͔ɺܧଓʹѱӨ ڹΛٴ΅͍ͯ͠ͳ͍͔ͷνΣοΫ) 5. 80%ެ։ɺத3Ҏ্ (ରൣғΛ֦େͯ͠ͷతKPIͷվળΛҡ͍࣋ͯ͠Δ͔ɺܧ ଓʹѱӨڹΛٴ΅͍ͯ͠ͳ͍͔ͷνΣοΫ) 6. 100%ެ։
Slide 14
Slide 14 text
·ͱΊͱ͜Ε͔Βͷ՝ • ਓ͕૿͑ͯେ͖͘ͳͬͯ͘Δͱϧʔϧ͕ඞཁʹͳͬͯ͘Δ • ͳʹΛ͍ͬͯΔͷ͔ΛͻͱʹΘ͔ΔΑ͏ʹ • ΛԼ͛ͨ͘ͳ͍ͷͰΈͰ҆શͳ΄͏ʹ͍ͬͯ͘ • ΠςϨʔγϣϯͷΛ͍͋͛ͨ • ूܭͷࣗಈԽ • ࣮ͷ݈શԽ • ϊϋڞ༗ͯٞͯ͠͠ΑΓΑ͍σʔλੳ෦࡞Γ͍ͨ • σʔλੳ෦తேձʹ͍ͭͯ • KPIπʔϧͷ࡞Γํ • ੳґཔͱͷઓ͍ํ • ΫϥΠΞϯτΞϓϦͷϩάઃܭ