A/Bテストをやるときに 気をつけていること

Transcript

1. ©2018 Wantedly, Inc.
   A/BςετΛ΍Δͱ͖ʹ

   ؾΛ͚͍ͭͯΔ͜ͱ
   17.Oct.2018 - Shingo Tomioka
   S-DEV TALKS #4
   

   View Slide

2. ©2018 Wantedly, Inc.
   w ෋Ԭਅޛ !NFFB@U
   
   w ΤϯδχΞBU8BOUFEMZ *OD
   w 8BOUFEMZ7JTJUͷ։ൃɾάϩʔε
   ࣗݾ঺հ
   

   View Slide

3. ©2018 Wantedly, Inc.
   ઃܭɾ४උஈ֊
   

   View Slide

4. ©2018 Wantedly, Inc.
   w ෳ਺ม͑ΔͱɺԿ͕ͲΕ͘Β͍ޮ͍ͨͷ͔Θ͔Γʹ͘͘ͳΔ
   w ྫɿϘλϯͷ഑ஔͱจݴ
   ม͑Δ఺͸ͳΔ΂͘Ұͭʹ͢Δ
   BUY
   ߪೖ
   A B
   

   View Slide

5. ©2018 Wantedly, Inc.
   w ༗ҙ͕ࠩग़ΔͷʹඞཁͳظؒΛࣄલʹݟੵ΋Δ
   w ࠓͷϢʔβʔ਺ͩͱ༗ҙ͕ࠩग़Δͷʹϲ݄͔͔Δ͔΋͠Εͳ͍
   w Q஋͕Լ͕ͬͨλΠϛϯάͰ੾Γ্͛ͨ͘ͳΔͷΛ๷͙
   ࣄલʹظؒΛܾΊΔ
   p஋
   

   View Slide

6. ©2018 Wantedly, Inc.
   w ͪΌΜͱ࣮૷ͨͭ͠΋ΓͰ΋ɺऔΕ͍ͯͳ͔ͬͨΓ͢Δ
   w ։ൃ؀ڥͱຊ൪؀ڥͷҧ͍
   w ಛఆͷ৚݅Λ࣋ͭϢʔβʔͷͱ͖औΕ͍ͯͳ͍
   ‎ σϓϩΠͨ͠ΒͳΔ΂͘ૣ֬͘ೝ͢Δ
   w ਺ࣈ͕ࢥͬͨΑΓগͳ͘ͳ͍͔
   σϓϩΠͨ͠ΒͳΔ΂͘ૣ͘ɺϩά͕ͪΌΜͱऔΕ͍ͯΔ͔֬ೝ͢Δ
   

   View Slide

7. ©2018 Wantedly, Inc.
   ܭଌɾධՁஈ֊
   

   View Slide

8. ©2018 Wantedly, Inc.
   w ΫΤϦ͕ؒҧ͍ͬͯΔ͜ͱ͸Α͋͘Δ
   w ΫΤϦ͕ؒҧ͍ͬͯͯ΋ɺͦΕͬΆ͍਺ࣈ͕ग़ͯ͠·͏
   w ਺ࣈ͕ग़Δͱɺ͙͢ղऍʹ૸Γͨ͘ͳΔ
   ‎ ਺ࣈ͕௚ײͱେ͖͘཭Εͯͳ͍͔֬ೝ͢Δ
   ‎ ՄೳͳΒ
   ෳ਺ͷܭΓ͔ͨͰݕࢉ͢Δ
   w ྫɿϑϩϯτଆͷϩάͱɺαʔόʔ΁ͷϦΫΤετϩά
   ΫΤϦͷνΣοΫ͸৻ॏʹ
   

   View Slide

9. ©2018 Wantedly, Inc.
   w ฏۉ஋͸֎Ε஋ͷӨڹΛड͚΍͍͢
   w ྫɿӾཡ਺͕৳ͼͨͱࢥͬͨΒɺਓͷϢʔβʔ͕

   େྔʹӾཡ͍͚ͯͨͩͩͬͨ͠
   ‎ ෼෍ΛݟΕ͹શମతͳ܏޲Λ

   ೺ѲͰ͖Δ
   ฏۉ஋Ͱ͸ͳ͘෼෍ΛݟΔ
   A
   B
   A B
   8.0 7.3
   

   View Slide

10. ©2018 Wantedly, Inc.
    w ໨৽͍͠6*΍ίϯςϯπ͸୹ظతʹΫϦοΫ͕૿͑΍͍͢
    ‎ ςετ։࢝ޙ͙͢ͷσʔλΛࣺͯΔ
    ‎ ৽نϢʔβʔʹ͠΅ͬͯݟͯΈΔ
    New Button ޮՌʹؾΛ͚ͭΔ
    

    View Slide

11. ©2018 Wantedly, Inc.
    w ϖʔδ಺ͷίϯςϯπ͸ΫϦοΫ਺Λୣ͍߹͏
    w ྫɿαΠυόʔͷݟͨ໨Λվળͨ͠ΒɺϝΠϯΧϥϜͷΫϦοΫ਺͕མͪͨ
    w ςετର৅Ͱͳ͍਺ࣈ͕࣮͸མ͍ͪͯΔ͔΋
    ‎ Өڹͷ͋Γͦ͏ͳଞͷࢦඪ΋֬ೝ͢Δ
    ଞͷࢦඪ͕ѱ͘ͳ͍ͬͯͳ͍͔֬ೝ͢Δ
    

    View Slide

12. ©2018 Wantedly, Inc.
    w ࣗ෼ͷԾઆʹ߹க͢ΔσʔλΛॏࢹ͠ɺ߹Θͳ͍σʔλΛܰࢹͯ͠
    ͠·͏܏޲
    w "#ςετΛ΍Δͱ͖͸ɺԿ͔͕Α͘ͳΔͱظ଴͍ͯ͠Δ
    w Α͘ͳͬͨ਺ࣈʹ஫໨ͯ͠ɺѱ͘ͳͬͨ਺ࣈΛܰࢹ͕ͪ͠
    ‎ όΠΞε͕͋Δ͜ͱΛҙ͓͚ࣝͯͩ͘͠Ͱ΋ྫྷ੩ʹ൑அͰ͖Δ
    ֬ূόΠΞεʹؾΛ͚ͭΔ
    

    View Slide

13. ©2018 Wantedly, Inc.
    w ΍͍ͬͯΔͷΛ๨Εͯςετ͕ͣͬͱ૸͍ͬͯΔঢ়ଶ
    w ظؒͷ௕͍ςετ΍ϓϩδΣΫτͷมΘΓ໨͸ಛʹ஫ҙ
    ‎ ऴྃ༧ఆ೔ʹϦϚΠϯμʔΛηοτ͓ͯ͘͠
    ‎ ఆظతʹաڈͷJTTVF΍ϩάΛ֬ೝ͢Δ
    ग़ͬ͠ͺͳ͠ʹ͠ͳ͍
    

    View Slide

14. ©2018 Wantedly, Inc.
    Enjoy A/B testing!
    

    View Slide