ABテストが上手く行かなかった話

Transcript

1. 鸐濼鏩閉桦ぢ♳ך׋׭ך倵瘻ך
   "#ذأزדֲתְַֻזַ׏׋鑧
   傈劤穗幥倜耀爡խ⚥䊛♰蛁㣽
   Otemachi.swift #02
   

   View Slide

2. Marina Nakagawa
   @marina10172013
   ೔ຊܦࡁ৽ฉࣾͷiOSΤϯδχΞ

   ిࢠ൛ΞϓϦɼࢴ໘ϏϡʔΞʔΞϓϦͷ։ൃ
   

   View Slide

3. ຊ೔࿩͢͜ͱ
   ࣮ࡍʹऔΓ૊Μͩ͜ͱ
   ௨஌ڐ୚཰Λ޲্ͤ͞ΔͨΊͷࢪࡦͷ֓ཁ
   1
   2
   3 ݁Ռͱ·ͱΊ
   

   View Slide

4. ௨஌ڐ୚཰Λ޲্ͤ͞ΔͨΊͷࢪࡦͷ֓ཁ
   1
   

   View Slide

5. ೔ܦిࢠ൛ͷ௨஌ڐ୚཰ʹ͍ͭͯ
   • ిࢠ൛ͷ௨஌ͷڐ୚ͷΞϥʔτΛग़͢ͷ͸ىಈ࣌
   • ςετ։࢝࣌ͷڐ୚཰͸໿52ˋҐ
   Կ΋͠ͳͯ͘΋໿52%͘Β͍ͷਓ͸
   ڐ୚ͯ͘͠Ε͍ͯΔ
   

   View Slide

6. ڐ୚཰52%ͷײ֮
   New data shows up to 60% of users opt-out of push notifications (Guest Post)ΑΓҾ༻

   (https://andrewchen.co/why-people-are-turning-off-push/)
   

   View Slide

7. ڐ୚཰52%ͷײ֮
   New data shows up to 60% of users opt-out of push notifications (Guest Post)ΑΓҾ༻

   (https://andrewchen.co/why-people-are-turning-off-push/)
   

   View Slide

8. ڐ୚཰52%ͷײ֮
   New data shows up to 60% of users opt-out of push notifications (Guest Post)ΑΓҾ༻

   (https://andrewchen.co/why-people-are-turning-off-push/)
   ҰൠతͳχϡʔεͷΞϓϦͱ

   ൺֱ͢Δͱଟ͍
   

   View Slide

9. ௨஌Λ։͘ͱ௚઀هࣄʹඈͿΑ͏ʹͳͬͨ
   

   View Slide

10. ࠓͦ͜௨஌Λ࢖ͬͯ΄͍͠ɾɾɾ
    

    View Slide

11. ௨஌ڐ୚཰Λ΋ͬͱ΋ͬͱ্͛ͯ!
    ϢʔβʔʹχϡʔεΛಧ͚͍ͨ
    

    View Slide

12. ௨஌ΛڐՄͯ͠΋Β͏ʹ͸
    ৐Γӽ͑Δඞཁ͕͋Δน͕͋Δ
    

    View Slide

13. View Slide

14. Ծઆ
    ΞϓϦͷॳճΠϯετʔϧ࣌ʹ

    ϓογϡ௨஌ͷҰྫΛࣔ͠༗༻Ͱ͋Δ͜ͱΛࣔͤΕ͹

    ௨஌ͷڐ୚཰্͕͕ΔͷͰ͸ͳ͍͔
    
    

    View Slide

15. ࣮ࡍʹऔΓ૊Μͩ͜ͱ
    2
    

    View Slide

16. ը໘ભҠਤ
    

    View Slide

17. ϙοϓΞοϓը໘ͷৄࡉ
    ͜ͷϘλϯΛԡͨ͠ਓ͚͕ͩ

    OSͷڐ୚௨஌ͷϙοϓΞοϓ͕Ͱͯ͘Δ
    

    View Slide

18. ϙοϓΞοϓը໘ͷৄࡉ
    

    View Slide

19. ϙοϓΞοϓը໘ͷৄࡉ
    ࠓ͸΍Ί͓ͯ͘ϘλϯΛԡͨ͠ਓ
    ࿮֎λοϓΛͨ͠ਓ
    ԼεϫΠϓΛͨ͠ਓ
    

    View Slide

20. ϙοϓΞοϓը໘ͷৄࡉ
    

    View Slide

21. ݁Ռͱ·ͱΊ
    3
    

    View Slide

22. ▪ ϙοϓΞοϓͳ͠ɿ໿40%
    
    ▪ ϙοϓΞοϓ͋Γɿ໿21%
    
    ௨஌ڐ୚཰:52%
    
    ࣮ࢪޙ
    
    ࣮ࢪલ
    
    A/Bςετͷ݁Ռ
    

    View Slide

23. A/Bςετͷ݁Ռ
    ϙοϓΞοϓΛදࣔͨ͠σόΠεͷํ͕

    ௨஌ONͷׂ߹͕൒਺ఔ౓ͩͬͨ
    

    View Slide

24. A/Bςετͷ݁Ռ
    ϙοϓΞοϓΛදࣔͨ͠σόΠεͷํ͕

    ௨஌ONͷׂ߹͕൒਺ఔ౓ͩͬͨ
    མͪͯΔɾɾɾɾɾ "
    

    View Slide

25. ݪҼ͸ʁ
    Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ
    1
    2 ϝϦοτΛϢʔβʔʹ఻͖͑Ε͍ͯͳ͔ͬͨ
    

    View Slide

26. Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ
    1
    

    View Slide

27. Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ
    1
    

    View Slide

28. Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ
    1
    ͜ͷϘλϯΛԡͨ͠ਓ͚͕ͩ

    OSͷڐ୚௨஌ͷϙοϓΞοϓ͕Ͱͯ͘Δ
    

    View Slide

29. ࠓ͸΍Ί͓ͯ͘ϘλϯΛԡͨ͠ਓ
    ࿮֎λοϓΛͨ͠ਓ
    ԼεϫΠϓΛͨ͠ਓ
    ໿70%ͷਓ͕OSͷڐ୚௨஌Λ

    ݟͳ͍͜ͱʹͳͬͯ͠·ͬͨ
    Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ
    1
    

    View Slide

30. ○ཁૉΛͻͱ͚ͭͩม͑ͯςετ͢Δͷ͕A/Bςετͳͷʹɺ
    ෳ਺ཁૉͰൺֱͤͯ͞͠·ͬͨ

    ○ࣗ෼͕ίϯςϯπͷத਎(ಈըͷ੍࡞΍දهͷ಺༰)ʹूத͠
    ͯ͠·͍ભҠͷ͜ͱ·Ͱߟ͍͑ͯͳ͔ͬͨ
    
    ○ԼϑϦοΫɺແࢹ͢Δͱ͍͏બ୒ࢶ͸ແͯ͘͠Α͔ͬͨ
    
    ഊҼ
    Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ
    1
    

    View Slide

31. Ծઆ
    ϓογϡ௨஌ͷҰྫΛࣔ͠༗༻Ͱ͋Δ͜ͱΛࣔͤΕ͹
    
    ௨஌ͷڐ୚཰্͕͕Δ͔͔֬ΊΔ
    ࢼ͢͜ͱ
    OSͷ௨஌ڐ୚͕ग़ΔલʹϙοϓΞοϓը໘Ͱ

    ༗༻Ͱ͋Δ͜ͱΛઆ໌͢Δ͜ͱͰ௨஌ڐ୚͸มΘΔͷ͔
    Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ
    1
    

    View Slide

32. Ծઆ
    ϓογϡ௨஌ͷҰྫΛࣔ͠༗༻Ͱ͋Δ͜ͱΛࣔͤΕ͹
    
    ௨஌ͷڐ୚཰্͕͕Δ͔͔֬ΊΔ
    ࢼ͢͜ͱ
    OSͷ௨஌ڐ୚͕ग़ΔલʹϙοϓΞοϓը໘Ͱ

    ༗༻Ͱ͋Δ͜ͱΛઆ໌͢Δ͜ͱͰ௨஌ڐ୚͸มΘΔͷ͔
    Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ
    1
    มߋ͢Δ΂͖͸ϙοϓΞοϓը໘Λग़͔͢ग़͞ͳ͍͔Ͱ͋Γ

    OSͷ௨஌ڐ୚ը໘Λग़͔͢ग़͞ͳ͍͔มߋΛՃ͑Δ΂͖Ͱ͸ͳ͍
    

    View Slide

33. ʮࠓ஌Δ΂͖ʯΛಀ͞ͳ͍
    ศརͳϓογϡ௨஌
    2 ϝϦοτΛϢʔβʔʹ఻͖͑Ε͍ͯͳ͔ͬͨ
    ○ҙࢥܾఆΛઌʹ͖͚࣋ͬͯͨͲ৘ใ͕গͳ͍͔Β͔΋͠Εͳ͍

    ○ಈը͸ॏཁͰ͸ͳ͍͔΋͠Εͳ͍

    ഑ஔͷ໰୊ͰจݴͱϘλϯΛ͚ۙͮͨ΄͏͕Α͔͔ͬͨ΋
    
    ○Ϙλϯͷબ୒ࢶͰ͋Δ

    ʮ͍·͸΍Ί͓ͯ͘ʯ͸·ͩνϟϯε͕༗ΔΑ͏ʹݟ͑Δ
    
    

    View Slide

34. ·ͱΊ
    A/BςετΛ࣮ࢪ͢Δ࣌͸ɺ

    ຊ౰ʹཁૉΛͻͱ͚ͭͩม͑ͯൺֱͯ͠Δͷ͔֬ೝ͢Δ
    ϙοϓΞοϓʹදࣔͨ͠ίϯςϯπ͕ຊ౰ʹྑ͔ͬͨͷ͔Ͳ͏͔͸Θ͔Βͳ͍

    จݴ΍ίϯςϯπʹΑͬͯҧ͍͕͋Δͷ͔ͱ͍͏ͷ΋ࢼͯ͠Έ͍ͨ
    

    View Slide

35. ͋Γ͕ͱ͏͍͟͝·ͨ͠
    

    View Slide