ABテストやってる？という問いかけと事例

Transcript

1. AB ςετ΍ͬͯ·͢ʁ
2. ࣗݾ঺հ • ৽Ҫ ܒଠ • גࣜձࣾϝϧΧϦͰಇ͍ͯ·͢ • ձࣾͰ͸͍ͪ͜͜ͱݺ͹Ε͍ͯ·͢
3. ࠓ೔ͷ಺༰ • ࿩͠·͢ • جຊతͳαʔόαΠυͷABςετͷ࣮૷ • iOS/Android ͱ࿈ܞ͢ΔAPI • ϝϧΧϦͰͷ࣮ྫ • ۤ࿑࿩ͱ͔ • ࿩͠·ͤΜ • Ұൠతͳπʔϧͷ঺հʢGoogle Optimize/FireBaseͱ͔ʣ
4. ࠓ೔ͷ಺༰ • ͍͢·ͤΜ • ιʔεΛެ։ͯ͠ͳ͍΍ͭͳΜͰࣗ෼Ͱͭ͘ Δ͔ɺ͏ͪೖͬͯɺOSSԽ͍ͯͩ͘͠͞
5. ௒ײँ ABςετͷجૅ෦෼͸͜ͷ ਓ͕ͭ͘Γ·ͨ͠ https://github.com/travail
6. travail ͷϞδϡʔϧͩͱଞʹ͜͜ Β΁Μ΋࢖ͬͯ·͢ • https://github.com/travail/php-Parallel- Prefork • Q4MͷWorkerॻ͘ͷʹ͔ͭͬͯ·͢
7. ࣭໰
8. Έͳ͞ΜABςετͯ͠·͢ʁ
9. ɾɾɾɾɾ
10. ͳΔ΄Ͳ
11. ϝϧΧϦͰͷABཱͪҐஔ
12. “୺తʹ༨ܭͳ΋ͷ3ͭɻͦΕ͸ ʮઆಘʯʮποίϛʯʮ৺഑ʯ ͩɻ ʢதུʣ ࠷ޙʹɺ৺഑ΛݮΒͨ͢Ίʹɺͱ ʹ͔͘ABςετΛߦܾͬͯΊΔ ͱ͍͏จԽΛਁಁ͍ͤͯ͞Δɻ” http://careerhack.en-japan.com/report/detail/755
13. ͍ͭ͝Ζʁ • ʢݸਓͷײ૝ʣٯʹ͍͑͹ɺʮ৺഑ʯ͠ͳ͘ ͯΑ͍ͳΒ΍Βͳͯ͘Α͍͠ɺ৺഑ͯ͠ͳΜ ͍ͯΒΕͳ͍࣌ظʹ΋΍Βͳͯ͘Α͍ͱࢥ͏ • ABςετΛ؅ཧ͢ΔਓݱΕΔλΠϛϯά • ϝϧΧϦͩͱΤϯδχΞ͕20ਓఔ౓ͷ࣌ظʹ ։͍࢝ͯ͠Δ
14. ɹϝϧΧϦͷΞʔΩςΫνϟ
15. ΞʔΩςΫνϟ • ϝϧΧϦʹ͸PascalͱݺΜͰ͍Δϩά෼ੳج൫ ͕͋Δ
16. Pascal http://tech.mercari.com/entry/2015/09/09/163007
17. ΞʔΩςΫνϟ • ϝϧΧϦʹ͸PascalͱݺΜͰ͍Δϩά෼ੳج൫ ͕͋Δ • αʔό͔Β΋eventͱͯ͠ABςετΛૹ৴
18. ΫϥΠΞϯτ্ͷUIΛมߋ͢Δ • ྫ: λΠϜϥΠϯͷݟͨ໨ͷมߋͳͲ
19. ABςετ(UIมߋ) .FSDBSJ
    "1* 1BTDBM
    4FSWFS
    1) ab_test_name1 2) நબ݁Ռ
20. ABςετ(UIมߋ) .FSDBSJ
    "1* 1BTDBM
    4FSWFS
    Send Event
21. APIͷϨεϙϯεมߋ͢Δ • ΫϥΠΞϯτमਖ਼ͤͣʹԠ౴Λมߋ͢Δ • ྫ: ʢ͋͘·ͰྫͰ͢ʣݕࡧ݁Ռͷมߋ΍දࣔ ൚༻ϝοηʔδͷมߋͳͲ
22. ABςετ(Ԡ౴มߋ) .FSDBSJ
    "1* 1BTDBM
    4FSWFS
    1) API Requet 4) AB ςετʹج͍ͮͨԠ౴ 2) நબɾநબʹج͍ͮͨϨεϙϯε࡞੒ 3) SendEvent
23. ΫϥΠΞϯτ͕ແؔ܎ͷ΋ͷ • batch ॲཧͳͲ • ΫϥΠΞϯτͷϢʔβΤʔδΣϯτͳͲऔಘ Ͱ͖ͳ͍߲໨ͱ͔͕ͳ͍Α͏ʹ஫ҙ
24. ABςετ(Ԡ౴มߋ) .FSDBSJ
    "1* 1BTDBM
    4FSWFS
    1) நબ݁Ռʹ΋ͱ͍ͮͯɺॲཧΘ͚ 2) SendEvent
25. ABςετͷίʔυ
26. $tracker = new Tracker(); $experiment_name = 'awesome_red_button'; $experiment_result = $tracker->getExperimentResult($experiment_name, $user_id, $uuid, ExperimentEvent::CLIENT_TYPE_IOS); $variant = $experiment_result->getVariant(); // Your A/B testing. if ($variant === 0) { // Control group } elseif ($variant === 1) { // Test group 1 } elseif ($variant === 2) { // Test group 2 } elseif ($variant === null) { // Something wrong } // Create Experiment event. $experiment_event = $experiment_result->createEvent(); // Log it. $tracker->sendEvent($experiment_event);
27. $experiment_name = 'awesome_red_button'; $experiment_result = $tracker- >getExperimentResult($experiment_name, $user_id,ɹ ExperimentEvent::CLIENT_TYPE_IOS);
28. $variant = $experiment_result->getVariant(); // Your A/B testing. if ($variant === 0) { // Control group } elseif ($variant === 1) { // Test group 1 } elseif ($variant === 2) { // Test group 2 } elseif ($variant === null) { // Something wrong }
29. // Create Experiment event. $experiment_event = $experiment_result->createEvent(); // Log it. $tracker->sendEvent($experiment_event);
30. ϩάͷ࢒͠ํ
31. ABςετͷநબͷཁ݅ • 1. AB ςετநબ݁Ռ • 2. ͦͷABςετϢʔβͷதؒͷಈ͖ • 3. ͦͷABςετ݁Ռʹࢸ͔ͬͨ
32. ABςετநબ
33. ABςετͷநબͷཁ݅ • 1ਓͷϢʔβͱɺ1ͭͷABςετͷ಺༰͕ͣΕ ͯ͸͍͚ͳ͍ • ্ཱͪ͛௚ͨ͢ͼʹɺϢʔβମݧ͔ΘͬͨΒς ετʹͳΒͳ͍Ͱ͠ΐ • 1ਓͷϢʔβ͕ɺෳ਺ͷABςετͰશ෦ಉ͡ɺ variantͩͱɺͪΌΜͱͨ͠ςετʹͳΒͳ͍
34. $user_id = 123;
 $ab_test_name = ‘ab_test_name’; hexdec (substr(md5($user_id . $expeliment_name . ‘salt’), 5)); // Ϣʔβ͝ͱʹݻఆ͔ͭABςετ͝ͱʹݻఆͷ਺஋͕Ͱ ΔͷͰ͋ͱ͸ྑ͠ͳʹ΍ͬͯ͘Ε͍
35. ઌఔͷίʔυ໰୊఺ • user_id ͕ͳ͍Ϣʔβʹະొ࿥Ϣʔβ • ΞϓϦΛΠϯετʔϧͨ࣌͠ʹ۠ผ͕ͭ͘Α ͏ͳ஋Λੜ੒ͯ͠ɺͦΕΛར༻͢Δͱ͔
36. ABςετ͋Δ͋Δ࿩
37. ίʔυ͕IFจͩΒ͚ʹ
38. ίʔυ͕IFจͩΒ͚ʹ • ے೑Ͱղܾ • class ઃܭͱ͔ • https://github.com/cookpad/chanko తͳΞϓ ϩʔνͱ͔ • ظؒΛܾΊͯɺίʔυΛ੔ཧͰ͖ΔΑ͏ʹͯ͠ ͓͘ʢاըͷਓͱ͔ͱѲ͓ͬͯ͘ʣ
39. ABςετʹ࢖ͬͯͳ͍
40. ABςετ͍ͬͯ͏͔ػೳͷεΠονʹ࢖ͬͯΔ • ݁ߏศར • ෛՙ͕৺഑ͳ৔߹ͳͲ͢͜ʔͣͭ͠ެ։ͨ͠ ͍৔߹ͱ͔ • ෆ۩߹ͰΫϥΠΞϯτ͕མͪͪΌ͏৔߹ͱ͔ ʹଈ࠲ʹαʔό͔ΒOffͬͨΓ