Back of First Screen AB test of Hatena Bookmark iOS app with Firebase AB Testing

Transcript

1. 1.

   2019/07/22 @Google AdMob Japan Meetup #5 Yutaro Muta @yutailang0119 ͸ͯͳϒοΫϚʔΫiOSΞϓϦͷ

   First Screen A/Bςετͷཪଆ with Firebase A/B Testing
2. 2.

   • Yutaro Muta @yutailang0119 • Hatena Co., Ltd. @Kyoto •

   Conference Staff • builderscon 2017, 2018 • try! Swift Tokyo 2019, 2020 • ࠷ۙͷਪ͠͸Combine framework Who am I ?
3. 3.

   ࿩͢͜ͱ • ͸ͯͳϒοΫϚʔΫͷiOSΞϓϦͰߦͬͨA/Bςετͷ֓ཁ • ্هʹ͓͚ΔFirebase A/B Testingͷ໾ׂ • ઃఆํ๏ •

   Tips
4. 4.

   ࿩͞ͳ͍͜ͱ • AdMobʹΑΔΞϓϦ಺޿ࠂ

5. 5.

   ͸ͯϒͷεϚϗΞϓϦΛ ࢖͍ͬͯΔਓʙ

6. 6.

   ͸ͯϒεϚϗΞϓϦ • https:b.hatena.ne.jp ͷεϚϗΞϓϦ • ίϝϯτ෇͖ͰϒοΫϚʔΫͰ͖Δ • ײ૝Λڞ༗Ͱ͖Δ • ࿩୊ͷϖʔδΛൃݟͰ͖Δ

   • iOS / Android
7. 7.

   ࠷ۙ͸ͯϒΞϓϦͷ ϗʔϜը໘͕มΘͬͨͳʔ ͱؾ͍ͮͨਓʙ

8. 8.

   Before

9. 9.

   After

10. 10.

    Before After

11. 11.

    Before After

12. 12.

    มߋ఺ • First ScreenͰΧςΰϦຖͷਓؾʹ੾Γସ͑ΒΕΔ • ݩʑ͸૯߹ͷਓؾͱ৽ணͷΈ͕දࣔ͞Ε͍ͯͨ • ΧςΰϦͷฒͼॱ͸ɺ೚ҙʹฒͼସ͑ΒΕΔ

13. 13.

    Ͱ΋ɺ First ScreenΛΤΠϠοͱ มߋ͢Δͷ͸ා͍ͳ͊

14. 14.

    ͦ͜ͰA/Bςετ

15. 15.

    A/BςετπʔϧʹٻΊͨ͜ͱ • [Must] A or Bͷ੾Γସ͑Λ֎෦ (ΞϓϦϦϦʔεαΠΫϧ֎) ͔Βߦ͑Δ • ঢ়ଶʹ߹Θͤͯը໘ଆΛ੾Γସ͑Δ

    • ςετதͷෆ۩߹΍݁Ռʹ໌Β͔ʹେ͖ͳ͕ࠩग़ͨࡍʹɺׂ߹Λมߋ • Ϗδωεతͳ͋Ε͜ΕʹରԠͰ͖ΔΑ͏ • [Better] ෼ੳπʔϧͱͷ૬ੑ΍ݕূͷ͠΍͢͞
16. 16.

    Firebase A/B Testing

17. 17.

    Firebase A/B Testing • ͢ͰʹFirebaseϓϥοτϑΥʔϜΛར༻͍ͯ͠Δͱɺಋೖίετ͕গͳ͍ • Google Analytics for Firebaseͱ࿈ܞ͕Մೳ

    • Remote Configͱͯ͠ѻ͏ͷͰɺखܰ
18. 18.

    Firebase A/B Testingͷઃఆ

19. 19.

    ઃఆελʔτ

20. 20.

    λʔήοτͷઃఆ

21. 21.

    λʔήοτͷઃఆ Analytics.logEvent(_:, parameters:) ͰૹΔΠϕϯτΛ࢖ͬͯɺ ಛఆͷૢ࡞Λͨ͠ϢʔβʔͷΈΛλʔήοτʹࢦఆՄೳ

22. 22.

    ໨ඪઃఆ

23. 23.

    ໨ඪઃఆ

24. 24.

    όϦΞϯτͷઃఆ

25. 25.

    ΞϓϦͷ࣮૷

26. 26.

    Remote Config͔Β஋Λऔಘ // Fetch from Remote Config func fetch() {

    let expiration: TimeInterval = (60 * 60 * 3) remoteConfig.fetch(withExpirationDuration: expiration, completionHandler: nil) } // Activate fetched Remote Config settings func activateFetched() { remoteConfig.activateFetched() }
27. 27.

    Remote ConfigͰऔಘͨ͠ઃఆ஋͔Βը໘Λग़͠෼͚Δ let viewController: UIViewController switch remoteConfig.viewSetting { case .a:

    viewController = AViewController() case .b: viewController = BViewController() } // Add to ContainerView viewController.view.translatesAutoresizingMaskIntoConstraints = false addChild(viewController) containerViewController.addSubview(viewController.view) viewController.didMove(toParent: self) NSLayoutConstraint.activate([NSLayoutConstraint]) // Switch ViewController by Remote Config let viewController: UIViewController switch remoteConfig { case .a: viewController = AViewController() case .b: viewController = BViewController() default: viewController = DefaultViewController() } // Add to ContainerView viewController.view.translatesAutoresizingMaskIntoConstraints = false addChild(viewController) containerViewController.addSubview(viewController.view) viewController.didMove(toParent: self) NSLayoutConstraint.activate([NSLayoutConstraint]())
28. 28.

    ⷁ ⷂ

29. 29.

    A/Bςετ։࢝

30. 30.

    ~͕࣌ྲྀΕͯ~

31. 31.

    ςετ݁Ռ • ͓Αͦ1ϲ݄΄ͲͷςετΛ࣮ࢪͨ͠ • ςετ࣮ࢪલ͔Β΄΅૝ఆ௨Γͷظؒ • ৽ը໘ͷํ͕ɺچը໘ΑΓ΋ઃఆͨ͠ࢦඪʹ͓͍ͯ༏ҐͳࠩΛಘΒΕͨ • ࢦඪ͸ൿີ

32. 32.

    ίϯιʔϧը໘

33. 33.

    ςετ׬ྃ

34. 34.

    ςετ݁ՌΛݩʹ... • ຬΛ࣋ͯ͠ɺ৽ը໘Λ100%ʹมߋ • AndroidͰ΋৽ը໘Λ࣮૷ɺϦϦʔε • ݁ՌɺAndroidͰ΋ྑ͍݁ՌΛಘΒΕͨ

35. 35.

    A/B Testing Q&A

36. 36.

    Q. ॳճىಈ࣌ʹ΋ A/BςετΛ൓ө͍ͨ͠

37. 37.

    Q. ॳճىಈ࣌ʹ΋A/BςετΛ൓ө͍ͨ͠ • Remote Config͸ɺωοτϫʔΫ௨৴Λ௨ͯ͡ઃఆ஋Λऔಘ͢Δ • ॳճFetch࣌ʹ͸ઃఆ஋͸ແ͍ • ίʔυ্Ͱͷdefault஋͸༻ҙͰ͖Δ •

    (ಛʹ) First ScreenͷA/BςετͰ͸ɺ൓өλΠϛϯάʹ޻෉͕ඞཁʹͳΔ
38. 38.

    ॳճFetchͷ޻෉Ҋ 1. Fetch͕׬ྃ͢Δ·Ͱɺը໘Λwait͢Δ Pros: ॳճ΋A/Bςετͷઃఆ஋Λ൓өͰ͖Δ Cons: ଴͕ͪ࣌ؒൃੜɺFetch͕ࣦഊͨ࣌͠͸Ͳ͏͢Δʁ 2. Fetch͕׬ྃͨ͠Βଈ࠲ʹ൓өͤ͞Δ ->

    ࠷ॳ͸default஋Ͱը໘Λදࣔ -> Fetchޙ͙͢ʹ RemoteConfig.activateFetched() -> ը໘ͷߋ৽ Pros: ࣮૷্ͷߟྀ఺͸গͳ͍ Cons: Ϣʔβʔ͸࢖༻தʹUI͕มߋ͞ΕΔͷͰɺڻ͔͘΋ 3. ࣍ճىಈ࣌·Ͱ଴ͭ -> ىಈ࣌ʹͷΈ RemoteConfig.activateFetched() Pros: ࣮૷্ͷߟྀ఺͸গͳ͍ Cons: 2ճ໨Ҏ߱ͷىಈʹͨͲΓண͚ͳ͍ͱɺςετͰ͖ͳ͍
39. 39.

    Q. A/Bςετͷσόοά େม͡Όͳ͍ʁ

40. 40.

    Q. A/Bςετͷσόοάେม͡Όͳ͍ʁ • betaϏϧυͰͷΈ Khan/SwiftTweaks Λ࢖͍ɺ੾Γସ͑Մೳʹͨ͠ • A/BςετҎ֎ʹ΋௨৴΍ͦͷଞઃఆͷ੾Γସ͑ɺσόοάʹ΋࢖͍ͬͯΔ

41. 41.

    &OKPZ
    "#
    5FTUJOH
    5IBOLT w NVUBZVUBSP!HNBJMDPN
    w IUUQTUXJUUFSDPNZVUBJMBOH
    w IUUQTHJUIVCDPNZVUBJMBOH