Back of First Screen AB test of Hatena Bookmark iOS app with Firebase AB Testing

Transcript

1. 2019/07/22 @Google AdMob Japan Meetup #5 Yutaro Muta @yutailang0119 ͸ͯͳϒοΫϚʔΫiOSΞϓϦͷ

   First Screen A/Bςετͷཪଆ with Firebase A/B Testing

2. • Yutaro Muta @yutailang0119 • Hatena Co., Ltd. @Kyoto •

   Conference Staff • builderscon 2017, 2018 • try! Swift Tokyo 2019, 2020 • ࠷ۙͷਪ͠͸Combine framework Who am I ?

3. ࿩͢͜ͱ • ͸ͯͳϒοΫϚʔΫͷiOSΞϓϦͰߦͬͨA/Bςετͷ֓ཁ • ্هʹ͓͚ΔFirebase A/B Testingͷ໾ׂ • ઃఆํ๏ •

   Tips

4. ࿩͞ͳ͍͜ͱ • AdMobʹΑΔΞϓϦ಺޿ࠂ

5. ͸ͯϒͷεϚϗΞϓϦΛ ࢖͍ͬͯΔਓʙ

6. ͸ͯϒεϚϗΞϓϦ • https:b.hatena.ne.jp ͷεϚϗΞϓϦ • ίϝϯτ෇͖ͰϒοΫϚʔΫͰ͖Δ • ײ૝Λڞ༗Ͱ͖Δ • ࿩୊ͷϖʔδΛൃݟͰ͖Δ

   • iOS / Android

7. ࠷ۙ͸ͯϒΞϓϦͷ ϗʔϜը໘͕มΘͬͨͳʔ ͱؾ͍ͮͨਓʙ

8. Before

9. After

10. Before After

11. Before After

12. มߋ఺ • First ScreenͰΧςΰϦຖͷਓؾʹ੾Γସ͑ΒΕΔ • ݩʑ͸૯߹ͷਓؾͱ৽ணͷΈ͕දࣔ͞Ε͍ͯͨ • ΧςΰϦͷฒͼॱ͸ɺ೚ҙʹฒͼସ͑ΒΕΔ

13. Ͱ΋ɺ First ScreenΛΤΠϠοͱ มߋ͢Δͷ͸ා͍ͳ͊

14. ͦ͜ͰA/Bςετ

15. A/BςετπʔϧʹٻΊͨ͜ͱ • [Must] A or Bͷ੾Γସ͑Λ֎෦ (ΞϓϦϦϦʔεαΠΫϧ֎) ͔Βߦ͑Δ • ঢ়ଶʹ߹Θͤͯը໘ଆΛ੾Γସ͑Δ

    • ςετதͷෆ۩߹΍݁Ռʹ໌Β͔ʹେ͖ͳ͕ࠩग़ͨࡍʹɺׂ߹Λมߋ • Ϗδωεతͳ͋Ε͜ΕʹରԠͰ͖ΔΑ͏ • [Better] ෼ੳπʔϧͱͷ૬ੑ΍ݕূͷ͠΍͢͞

16. Firebase A/B Testing

17. Firebase A/B Testing • ͢ͰʹFirebaseϓϥοτϑΥʔϜΛར༻͍ͯ͠Δͱɺಋೖίετ͕গͳ͍ • Google Analytics for Firebaseͱ࿈ܞ͕Մೳ

    • Remote Configͱͯ͠ѻ͏ͷͰɺखܰ

18. Firebase A/B Testingͷઃఆ

19. ઃఆελʔτ

20. λʔήοτͷઃఆ

21. λʔήοτͷઃఆ Analytics.logEvent(_:, parameters:) ͰૹΔΠϕϯτΛ࢖ͬͯɺ ಛఆͷૢ࡞Λͨ͠ϢʔβʔͷΈΛλʔήοτʹࢦఆՄೳ

22. ໨ඪઃఆ

23. ໨ඪઃఆ

24. όϦΞϯτͷઃఆ

25. ΞϓϦͷ࣮૷

26. Remote Config͔Β஋Λऔಘ // Fetch from Remote Config func fetch() {

    let expiration: TimeInterval = (60 * 60 * 3) remoteConfig.fetch(withExpirationDuration: expiration, completionHandler: nil) } // Activate fetched Remote Config settings func activateFetched() { remoteConfig.activateFetched() }

27. Remote ConfigͰऔಘͨ͠ઃఆ஋͔Βը໘Λग़͠෼͚Δ let viewController: UIViewController switch remoteConfig.viewSetting { case .a:

    viewController = AViewController() case .b: viewController = BViewController() } // Add to ContainerView viewController.view.translatesAutoresizingMaskIntoConstraints = false addChild(viewController) containerViewController.addSubview(viewController.view) viewController.didMove(toParent: self) NSLayoutConstraint.activate([NSLayoutConstraint]) // Switch ViewController by Remote Config let viewController: UIViewController switch remoteConfig { case .a: viewController = AViewController() case .b: viewController = BViewController() default: viewController = DefaultViewController() } // Add to ContainerView viewController.view.translatesAutoresizingMaskIntoConstraints = false addChild(viewController) containerViewController.addSubview(viewController.view) viewController.didMove(toParent: self) NSLayoutConstraint.activate([NSLayoutConstraint]())

28. ⷁ ⷂ

29. A/Bςετ։࢝

30. ~͕࣌ྲྀΕͯ~

31. ςετ݁Ռ • ͓Αͦ1ϲ݄΄ͲͷςετΛ࣮ࢪͨ͠ • ςετ࣮ࢪલ͔Β΄΅૝ఆ௨Γͷظؒ • ৽ը໘ͷํ͕ɺچը໘ΑΓ΋ઃఆͨ͠ࢦඪʹ͓͍ͯ༏ҐͳࠩΛಘΒΕͨ • ࢦඪ͸ൿີ

32. ίϯιʔϧը໘

33. ςετ׬ྃ

34. ςετ݁ՌΛݩʹ... • ຬΛ࣋ͯ͠ɺ৽ը໘Λ100%ʹมߋ • AndroidͰ΋৽ը໘Λ࣮૷ɺϦϦʔε • ݁ՌɺAndroidͰ΋ྑ͍݁ՌΛಘΒΕͨ

35. A/B Testing Q&A

36. Q. ॳճىಈ࣌ʹ΋ A/BςετΛ൓ө͍ͨ͠

37. Q. ॳճىಈ࣌ʹ΋A/BςετΛ൓ө͍ͨ͠ • Remote Config͸ɺωοτϫʔΫ௨৴Λ௨ͯ͡ઃఆ஋Λऔಘ͢Δ • ॳճFetch࣌ʹ͸ઃఆ஋͸ແ͍ • ίʔυ্Ͱͷdefault஋͸༻ҙͰ͖Δ •

    (ಛʹ) First ScreenͷA/BςετͰ͸ɺ൓өλΠϛϯάʹ޻෉͕ඞཁʹͳΔ

38. ॳճFetchͷ޻෉Ҋ 1. Fetch͕׬ྃ͢Δ·Ͱɺը໘Λwait͢Δ Pros: ॳճ΋A/Bςετͷઃఆ஋Λ൓өͰ͖Δ Cons: ଴͕ͪ࣌ؒൃੜɺFetch͕ࣦഊͨ࣌͠͸Ͳ͏͢Δʁ 2. Fetch͕׬ྃͨ͠Βଈ࠲ʹ൓өͤ͞Δ ->

    ࠷ॳ͸default஋Ͱը໘Λදࣔ -> Fetchޙ͙͢ʹ RemoteConfig.activateFetched() -> ը໘ͷߋ৽ Pros: ࣮૷্ͷߟྀ఺͸গͳ͍ Cons: Ϣʔβʔ͸࢖༻தʹUI͕มߋ͞ΕΔͷͰɺڻ͔͘΋ 3. ࣍ճىಈ࣌·Ͱ଴ͭ -> ىಈ࣌ʹͷΈ RemoteConfig.activateFetched() Pros: ࣮૷্ͷߟྀ఺͸গͳ͍ Cons: 2ճ໨Ҏ߱ͷىಈʹͨͲΓண͚ͳ͍ͱɺςετͰ͖ͳ͍

39. Q. A/Bςετͷσόοά େม͡Όͳ͍ʁ

40. Q. A/Bςετͷσόοάେม͡Όͳ͍ʁ • betaϏϧυͰͷΈ Khan/SwiftTweaks Λ࢖͍ɺ੾Γସ͑Մೳʹͨ͠ • A/BςετҎ֎ʹ΋௨৴΍ͦͷଞઃఆͷ੾Γସ͑ɺσόοάʹ΋࢖͍ͬͯΔ

41. &OKPZ
    "#
    5FTUJOH
    5IBOLT w NVUBZVUBSP!HNBJMDPN
    w IUUQTUXJUUFSDPNZVUBJMBOH
    w IUUQTHJUIVCDPNZVUBJMBOH