Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

고객 경험을 개선하는 A/B 테스트 기반 모바일 앱 개발

고객 경험을 개선하는 A/B 테스트 기반 모바일 앱 개발

- 2021년 11월 3일 GDG DevFest Seoul
- 2021년 10월 22일 CJ AI-Biz Meetup
- 2021년 10월 16일 DDD 개발자 세미나
위 행사에서 발표한 자료입니다.

패스트캠퍼스 더레드 강의 자료의 일부이기도 합니다.
강의: https://fastcampus.co.kr/dev_red_lsm

A/B 테스트를 통해 데이터 기반으로 앱 서비스를 만드는 과정을 모바일 개발자 관점에서 정리하였습니다. 데이터 기반으로 서비스 가치를 창출하는 모바일 개발자의 새로운 성장 방향을 소개합니다.

Seungmin 마량

October 16, 2021
Tweet

More Decks by Seungmin 마량

Other Decks in Technology

Transcript

  1. Ҋё ҃೷ਸ ѐࢶೞח A/B పझ౟ ӝ߈ ݽ߄ੌ জ ѐߊ ߛ௼࢟۞٘

    UX Strategy ౱ Product Manager Google Developers Experts Android Korea ੉थ޹
  2. ࢲ࠺झ ѐߊ਷… ӏݽо ੓ח ࢲ࠺झח ޖ঺੉ ੢੼੉Ҋ ѐࢶ೧ঠ ೞח૑ ঌӝ

    য۵णפ׮. Ӓېࢲ ؘ੉ఠܳ ࠁҊ ੋࢎ੉౟ܳ ঳য ӝദ ੿ഛبܳ ֫ৈঠ ೤פ׮. ӏݽо ੓ח ࢲ࠺झח ݅ٚ ӝמ੉ ࢎਊ੗о ਗೞח Ѫੋ૑ ঌӝ য۵णפ׮. Ӓېࢲ प೷ਵ۽ ࢎਊ੗ ೯زਸ ஏ੿ೞҊ ഛܫ੉ ֫਷ Ѿ੿ਸ ೧ঠ ೤פ׮. ӏݽо ੓ח ઑ૒਷ ইޖܻ ؀಴о ૒ҙ੉ જইب ݽٚ Ѫਸ Ѿ੿ೡ ࣻ হणפ׮. Ӓېࢲ ࣁࠗ ౱ب ੿ഛೠ ੄ࢎѾ੿ਸ ೡ ࣻ ੓ب۾ प೷೧ঠ ೤פ׮.
  3. ݽ߄ੌ ѐߊ੗ח… ঱যח ߊ੹ೞҊ, пઙ ో੉ աয়ݶࢲ ѐߊ਷ ազ੉ एਕ૑Ҋ

    ੓णפ׮. ӝࣿਸ ੜ ׮ܖח Ѫ݅ਵ۽ ࢿ੢ೞח ѐߊ੗੄ द؀ח ૑աоҊ ੓णפ׮. ӝࣿਸ ࣻױਵ۽ॄ ࢲ࠺झܳ ݅٘ח ѐߊ੗۽ ࢿ੢ೞৈঠ ೤פ׮. ࢲ࠺झীࢲ ӝഥܳ ଺Ҋ झझ۽ ࢿҗܳ ૐݺ೧ঠ ೤פ׮.
  4. ੉ ъ੄ח… प೷ ӝ߈ਵ۽ ݽ߄ੌ জ ࢲ࠺झܳ ѐࢶೞח җ੿ਸ ѐߊ੗

    ҙ੼ীࢲ ੿ܻೞ৓णפ׮. 1. ؘ੉ఠ ӝ߈ਵ۽ ݽ߄ੌ জ ࢲ࠺झܳ ݅٘ח ੹୓੸ੋ җ੿ਸ ੉೧೤פ׮. 2. ੉߮౟, ೖ୛೒ۨӒ, प೷ 3о૑ ఃਕ٘ܳ ੉೧೤פ׮. 3. ؘ੉ఠܳ ӝ߈ਵ۽ ࢲ࠺झ о஖ܳ ହ୹ೞח ݽ߄ੌ ѐߊ੗੄ ࢜۽਍ ࢿ੢ ߑೱਸ ੉೧೤פ׮.
  5. оࢸࢸ੿ -> प೷ -> ؘ੉ఠ ࠙ࢳ -> Ѿ੿ -> ੋࢎ੉౟

    ೟ण ۽Ӧ (੉߮౟ ѐߊ) ؘ੉ఠח যڌѱ ݽਸ ࣻ ੓ਸө
  6. ੉߮౟ ҳࢿ ചݶ ೯ز ఋ੉߁ য٣ীࢲ ೯زೞ৓חо যڃ ೯زਸ ೞ৓חо

    ঱ઁ ۽Ӧਸ ೡ Ѫੋо ੉ܴ ചݶ, ೯ز, ఋ੉߁ਸ ೞա੄ ੉ܴਵ۽ ಴അ ࣘࢿ э਷ ೯ز੉যب ׮ܳ ࣻ ੓ח ࢎਊ੗ / ੉߮౟ ࢚ടਸ ಴അ
  7. ੉߮౟ ੿੄ ৘द No, ചݶ ࢎਊ੗ ೯زҗ ఋ੉߁ ੉ܴ ࣘࢿ੉ܴ

    / ч 1 ੗࢑ ചݶ ૓ੑ enter__assets_home 2 - ੗࢑ زӝച ࢿҕ success__sync_assets 3 ஠٘୶ୌ ࢚ಿ ௿ܼ click__product_card ੉ܴ: id / ч: ࢚ಿ ই੉٣ ч ৘द: 3425, 1829 ੉ܴ: company / ч: ӝҙ ੉ܴ ч ৘द: नೠ஠٘, Ҵ޹஠٘ 4 ݽٚ ചݶ জ द੘ ನӒۄ਍٘ ചݶ੉ 0ѐীࢲ 1ѐо غח द੼ open_app ੉ܴ: os ч ৘द: android, ios, web
  8. binding.contactButton.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail()) startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${houseLiveData.value.contact}")) ) }

    fun logEvent(event: EventDefinition) { firebaseAnalytics.logEvent(event.eventName, Bundle()) amplitude.logEvent(event.eventName) } ੉߮౟ ௏٘ बӝ
  9. binding.contactButton.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail()) startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${houseLiveData.value.contact}")) ) }

    fun logEvent(event: EventDefinition) { firebaseAnalytics.logEvent(event.eventName, Bundle()) amplitude.logEvent(event.eventName) } ੉߮౟ ௏٘ बӝ ߡౡ ௿ܼ द ੉߮౟ ੹࣠
  10. binding.contactButton.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail()) startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${houseLiveData.value.contact}")) ) }

    fun logEvent(event: EventDefinition) { firebaseAnalytics.logEvent(event.eventName, Bundle()) amplitude.logEvent(event.eventName) } ੉߮౟ ௏٘ बӝ Firebase, Amplitudeী ੉߮౟ ੹࣠
  11. Enter ੉߮౟ যڌѱ ҳഅೡө Base ചݶ ௿ېझী ੉߮౟ ௏٘ ࢗੑ

    ചݶ߹۽ ੉߮౟ ੉ܴ ૑੿ ചݶ ૓ੑ ੉߮౟
  12. Enter ੉߮౟ ௏٘ class BaseActivity : Activity() { protected open

    var enterEvent: EventDefinition? = null override fun onResume() { super.onResume() if (enterEvent != null) { analyticsManager.logEvent(enterEvent!!) } } } class UploadTypeActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.uploadType() } class HouseListActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.houseList() }
  13. Enter ੉߮౟ ௏٘ class BaseActivity : Activity() { protected open

    var enterEvent: EventDefinition? = null override fun onResume() { super.onResume() if (enterEvent != null) { analyticsManager.logEvent(enterEvent!!) } } } class UploadTypeActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.uploadType() } class HouseListActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.houseList() } ചݶ૓ੑ ࢤݺ઱ӝী ੉߮౟ ௏٘ ࢗੑ
  14. Enter ੉߮౟ ௏٘ class BaseActivity : Activity() { protected open

    var enterEvent: EventDefinition? = null override fun onResume() { super.onResume() if (enterEvent != null) { analyticsManager.logEvent(enterEvent!!) } } } class UploadTypeActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.uploadType() } class HouseListActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.houseList() } ചݶ݃׮ ੉ܴ ੿੄
  15. Active User(AU) ӝળ ੉߮౟ ࢲ࠺झ ೨ब ੉߮౟ ‘জਸ ࢎਊೞҊ ੓ח

    ਬ੷੉׮’ܳ ౸ױೡ ࣻ ੓ח ೯زਸ ੿੄೧ঠ ೤פ׮. ࢲ࠺झо ઺ਃೞѱ ࢤпೞח ೨ब ࢎਊ੗ ೯زਸ ੿੄೧ঠ ೤פ׮. ࢲ࠺झ ઱ਃ ૑಴ܳ ؀಴ೞח ੉߮౟ܳ ੿೧ঠ ೤פ׮
  16. Active User(AU) ӝળ ੉߮౟ ৘द झ೒ېद ചݶਸ ૓ੑೞݶ AU۽ ನೣೠ׮

    ۽Ӓੋ റ ക ചݶী ૓ੑೞݶ AU۽ ನೣೠ׮ যڃ ചݶ੉ٚ ನӒۄ਍٘۽ ৢۄয়ݶ AU۽ ನೣೠ׮
  17. ࢲ࠺झ ೨ब ੉߮౟ ৘द ழݠझ ࢲ࠺झ੉޲۽ Ѿઁ ೯زਸ ઱ਃ ੉߮౟۽

    ஏ੿ೠ׮ SNS੉޲۽ ஶబஎ ࢤࢿ ೯زਸ ઱ਃ ੉߮౟۽ ஏ੿ೠ׮ ࠗز࢑ ઺ѐ ࢲ࠺झ੉޲۽ ࠗز࢑ োۅ ೯زਸ ઱ਃ ੉߮౟۽ ஏ੿ೠ׮
  18. ੉߮౟ ߊࢤ ࣻܳ ࠁח AU ࠙ࢳ enter - DAU 10݅ݺ

    ઺, 3݅ݺ੉ Ѥъ ചݶী ٜযৡ׮ user property - Ѥъ ചݶী ٜযয়ח 3݅ݺ ઺ 2݅ݺ੉ ѤъѨ૓ োز੉ غয੓׮ click - Ѥъ ചݶী ٜযয়ח 3݅ݺ ઺, 1ୌݺ੉ ߡౡਸ ׂ۞ ѤъѨ૓ োزਸ दبೠ׮
  19. ੉߮౟ ߊࢤ ࣻܳ ࠁח AU ࠙ࢳ enter - DAU 10݅ݺ

    ઺, 3݅ݺ੉ Ѥъ ചݶী ٜযৡ׮ user property - Ѥъ ചݶী ٜযয়ח 3݅ݺ ઺ 2݅ݺ੉ ѤъѨ૓ োز੉ غয੓׮ click - Ѥъ ചݶী ٜযয়ח 3݅ݺ ઺, 1ୌݺ੉ ߡౡਸ ׂ۞ ѤъѨ૓ োزਸ दبೠ׮ Ѥъ ӝמ਷ ਬ੷੄ 30%о ࠁҊ, Ӓ઺ 66%о োزغয ੓׮ োزغয੓૑ ঋ਷ ࢎਊ੗ ઺ 10%о োزਸ दبೠ׮
  20. ౠ੿ ೯ز ੉റ ੌ੿ दр ٍ جইয়ח ܻబ࣌ ࠙ࢳ ੗࢑

    োزਸ दب೮؍ ࢎۈ਷ दبೞ૑ ঋও؍ ࢎۈࠁ׮ (Click) 7ੌ ղ জਸ ੤ߑޙೞח ࠺ਯ੉ (open_app) 10% ֫׮
  21. ౠ੿ ೯ز ੉റ ੌ੿ दр ٍ جইয়ח ܻబ࣌ ࠙ࢳ ੗࢑

    োزਸ दب೮؍ ࢎۈ਷ दبೞ૑ ঋও؍ ࢎۈࠁ׮ (Click) 7ੌ ղ জਸ ੤ߑޙೞח ࠺ਯ੉ (open_app) 10% ֫׮ ੗࢑ োزਸ औѱ दبೡ ࣻ ੓ب۾ ѐࢶ೧ঠѷ׮
  22. ױ҅߹۽ ੉ఎܫਸ ࠁח ௏ഐ౟ ࠙ࢳ ഥਗоੑਸ ਤ೧ Ѣ୛ঠೞח 3ѐ੄ ചݶ

    ઺ ࠄੋੋૐਸ ೞח 2ߣ૩ ചݶীࢲ ੉ఎ੉ ݆׮
  23. ഥਗоੑਸ ਤ೧ Ѣ୛ঠೞח 3ѐ੄ ചݶ ઺ ࠄੋੋૐਸ ೞח 2ߣ૩ ചݶীࢲ

    ੉ఎ੉ ݆׮ ࠄੋੋૐਸ औѱ ೡ ࣻ ੓ب۾ ѐࢶ೧ঠѷ׮ ױ҅߹۽ ੉ఎܫਸ ࠁח ௏ഐ౟ ࠙ࢳ
  24. ӝמՙܻ ઱ਃ ૑಴ ࠺Ү জ ੹୓ AU ઺ ੗࢑ AUח

    80%, о҅ࠗ AUח 60%, ੗زର AUח 10%੉׮ ֢റח AUо ծ૑ ঋ૑݅, ܻబ࣌੉ ݽٚ ӝמ ઺ ઁੌ ծ׮
  25. ӝמՙܻ ઱ਃ ૑಴ ࠺Ү জ ੹୓ AU ઺ ੗࢑ AUח

    80%, о҅ࠗ AUח 60%, ੗زର AUח 10%੉׮ ֢റח AUо ծ૑ ঋ૑݅, ܻబ࣌੉ ݽٚ ӝמ ઺ ઁੌ ծ׮ ੗زର৬ ֢റח ࢲ࠺झ੄ ઱ਃ ӝמ੉ ইפ׮
  26. Feature Flag۽ ч ߸҃ೞӝ private fun setContactButton() { binding.contactButton.text =

    RemoteConfig.getString(ConfigVariable.ContactButtonText) }
  27. Feature Flag۽ ч ߸҃ೞӝ private fun setContactButton() { binding.contactButton.text =

    RemoteConfig.getString(ConfigVariable.ContactButtonText) }
  28. private fun setContactButton() { if (RemoteConfig.getBoolean(ConfigVariable.MoveContactButtonBottomToFloating)) { binding.contactButton.visibility = View.GONE

    binding.contactButtonFab.visibility = View.VISIBLE } else { binding.contactButton.visibility = View.VISIBLE binding.contactButtonFab.visibility = View.GONE } } Feature Flag۽ ӝמ ఃҊ Սӝ
  29. private fun setContactButton() { if (RemoteConfig.getBoolean(ConfigVariable.MoveContactButtonBottomToFloating)) { binding.contactButton.visibility = View.GONE

    binding.contactButtonFab.visibility = View.VISIBLE } else { binding.contactButton.visibility = View.VISIBLE binding.contactButtonFab.visibility = View.GONE } } Feature Flag۽ ӝמ ఃҊ Սӝ
  30. زदী ݆਷ Ѫ੉ ߸҃غח ݽ߄ੌ ࢲ࠺झ ղ ੘স੄ ৔ೱبܳ ঌ۰ݶ

    प೷੉ ೙ਃ ղ ੘স੄ ৔ೱبо ҾӘ೧ਃ
  31. ࢎਊ੗ Ӓܛ ա־ӝ Ӓܛ߹۽ ӝמਸ ׮ܰѱ ҳഅ Ӓܛ߹۽ ؘ੉ఠܳ ࠙ࢳ

    ੘স੄ ৔ೱبܳ ੿ഛ൤ ঌӝਤೠ प೷ ҳഅ
  32. ࢎਊ੗ Ӓܛ ա־ӝ Ӓܛ߹۽ ӝמਸ ׮ܰѱ ҳഅ Ӓܛ߹۽ ؘ੉ఠܳ ࠙ࢳ

    ੘স੄ ৔ೱبܳ ੿ഛ൤ ঌӝਤೠ प೷ ҳഅ
  33. प೷ ؀࢚ ઺ 50%о प೷ҵ ݻݺঀ ա־חо ࢎਊ੗ 100% ઺

    50%о प೷ҵ ࢎਊ੗ 50% ઺ 25%о प೷ҵ
  34. ࢎਊ੗ Ӓܛ ա־ӝ Ӓܛ߹۽ ӝמਸ ׮ܰѱ ҳഅ Ӓܛ߹۽ ؘ੉ఠܳ ࠙ࢳ

    ੘স੄ ৔ೱبܳ ੿ഛ൤ ঌӝਤೠ प೷ ҳഅ
  35. private fun setContactButton() { if (RemoteConfig.getBoolean(ConfigVariable.MoveContactButtonBottomToFloating)) { binding.contactButton.visibility = View.GONE

    binding.contactButtonFab.visibility = View.VISIBLE } else { binding.contactButton.visibility = View.VISIBLE binding.contactButtonFab.visibility = View.GONE } } Feature Flagী ٮۄ ӝמ ࠙ӝ
  36. private fun setContactButton() { if (RemoteConfig.getBoolean(ConfigVariable.MoveContactButtonBottomToFloating)) { binding.contactButton.visibility = View.GONE

    binding.contactButtonFab.visibility = View.VISIBLE } else { binding.contactButton.visibility = View.VISIBLE binding.contactButtonFab.visibility = View.GONE } } Feature Flagী ٮۄ ӝמ ࠙ӝ
  37. Feature Flagী ٮۄ ੉߮౟ ѐߊ binding.contactButton.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail())

    startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${contact}")) ) } binding.contactButtonFab.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail()) startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${contact}")) ) }
  38. Feature Flagী ٮۄ ੉߮౟ ѐߊ binding.contactButton.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail())

    startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${contact}")) ) } binding.contactButtonFab.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail()) startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${contact}")) ) } ೞա੄ प೷ীࢲ ࠙ӝغח ׮ܲ ӝמٜীࢲ э਷ ੉߮౟ܳ ੹࣠
  39. ࢎਊ੗ Ӓܛ ա־ӝ Ӓܛ߹۽ ӝמਸ ׮ܰѱ ҳഅ Ӓܛ߹۽ ؘ੉ఠܳ ࠙ࢳ

    ੘স੄ ৔ೱبܳ ੿ഛ൤ ঌӝਤೠ प೷ ҳഅ
  40. ݆਷ ੉߮౟ ؘ੉ఠ ઺ ޖ঺ਸ ࠺Үೡө प೷ оࢸী ؀਽ೞח Primary

    Metric ೞա੄ ૑಴ܳ ࠺Ү Second Metric਷ ଵҊ
  41. प೷ оࢸҗ Metric ৘द оࢸ ಹद੄ о҅ࠗ ৘࢑ࢸ੿ ޙҳܳ ߸҃ೞ

    ݶ, ৘࢑ࢸ੿ ৮ܐࣻо ֫ই૕ Ѫ੉׮. оࢸ ୶ୌచ ࢚ױ ߓցܳ ࢏ઁೞݶ ࢚ಿ ࣂ࣌੉ ਤ۽ ৢۄ৬, ࢚ಿ ௿ܼࣻо ֫ই૕ Ѫ੉׮. Primary Metric ৘࢑ࢸ੿ ৮ܐ ࣻ Second Metric ৘࢑ࢸ੿ ചݶ ߑޙ ࣻ Primary Metric ݽٚ ࢚ಿ ௿ܼ ࣻ ೤ Second Metric ୶ୌచ ߑޙ ࣻ ஠٘, ࠁ೷, ؀୹ ١ п ࢚ಿٜ੄ ௿ܼ ࣻ प೷ ࢸ҅ ױ҅ীࢲ оࢸҗ Metric ੿੄
  42. ؘ੉ఠ ӝ߈ਵ۽ ࢲ࠺झ о஖ ହ୹ਸ ઱بೞח ѐߊ੗ ӝഥ ߊҷ प೷

    ࢸ҅ ѐߊ ؘ੉ఠ ࠙ࢳ ࢿҗ ب୹ ੋࢎ੉౟ ೟ण ׮द ӝഥ ߊҷ
  43. ӝഥ ߊҷ प೷ ࢸ҅ ѐߊ ؘ੉ఠ ࠙ࢳ ࢿҗ ب୹ ੋࢎ੉౟

    ೟ण ׮द ӝഥ ߊҷ ࢲ࠺झী যڃ ߸҃ਸ ઱ݶ ௾ ࢿҗо զө? оࢸ Ҋউ ؘ੉ఠ ӝ߈ਵ۽ ࢲ࠺झ о஖ ହ୹ਸ ઱بೞח ѐߊ੗
  44. ӝഥ ߊҷ प೷ ࢸ҅ ѐߊ ؘ੉ఠ ࠙ࢳ ࢿҗ ب୹ ੋࢎ੉౟

    ೟ण ׮द ӝഥ ߊҷ ؘ੉ఠ ӝ߈ਵ۽ ࢲ࠺झ о஖ ହ୹ਸ ઱بೞח ѐߊ੗ оࢸਸ ࣻ஖ചػ ࢿҗ۽ ੿ഛ൤ Ѩૐ
  45. ӝഥ ߊҷ प೷ ࢸ҅ ѐߊ ؘ੉ఠ ࠙ࢳ ࢿҗ ب୹ ੋࢎ੉౟

    ೟ण ׮द ӝഥ ߊҷ ೟णೠ ੋࢎ੉౟۽ ؊ ੿ഛೠ оࢸ Ҋউ ؘ੉ఠ ӝ߈ਵ۽ ࢲ࠺झ о஖ ହ୹ਸ ઱بೞח ѐߊ੗
  46. ݽ߄ੌ ѐߊ੗ח ӝഥо ݆׮ ࢎਊ੗৬ ݏ׿ח ݽ߄ੌ ѐߊ ӝഥܳ ߊҷೞҊ

    प೷ਵ۽ ૐݺೞࣁਃ ࢲ࠺झ৬ ழܻযо ೣԋ ࢿ੢೤פ׮