Upgrade to Pro — share decks privately, control downloads, hide ads and more …

고객 경험을 개선하는 A/B 테스트 기반 모바일 앱 개발

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →

고객 경험을 개선하는 A/B 테스트 기반 모바일 앱 개발

- 2021년 11월 3일 GDG DevFest Seoul
- 2021년 10월 22일 CJ AI-Biz Meetup
- 2021년 10월 16일 DDD 개발자 세미나
위 행사에서 발표한 자료입니다.

패스트캠퍼스 더레드 강의 자료의 일부이기도 합니다.
강의: https://fastcampus.co.kr/dev_red_lsm

A/B 테스트를 통해 데이터 기반으로 앱 서비스를 만드는 과정을 모바일 개발자 관점에서 정리하였습니다. 데이터 기반으로 서비스 가치를 창출하는 모바일 개발자의 새로운 성장 방향을 소개합니다.

Seungmin 마량

October 16, 2021

Transcript

106 slides
  1. Ҋё ҃೷ਸ ѐࢶೞח A/B పझ౟ ӝ߈ ݽ߄ੌ জ ѐߊ ߛ௼࢟۞٘

    UX Strategy ౱ Product Manager Google Developers Experts Android Korea ੉थ޹
  2. ࢲ࠺झ ѐߊ਷… ӏݽо ੓ח ࢲ࠺झח ޖ঺੉ ੢੼੉Ҋ ѐࢶ೧ঠ ೞח૑ ঌӝ

    য۵णפ׮. Ӓېࢲ ؘ੉ఠܳ ࠁҊ ੋࢎ੉౟ܳ ঳য ӝദ ੿ഛبܳ ֫ৈঠ ೤פ׮. ӏݽо ੓ח ࢲ࠺झח ݅ٚ ӝמ੉ ࢎਊ੗о ਗೞח Ѫੋ૑ ঌӝ য۵णפ׮. Ӓېࢲ प೷ਵ۽ ࢎਊ੗ ೯زਸ ஏ੿ೞҊ ഛܫ੉ ֫਷ Ѿ੿ਸ ೧ঠ ೤פ׮. ӏݽо ੓ח ઑ૒਷ ইޖܻ ؀಴о ૒ҙ੉ જইب ݽٚ Ѫਸ Ѿ੿ೡ ࣻ হणפ׮. Ӓېࢲ ࣁࠗ ౱ب ੿ഛೠ ੄ࢎѾ੿ਸ ೡ ࣻ ੓ب۾ प೷೧ঠ ೤פ׮.
  3. ݽ߄ੌ ѐߊ੗ח… ঱যח ߊ੹ೞҊ, пઙ ో੉ աয়ݶࢲ ѐߊ਷ ազ੉ एਕ૑Ҋ

    ੓णפ׮. ӝࣿਸ ੜ ׮ܖח Ѫ݅ਵ۽ ࢿ੢ೞח ѐߊ੗੄ द؀ח ૑աоҊ ੓णפ׮. ӝࣿਸ ࣻױਵ۽ॄ ࢲ࠺झܳ ݅٘ח ѐߊ੗۽ ࢿ੢ೞৈঠ ೤פ׮. ࢲ࠺झীࢲ ӝഥܳ ଺Ҋ झझ۽ ࢿҗܳ ૐݺ೧ঠ ೤פ׮.
  4. ੉ ъ੄ח… प೷ ӝ߈ਵ۽ ݽ߄ੌ জ ࢲ࠺झܳ ѐࢶೞח җ੿ਸ ѐߊ੗

    ҙ੼ীࢲ ੿ܻೞ৓णפ׮. 1. ؘ੉ఠ ӝ߈ਵ۽ ݽ߄ੌ জ ࢲ࠺झܳ ݅٘ח ੹୓੸ੋ җ੿ਸ ੉೧೤פ׮. 2. ੉߮౟, ೖ୛೒ۨӒ, प೷ 3о૑ ఃਕ٘ܳ ੉೧೤פ׮. 3. ؘ੉ఠܳ ӝ߈ਵ۽ ࢲ࠺झ о஖ܳ ହ୹ೞח ݽ߄ੌ ѐߊ੗੄ ࢜۽਍ ࢿ੢ ߑೱਸ ੉೧೤פ׮.
  5. оࢸࢸ੿ -> प೷ -> ؘ੉ఠ ࠙ࢳ -> Ѿ੿ -> ੋࢎ੉౟

    ೟ण ۽Ӧ (੉߮౟ ѐߊ) ؘ੉ఠח যڌѱ ݽਸ ࣻ ੓ਸө
  6. ੉߮౟ ҳࢿ ചݶ ೯ز ఋ੉߁ য٣ীࢲ ೯زೞ৓חо যڃ ೯زਸ ೞ৓חо

    ঱ઁ ۽Ӧਸ ೡ Ѫੋо ੉ܴ ചݶ, ೯ز, ఋ੉߁ਸ ೞա੄ ੉ܴਵ۽ ಴അ ࣘࢿ э਷ ೯ز੉যب ׮ܳ ࣻ ੓ח ࢎਊ੗ / ੉߮౟ ࢚ടਸ ಴അ
  7. ੉߮౟ ੿੄ ৘द No, ചݶ ࢎਊ੗ ೯زҗ ఋ੉߁ ੉ܴ ࣘࢿ੉ܴ

    / ч 1 ੗࢑ ചݶ ૓ੑ enter__assets_home 2 - ੗࢑ زӝച ࢿҕ success__sync_assets 3 ஠٘୶ୌ ࢚ಿ ௿ܼ click__product_card ੉ܴ: id / ч: ࢚ಿ ই੉٣ ч ৘द: 3425, 1829 ੉ܴ: company / ч: ӝҙ ੉ܴ ч ৘द: नೠ஠٘, Ҵ޹஠٘ 4 ݽٚ ചݶ জ द੘ ನӒۄ਍٘ ചݶ੉ 0ѐীࢲ 1ѐо غח द੼ open_app ੉ܴ: os ч ৘द: android, ios, web
  8. binding.contactButton.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail()) startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${houseLiveData.value.contact}")) ) }

    fun logEvent(event: EventDefinition) { firebaseAnalytics.logEvent(event.eventName, Bundle()) amplitude.logEvent(event.eventName) } ੉߮౟ ௏٘ बӝ
  9. binding.contactButton.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail()) startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${houseLiveData.value.contact}")) ) }

    fun logEvent(event: EventDefinition) { firebaseAnalytics.logEvent(event.eventName, Bundle()) amplitude.logEvent(event.eventName) } ੉߮౟ ௏٘ बӝ ߡౡ ௿ܼ द ੉߮౟ ੹࣠
  10. binding.contactButton.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail()) startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${houseLiveData.value.contact}")) ) }

    fun logEvent(event: EventDefinition) { firebaseAnalytics.logEvent(event.eventName, Bundle()) amplitude.logEvent(event.eventName) } ੉߮౟ ௏٘ बӝ Firebase, Amplitudeী ੉߮౟ ੹࣠
  11. Enter ੉߮౟ যڌѱ ҳഅೡө Base ചݶ ௿ېझী ੉߮౟ ௏٘ ࢗੑ

    ചݶ߹۽ ੉߮౟ ੉ܴ ૑੿ ചݶ ૓ੑ ੉߮౟
  12. Enter ੉߮౟ ௏٘ class BaseActivity : Activity() { protected open

    var enterEvent: EventDefinition? = null override fun onResume() { super.onResume() if (enterEvent != null) { analyticsManager.logEvent(enterEvent!!) } } } class UploadTypeActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.uploadType() } class HouseListActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.houseList() }
  13. Enter ੉߮౟ ௏٘ class BaseActivity : Activity() { protected open

    var enterEvent: EventDefinition? = null override fun onResume() { super.onResume() if (enterEvent != null) { analyticsManager.logEvent(enterEvent!!) } } } class UploadTypeActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.uploadType() } class HouseListActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.houseList() } ചݶ૓ੑ ࢤݺ઱ӝী ੉߮౟ ௏٘ ࢗੑ
  14. Enter ੉߮౟ ௏٘ class BaseActivity : Activity() { protected open

    var enterEvent: EventDefinition? = null override fun onResume() { super.onResume() if (enterEvent != null) { analyticsManager.logEvent(enterEvent!!) } } } class UploadTypeActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.uploadType() } class HouseListActivity : BaseActivity() { override var enterEvent: EventDefinition? = EnterEventDefinitions.houseList() } ചݶ݃׮ ੉ܴ ੿੄
  15. Active User(AU) ӝળ ੉߮౟ ࢲ࠺झ ೨ब ੉߮౟ ‘জਸ ࢎਊೞҊ ੓ח

    ਬ੷੉׮’ܳ ౸ױೡ ࣻ ੓ח ೯زਸ ੿੄೧ঠ ೤פ׮. ࢲ࠺झо ઺ਃೞѱ ࢤпೞח ೨ब ࢎਊ੗ ೯زਸ ੿੄೧ঠ ೤פ׮. ࢲ࠺झ ઱ਃ ૑಴ܳ ؀಴ೞח ੉߮౟ܳ ੿೧ঠ ೤פ׮
  16. Active User(AU) ӝળ ੉߮౟ ৘द झ೒ېद ചݶਸ ૓ੑೞݶ AU۽ ನೣೠ׮

    ۽Ӓੋ റ ക ചݶী ૓ੑೞݶ AU۽ ನೣೠ׮ যڃ ചݶ੉ٚ ನӒۄ਍٘۽ ৢۄয়ݶ AU۽ ನೣೠ׮
  17. ࢲ࠺झ ೨ब ੉߮౟ ৘द ழݠझ ࢲ࠺झ੉޲۽ Ѿઁ ೯زਸ ઱ਃ ੉߮౟۽

    ஏ੿ೠ׮ SNS੉޲۽ ஶబஎ ࢤࢿ ೯زਸ ઱ਃ ੉߮౟۽ ஏ੿ೠ׮ ࠗز࢑ ઺ѐ ࢲ࠺झ੉޲۽ ࠗز࢑ োۅ ೯زਸ ઱ਃ ੉߮౟۽ ஏ੿ೠ׮
  18. ੉߮౟ ߊࢤ ࣻܳ ࠁח AU ࠙ࢳ enter - DAU 10݅ݺ

    ઺, 3݅ݺ੉ Ѥъ ചݶী ٜযৡ׮ user property - Ѥъ ചݶী ٜযয়ח 3݅ݺ ઺ 2݅ݺ੉ ѤъѨ૓ োز੉ غয੓׮ click - Ѥъ ചݶী ٜযয়ח 3݅ݺ ઺, 1ୌݺ੉ ߡౡਸ ׂ۞ ѤъѨ૓ োزਸ दبೠ׮
  19. ੉߮౟ ߊࢤ ࣻܳ ࠁח AU ࠙ࢳ enter - DAU 10݅ݺ

    ઺, 3݅ݺ੉ Ѥъ ചݶী ٜযৡ׮ user property - Ѥъ ചݶী ٜযয়ח 3݅ݺ ઺ 2݅ݺ੉ ѤъѨ૓ োز੉ غয੓׮ click - Ѥъ ചݶী ٜযয়ח 3݅ݺ ઺, 1ୌݺ੉ ߡౡਸ ׂ۞ ѤъѨ૓ োزਸ दبೠ׮ Ѥъ ӝמ਷ ਬ੷੄ 30%о ࠁҊ, Ӓ઺ 66%о োزغয ੓׮ োزغয੓૑ ঋ਷ ࢎਊ੗ ઺ 10%о োزਸ दبೠ׮
  20. ౠ੿ ೯ز ੉റ ੌ੿ दр ٍ جইয়ח ܻబ࣌ ࠙ࢳ ੗࢑

    োزਸ दب೮؍ ࢎۈ਷ दبೞ૑ ঋও؍ ࢎۈࠁ׮ (Click) 7ੌ ղ জਸ ੤ߑޙೞח ࠺ਯ੉ (open_app) 10% ֫׮
  21. ౠ੿ ೯ز ੉റ ੌ੿ दр ٍ جইয়ח ܻబ࣌ ࠙ࢳ ੗࢑

    োزਸ दب೮؍ ࢎۈ਷ दبೞ૑ ঋও؍ ࢎۈࠁ׮ (Click) 7ੌ ղ জਸ ੤ߑޙೞח ࠺ਯ੉ (open_app) 10% ֫׮ ੗࢑ োزਸ औѱ दبೡ ࣻ ੓ب۾ ѐࢶ೧ঠѷ׮
  22. ױ҅߹۽ ੉ఎܫਸ ࠁח ௏ഐ౟ ࠙ࢳ ഥਗоੑਸ ਤ೧ Ѣ୛ঠೞח 3ѐ੄ ചݶ

    ઺ ࠄੋੋૐਸ ೞח 2ߣ૩ ചݶীࢲ ੉ఎ੉ ݆׮
  23. ഥਗоੑਸ ਤ೧ Ѣ୛ঠೞח 3ѐ੄ ചݶ ઺ ࠄੋੋૐਸ ೞח 2ߣ૩ ചݶীࢲ

    ੉ఎ੉ ݆׮ ࠄੋੋૐਸ औѱ ೡ ࣻ ੓ب۾ ѐࢶ೧ঠѷ׮ ױ҅߹۽ ੉ఎܫਸ ࠁח ௏ഐ౟ ࠙ࢳ
  24. ӝמՙܻ ઱ਃ ૑಴ ࠺Ү জ ੹୓ AU ઺ ੗࢑ AUח

    80%, о҅ࠗ AUח 60%, ੗زର AUח 10%੉׮ ֢റח AUо ծ૑ ঋ૑݅, ܻబ࣌੉ ݽٚ ӝמ ઺ ઁੌ ծ׮
  25. ӝמՙܻ ઱ਃ ૑಴ ࠺Ү জ ੹୓ AU ઺ ੗࢑ AUח

    80%, о҅ࠗ AUח 60%, ੗زର AUח 10%੉׮ ֢റח AUо ծ૑ ঋ૑݅, ܻబ࣌੉ ݽٚ ӝמ ઺ ઁੌ ծ׮ ੗زର৬ ֢റח ࢲ࠺झ੄ ઱ਃ ӝמ੉ ইפ׮
  26. Feature Flag۽ ч ߸҃ೞӝ private fun setContactButton() { binding.contactButton.text =

    RemoteConfig.getString(ConfigVariable.ContactButtonText) }
  27. Feature Flag۽ ч ߸҃ೞӝ private fun setContactButton() { binding.contactButton.text =

    RemoteConfig.getString(ConfigVariable.ContactButtonText) }
  28. private fun setContactButton() { if (RemoteConfig.getBoolean(ConfigVariable.MoveContactButtonBottomToFloating)) { binding.contactButton.visibility = View.GONE

    binding.contactButtonFab.visibility = View.VISIBLE } else { binding.contactButton.visibility = View.VISIBLE binding.contactButtonFab.visibility = View.GONE } } Feature Flag۽ ӝמ ఃҊ Սӝ
  29. private fun setContactButton() { if (RemoteConfig.getBoolean(ConfigVariable.MoveContactButtonBottomToFloating)) { binding.contactButton.visibility = View.GONE

    binding.contactButtonFab.visibility = View.VISIBLE } else { binding.contactButton.visibility = View.VISIBLE binding.contactButtonFab.visibility = View.GONE } } Feature Flag۽ ӝמ ఃҊ Սӝ
  30. زदী ݆਷ Ѫ੉ ߸҃غח ݽ߄ੌ ࢲ࠺झ ղ ੘স੄ ৔ೱبܳ ঌ۰ݶ

    प೷੉ ೙ਃ ղ ੘স੄ ৔ೱبо ҾӘ೧ਃ
  31. ࢎਊ੗ Ӓܛ ա־ӝ Ӓܛ߹۽ ӝמਸ ׮ܰѱ ҳഅ Ӓܛ߹۽ ؘ੉ఠܳ ࠙ࢳ

    ੘স੄ ৔ೱبܳ ੿ഛ൤ ঌӝਤೠ प೷ ҳഅ
  32. ࢎਊ੗ Ӓܛ ա־ӝ Ӓܛ߹۽ ӝמਸ ׮ܰѱ ҳഅ Ӓܛ߹۽ ؘ੉ఠܳ ࠙ࢳ

    ੘স੄ ৔ೱبܳ ੿ഛ൤ ঌӝਤೠ प೷ ҳഅ
  33. प೷ ؀࢚ ઺ 50%о प೷ҵ ݻݺঀ ա־חо ࢎਊ੗ 100% ઺

    50%о प೷ҵ ࢎਊ੗ 50% ઺ 25%о प೷ҵ
  34. ࢎਊ੗ Ӓܛ ա־ӝ Ӓܛ߹۽ ӝמਸ ׮ܰѱ ҳഅ Ӓܛ߹۽ ؘ੉ఠܳ ࠙ࢳ

    ੘স੄ ৔ೱبܳ ੿ഛ൤ ঌӝਤೠ प೷ ҳഅ
  35. private fun setContactButton() { if (RemoteConfig.getBoolean(ConfigVariable.MoveContactButtonBottomToFloating)) { binding.contactButton.visibility = View.GONE

    binding.contactButtonFab.visibility = View.VISIBLE } else { binding.contactButton.visibility = View.VISIBLE binding.contactButtonFab.visibility = View.GONE } } Feature Flagী ٮۄ ӝמ ࠙ӝ
  36. private fun setContactButton() { if (RemoteConfig.getBoolean(ConfigVariable.MoveContactButtonBottomToFloating)) { binding.contactButton.visibility = View.GONE

    binding.contactButtonFab.visibility = View.VISIBLE } else { binding.contactButton.visibility = View.VISIBLE binding.contactButtonFab.visibility = View.GONE } } Feature Flagী ٮۄ ӝמ ࠙ӝ
  37. Feature Flagী ٮۄ ੉߮౟ ѐߊ binding.contactButton.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail())

    startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${contact}")) ) } binding.contactButtonFab.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail()) startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${contact}")) ) }
  38. Feature Flagী ٮۄ ੉߮౟ ѐߊ binding.contactButton.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail())

    startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${contact}")) ) } binding.contactButtonFab.setOnClickListener { view -> analyticsManager.logEvent(EventDefinitions.clickContactHouseDetail()) startActivity( Intent(Intent.ACTION_DIAL, Uri.parse("tel:${contact}")) ) } ೞա੄ प೷ীࢲ ࠙ӝغח ׮ܲ ӝמٜীࢲ э਷ ੉߮౟ܳ ੹࣠
  39. ࢎਊ੗ Ӓܛ ա־ӝ Ӓܛ߹۽ ӝמਸ ׮ܰѱ ҳഅ Ӓܛ߹۽ ؘ੉ఠܳ ࠙ࢳ

    ੘স੄ ৔ೱبܳ ੿ഛ൤ ঌӝਤೠ प೷ ҳഅ
  40. ݆਷ ੉߮౟ ؘ੉ఠ ઺ ޖ঺ਸ ࠺Үೡө प೷ оࢸী ؀਽ೞח Primary

    Metric ೞա੄ ૑಴ܳ ࠺Ү Second Metric਷ ଵҊ
  41. प೷ оࢸҗ Metric ৘द оࢸ ಹद੄ о҅ࠗ ৘࢑ࢸ੿ ޙҳܳ ߸҃ೞ

    ݶ, ৘࢑ࢸ੿ ৮ܐࣻо ֫ই૕ Ѫ੉׮. оࢸ ୶ୌచ ࢚ױ ߓցܳ ࢏ઁೞݶ ࢚ಿ ࣂ࣌੉ ਤ۽ ৢۄ৬, ࢚ಿ ௿ܼࣻо ֫ই૕ Ѫ੉׮. Primary Metric ৘࢑ࢸ੿ ৮ܐ ࣻ Second Metric ৘࢑ࢸ੿ ചݶ ߑޙ ࣻ Primary Metric ݽٚ ࢚ಿ ௿ܼ ࣻ ೤ Second Metric ୶ୌచ ߑޙ ࣻ ஠٘, ࠁ೷, ؀୹ ١ п ࢚ಿٜ੄ ௿ܼ ࣻ प೷ ࢸ҅ ױ҅ীࢲ оࢸҗ Metric ੿੄
  42. ؘ੉ఠ ӝ߈ਵ۽ ࢲ࠺झ о஖ ହ୹ਸ ઱بೞח ѐߊ੗ ӝഥ ߊҷ प೷

    ࢸ҅ ѐߊ ؘ੉ఠ ࠙ࢳ ࢿҗ ب୹ ੋࢎ੉౟ ೟ण ׮द ӝഥ ߊҷ
  43. ӝഥ ߊҷ प೷ ࢸ҅ ѐߊ ؘ੉ఠ ࠙ࢳ ࢿҗ ب୹ ੋࢎ੉౟

    ೟ण ׮द ӝഥ ߊҷ ࢲ࠺झী যڃ ߸҃ਸ ઱ݶ ௾ ࢿҗо զө? оࢸ Ҋউ ؘ੉ఠ ӝ߈ਵ۽ ࢲ࠺झ о஖ ହ୹ਸ ઱بೞח ѐߊ੗
  44. ӝഥ ߊҷ प೷ ࢸ҅ ѐߊ ؘ੉ఠ ࠙ࢳ ࢿҗ ب୹ ੋࢎ੉౟

    ೟ण ׮द ӝഥ ߊҷ ؘ੉ఠ ӝ߈ਵ۽ ࢲ࠺झ о஖ ହ୹ਸ ઱بೞח ѐߊ੗ оࢸਸ ࣻ஖ചػ ࢿҗ۽ ੿ഛ൤ Ѩૐ
  45. ӝഥ ߊҷ प೷ ࢸ҅ ѐߊ ؘ੉ఠ ࠙ࢳ ࢿҗ ب୹ ੋࢎ੉౟

    ೟ण ׮द ӝഥ ߊҷ ೟णೠ ੋࢎ੉౟۽ ؊ ੿ഛೠ оࢸ Ҋউ ؘ੉ఠ ӝ߈ਵ۽ ࢲ࠺झ о஖ ହ୹ਸ ઱بೞח ѐߊ੗
  46. ݽ߄ੌ ѐߊ੗ח ӝഥо ݆׮ ࢎਊ੗৬ ݏ׿ח ݽ߄ੌ ѐߊ ӝഥܳ ߊҷೞҊ

    प೷ਵ۽ ૐݺೞࣁਃ ࢲ࠺झ৬ ழܻযо ೣԋ ࢿ੢೤פ׮

More Decks by Seungmin 마량

Other Decks in Technology