Upgrade to Pro — share decks privately, control downloads, hide ads and more …

モバイルアプリのABテスト / CookpadTechConf2017

モバイルアプリのABテスト / CookpadTechConf2017

Cookpad TechConf 2017 ( https://techconf.cookpad.com/2017/ ) で発表したモバイルアプリのABテストのお話です

KATO Ryu (komatatsu)

January 21, 2017
Tweet

More Decks by KATO Ryu (komatatsu)

Other Decks in Technology

Transcript

  1. ϞόΠϧΞϓϦͷ"#ςετ
    ͔ͱ͏ΓΎ͏

    View Slide

  2. αʔϏε։ൃ෦Ϩγϐάϧʔϓ
    "OESPJEΤϯδχΞ
    $*ͳͲ͍ΖΜͳ΋ͷͷࣗಈԽ͕͖͢
    Ճ౻ཾ @k0matatsu

    View Slide

  3. ͓͠ͳ͕͖
    w 8FCͷ"#ςετ
    w ϞόΠϧΞϓϦͷ"#ςετ
    w ར༻ࣄྫ

    View Slide

  4. 8FCͷ"#ςετ

    View Slide

  5. $IBOLP&BTZ"CʹΑΔ"#ςετ
    $IBOLP
    w ϓϩτλΠϐϯάπʔϧ
    w औΓ֎͠΍͍͢Α͏ʹػೳ௥Ճ͢Δҝͷ࢓૊Έ
    &BTZ"C
    w "#ςετ༻ͷ$IBOLPFYUFOTJPO
    w $IBOLPͰ࡞ͬͨػೳΛಠࣗʹఆٛͨ͠Ϣʔβʔηάϝϯτ
    ʹैͬͯ؆୯ʹৼΓ෼͚Δࣄ͕Ͱ͖Δ
    ͜ΕΒΛ૊Έ߹ΘͤΔࣄͰ"#ςετΛ࣮ݱ͍ͯ͠·ͨ͠

    View Slide

  6. w 8FC΄ͲසൟʹσϓϩΠͰ͖ͳ͍
    w ి೾ঢ়گʹΑͬͯ௨৴ʹࣦഊ͢Δ৔߹͕͋Δ
    w ΞοϓσʔτΛڧ੍Ͱ͖ͳ͍
    w ͢΂ͯͷϢʔβʔʹߦ͖౉Δ·Ͱ͕͔͔࣌ؒΔ
    ɹϞόΠϧΞϓϦ͸8FCͱൺ΂Δͱաࠅͳ؀ڥ
    8FCͱϞόΠϧΞϓϦͷҧ͍

    View Slide

  7. ϞόΠϧΞϓϦͷ
    "#ςετ

    View Slide

  8. ϞόΠϧ޲͚"#ιϦϡʔγϣϯ
    w 044
    w $MVUDIJP
    w 4XJUDICPBSE
    w αʔϏε
    w 0QUJNJ[FMZ
    w "QQUJNJ[F
    w 'JSFCBTF3FNPUF$POpH
    ͳͲͳͲ

    View Slide

  9. ࣾ಺ͷओͳཁ݅
    w ؅ཧ͞Ε͍ͯΔػೳͱύλʔϯΛऔಘͰ͖Δ
    w ֤ػೳͰͲͷύλʔϯ͕ద༻͞Ε͍ͯΔ͔Θ͔Δ
    w ֤ػೳ͕ύλʔϯͱ͸ผʹ༗ޮແޮͷঢ়ଶΛ࣋ͭ
    w ಛఆͷϢʔβʔʹͲͷػೳŋύλʔϯΛద༻͢Δ͔ಈత
    ʹมߋՄೳ
    w ػೳͱύλʔϯͷׂ౰͸αʔόʔଆͰߦ͏
    w ࣾ಺ʹଘࡏ͢ΔطଘͷϢʔβʔηάϝϯτΛྲྀ༻͍ͨ͠
    w ,1*؅ཧ΍τϥοΩϯάʹ͸طଘπʔϧΛ࢖͏

    View Slide

  10. VTFSGFBUVSFT
    w Ϣʔβʔ͝ͱʹద༻͞Ε͍ͯΔػೳͷ૊Έ߹ΘͤΛऔಘ
    w ΦϑϥΠϯ࣌ʹɺ࠷ޙʹ௨৴੒ޭͨ͠৔߹ͷ஋Λ࠶ར༻
    w ۙʑͰ੒ޭͯ͠ͳ͍࣌͸ػೳΛແޮԽ
    w ΞϓϦ͕όοΫάϥ΢ϯυ͔Β෮ؼͨ͠Β࠶औಘ
    ͳͲͳͲ

    ϞόΠϧͷաࠅͳ؀ڥʹ଱͑Δػೳ

    View Slide

  11. w ZBNMͰઃఆΛॻ͍͓ͯ͘ͱϢʔβʔ͝ͱʹ౰ͯ͸·Δ
    ஋͕ฦ͞ΕΔ

    View Slide

  12. w ͜ͷΑ͏ͳ+40/͕ؼͬͯ͘ΔͷͰΞϓϦଆͰ࣮૷Λ੾
    Γସ͑Δ
    w σϑΥϧτͰ͸දࣔ͠ͳ͍ͳͲࣦഊ࣌΋ࢹ໺ʹೖΕΔ
    “features”: {
    “cooking_log_android”: {
    “name”: “ྉཧه࿥”,
    “patterns”: {
    “standard”: { “name”: “ඪ४࢓༷” },
    “notice_card”: { “name”: “Χʔυ൛ͷ͓஌Βͤ” },
    “notice_notification”: { “name”: “௨஌൛ͷ͓஌Βͤ” },
    “disable_rollout”: { “name”: “ର৅֎” },
    },
    “enabled”: true,
    “selected”: “standard”
    ػೳ໊
    ࣗ෼ʹׂΓ౰ͯΒΕͨύλʔϯ
    ػೳ͝ͱͷύλʔϯͷϦετ
    Ϧετ͸ࣗ෼ʹׂΓ౰ͯΒΕ͍ͯΔ
    ΋ͷҎ֎ͷύλʔϯ΋ฦͬͯ͘Δ

    View Slide

  13. ͳͥ಺੡ͨ͠ͷ͔ʁ
    w ͢Ͱʹ͋Δ΋ͷΛར༻͍ͨ͠
    w Ϣʔβʔηάϝϯτ
    w ϩάج൫
    w ݱ࣌఺Ͱ͸Ϣʔεέʔε͕ݶΒΕ͍ͯΔ
    w ཁૉͷग़͠෼͚΍ػೳͷ੾Γସ͙͑Β͍
    w 8FC͔ΒΞϓϦΛॻ͖׵͑ΔΑ͏ͳ͜ͱ͸͠ͳ͍

    View Slide

  14. ར༻ࣄྫ

    View Slide

  15. ϨγϐΤσΟλ
    w ඪ४͔ΜͨΜͷछྨͷΤσΟλΛ࢖ͬͯ౤ߘ཰ʹӨڹ
    ͕͋Δ͔Λௐ΂Δ"#ςετΛߦͬͨ
    w ܭଌ͸%#ʹΫΤϦΛ౤͛ͯTQSFBETIFFUͰूܭ
    w ͔ΜͨΜΤσΟλͷํ͕एׯ਺஋͕ѱ͍ͱ͍͏݁Ռʹʜ

    View Slide

  16. ϨγϐΤσΟλ

    View Slide

  17. ྉཧ͖Ζ͘
    w ୺຤಺ͷྉཧࣸਅΛࣗಈه࿥͠
    ͍ͯ͘৽ػೳΛ௥Ճ
    w 04ͷWFSTJPOͳͲͰߜͬͨϢʔ
    βʔʹର͠ެ։཰Λঃʑʹ͋͛
    ͳ͕ΒϩʔϧΞ΢τ
    w छҎ্ͷϩάΛऔ͓ͬͯΓɺ
    ར༻ऀͷ༷ࢠΛݟͳ͕Βެ։཰
    ͷௐ੔ͳͲͷରԠΛߦ͍ͬͯͨ

    View Slide

  18. ಋઢ௥Ճ
    w ৽ػೳ΁ͷಋઢΛ
    /PUJpDBUJPOͱ%JBMPHͱͷ
    ύλʔϯͰݕূ
    w ભҠ਺ɺڐ୚཰ɺ཭୤཰ͳ
    ͲΛݟͯ૯߹తʹ൑அͨ͠
    ݁Ռɺ/PUJpDBUJPOͷํ͕
    ʮݟ͔ͯΒڐ୚ը໘ʹભҠ
    ͢Δׂ߹͕ߴ͍ʯͱ͍͏͜
    ͱ͕Θ͔ͬͨ

    View Slide

  19. ·ͱΊ

    View Slide

  20. ·ͱΊ
    8FCͱಉ͡Α͏ʹ͸ग़དྷͳ͍
    ͳʹΛͲ͜·Ͱ΍Γ͍͔ͨɺ
    ཁٻʹ͋Θͤͯ࠷దͳํ๏Λ୳ͦ͏

    View Slide

  21. ͋Γ͕ͱ͏͍͟͝·ͨ͠

    View Slide