Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AB test infrastructure

chimame
November 04, 2018

AB test infrastructure

JAWS FESTA 2018 OSAKA

chimame

November 04, 2018
Tweet

More Decks by chimame

Other Decks in Technology

Transcript

  1. "#UFTUJOGSBTUSVDUVSF
    +"84'&45"04","

    吳䒭⠓爡ؒ؎ث٦ي
    歊⿵ ⵃ㕼

    View Slide

  2. 荈䊹稱➜

    View Slide

  3. h"UFBN *OD
    荈䊹稱➜
    p/BNF 歊⿵ ⵃ㕼 ׋ת׋ ׶ה
    !SJUP
    p+PC TPGUXBSFEFWFMPQFS
    p4LJMMTFU 3VCZPO3BJMT 3FBDU (SBQI2-
    %PDLFS
    p0SHBOJ[FS 3BJMTGPMMPXVQ0TBLB
    p5XJUUFS [email protected]

    View Slide

  4. ؟٦ؽأ稱➜

    View Slide

  5. h"UFBN *OD
    ٍؗحءؚٝ٥ؕ٦س
    ٗ٦ٝ筨さ嗚稊؟؎ز
    穠㭴䒭㜥䞔㜠؟؎ز
    䒷馉׃嫰鯰٥✮秈؟؎ز
    ׉ך➭
    ـٓ؎
    تٕꟼ鸬
    荈⹛鮦
    ꟼ鸬
    䒷馉׃
    ꟼ鸬
    ꆃ輐
    ًر؍،
    鮦叨㹀٥鮦顠《؟؎ز
    㥍䚍ぢֽ欰椚傈✮䟝ֶ״ן⡤锃盖椚،فٔ
    فؚٗٓوぢֽך䪮遭䞔㜠Ⱏ剣؟٦ؽأ
    ♧䏝ךⰅ⸂ד剑㣐爡ך䒷馉׃噟罏ַ׵鋅琎׾《䖤
    䲿䵿䒷馉׃噟罏侧כⰋ㕂爡⟃♳
    噟歲ءؑ،زحفծ䒷馉׃ワ鴟؟٦ؽأ׮㢳侧㾜Ꟛ
    ♧䏝ךⰅ⸂ד剑㣐爡ך顠《׶噟罏ַ׵鋅琎׾《䖤
    ⿑姻ז㛇彊׾弫׋׃׋♧崧噟罏ה䲿䵿
    噟歲ءؑ،زحف
    Ⰻ㕂⟃♳ך穠㭴䒭㜥ַ׵⠓㜥׾׀稱➜
    فٗך،سغ؎ؠ٦ָ䌢꽎ׅ׷㹋䏄莧׾Ⰻ㕂䏄莧㾜Ꟛ
    չعشًِ㹀겘ؐؒر؍ؚٝպծչعشًِ1IPUPպ瘝׮㾜Ꟛ
    ⿑鼅׃׋㣐䩛ꋒ遤ծ嶊顤罏ꆃ輐ך؟٦ؽأ׾䗡䎿嫰鯰
    չشؽشؽٍؗحءؚٝպח⸇ִծչشؽشؽؙٖآحزؕ٦سպ
    չشؽشؽ⡝㸓ٗ٦ٝպչشؽشؽ'9պ׾㾜Ꟛ
    ٓ؎ؿأة؎ٕ؟ه٦ز✲噟 ⚺銲؟٦ؽأ

    View Slide

  6. h"UFBN *OD
    شؽشؽ'9
    ֻ׷ת%&-*
    ؿ؋؎ٝسفٗ
    ٓ؎ؿأة؎ٕ؟ه٦ز✲噟 倜鋉؟٦ؽأ
    -JGFٓ؎ؿسحز
    4PMEJإر؍
    شؽشؽ⡝㸓ٗ٦ٝ ׅתְֲ׷
    NJOPSJFىظٔؒ
    荈⹛鮦ꟼ鸬
    ꆃ輐ًر؍،
    ׉ך➭欰崞ꟼ鸬
    ׉ך➭ٓ؎ؿؒٝر؍ؚٝ
    ꆃ輐ًر؍،
    ׉ך➭㹺鎘٥眍秈
    ׉ך➭♶⹛欵
    倜鋉؟٦ؽأ
    ׉ך➭قٕأ؛،
    չٕٓ٦ٝպ
    ِ٦ؠ٦ך㡮
    ַ׵⡲׵׸׋
    㦄崞؟فٔ&$
    ♶⹛欵㡰⽱䞔
    㜠 嗚 稊 ؟ ؎
    ز٥♶⹛欵♧
    䭍叨㹀؟؎ز
    ٓ ؎ ؿ ؒ ٝ
    ر؍ؚٝ䞔㜠
    嗚稊؟؎ز
    ⚥〢荈⹛鮦ך
    ؎ٝة٦
    طحز顋㡰
    فָٗ侄ִ׷
    ⡝ 㸓 ٗ ٦ ٝ
    䞔㜠؟؎ز
    鸐⥋顤٥㹺鎘
    鋅湫׃؟؎ز
    ⴱ 䗰 罏 ך '9
    ز ٖ ٦ س ؟
    ه٦ز،فٔ
    ء ٗ ، ٔ ٥
    عث꽀ꤐזו
    ⳿䓸鏝㉏؟٦
    ؽأך嗚稊٥
    ✮秈؟؎ز

    View Slide

  7. ؟٦ؽأ何㊣铬겗

    View Slide

  8. h"UFBN *OD
    ؟٦ؽأ何㊣
    Ⰵ⸂׃חְֻؿؓ٦ي׾鏣縧ׅ׷הِ٦ؠךꨄ膴ָ涪欰

    View Slide

  9. h"UFBN *OD
    ؟٦ؽأ何㊣
    ⵃ欽罏ָ実׭׷䞔㜠װ鋅װְׅ٥Ⰵ⸂׃װְׅ؟٦ؽأה׃ג何㊣ׅ׷䗳
    銲ָ֮׷

    View Slide

  10. h"UFBN *OD
    "#ذأزخ٦ٕ
    "#ذأز׮♧鿇خ٦ٕ׾嗚鏾٥㼪Ⰵ׃ג㹋倵

    View Slide

  11. h"UFBN *OD
    "#ذأزך㹋倵倯岀
    知⽃ז"#ذأز㹋倵חכ⵸鶢ךخ٦ٕ׾⢪欽ׅ׷ָ%0.圓䧭ָ㣐ֹֻ㢌
    ׻׷״ֲז"#ذأزכ爡ⰻדꟚ涪ׁ׸׋杝荈فؚٗٓيSVCZHFNT׾
    ⢪欽׃ג㹋倵կ
    鑩䔲ل٦آך"#ذأز׾㹋倵ׅ׷׋׭ח鑩䔲ل٦آךا٦أ׾"#㹋倵ر
    ؍ؙٖزٔח؝ؾ٦׃ג㹋倵կ

    A
    B

    View Slide

  12. h"UFBN *OD
    "#ذأزך㹋倵倯岀
    ،فٔ
    ؛٦ءّٝ
    3VCZPO3BJMT
    4ZNGPOZ
    ىسٕؐؑ،
    /HJOY 1VNB "QBDIF
    .Z42- 1PTUHSF42-
    3FEJT .FNDBDIFE
    ؎ٝؿٓأ
    زؙٓثٍ
    طحزٙ٦ؙ
    إُؗٔذ؍ ⟎䟝⻉

    View Slide

  13. h"UFBN *OD
    "#ذأزך㹋倵ח״׷铬겗
    p ♧㹀ךٕ٦ٕדر؍ؙٖزٔ圓鸡׾⡲䧭׃זְהְֽזְָ➬圫ך椚
    鍑ָꨇ׃ְ
    p ؝ؾلׁ׸׋؝٦سָ㣐ꆀח涪欰׃ծ⟎חⰋ⡤ח⥜姻ָ䗳銲הז׏׋
    㜥さח⥜姻ꆀָ花㣐חז׷
    p "#ذأز׾遤׏גְ׷ل٦آדず♧ךػ٦ءٍٕ׾⿫撑׃גְ׷㜥さ
    ח♶Ⱗさ׾饯ֿ׃װְׅ

    View Slide

  14. 鍑寸倯岀
    GPS&$4

    View Slide

  15. h"UFBN *OD
    倜"#ذأزך㹋倵倯岀
    ،فٔ
    ؛٦ءّٝ
    3VCZPO3BJMT
    4ZNGPOZ
    ىسٕؐؑ،
    /HJOY 1VNB "QBDIF
    .Z42- 1PTUHSF42-
    3FEJT .FNDBDIFE
    ؎ٝؿٓ
    أزؙٓثٍ
    طحزٙ٦ؙ
    إُؗٔذ؍ ⟎䟝⻉
    ،فٔ؛٦ءّٝח⣛㶷׃זְ"#ذأز׾〳腉הׅ׷

    View Slide

  16. h"UFBN *OD
    倜"#ذأزך㹋倵倯岀
    ؝ٝذش׾⢪׏ג،فٔ؛٦ءّٝ׀ה"#ذأز׾㹋倵
    "
    #

    View Slide

  17. h"UFBN *OD
    倜"#ذأزך㹋倵倯岀
    ـٓٝث׀הח؝ٝذش׾⡲䧭׃ծرفٗ؎׾遤ֲ
    .BTUFSـٓٝث
    NFSHFEFQMPZ
    "#欽ـٓٝث
    NFSHFEFQMPZ

    View Slide

  18. h"UFBN *OD
    倜"#ذأزך㹋倵铬겗
    ،ؙإأ鿪䏝חل٦آָ㢌׻׏ג׃תֲ
    "
    #

    View Slide

  19. h"UFBN *OD
    倜"#ذأزך㹋倵铬겗
    䌢חずׄ؝ٝذشַ׵ؙٔؒأز׾鵤׃ג׮׵ֲ
    #
    ず♧؝ٝذشפךؙٔؒأز׾笝䭯
    ׅ׷׋׭חכ"-#ךUBSHFUHSPVQך
    4UJDLZ4FTTJPO׾剣⸬חׅ׷

    View Slide

  20. h"UFBN *OD
    ⽃秪ח؝ٝذش׾㟓װׅה"#ذأزך鎘庠ח儗꟦ַַָ׷
    涫ꐮ歗꬗ 然钠歗꬗
    倜"#ذأزך㹋倵铬겗
    "
    #
    B
    C
    涫ꐮ歗꬗
    ذأز
    然钠歗꬗
    ذأز
    ؔٔآشٕ
    B
    "
    "#ذأز׾遤ֲ؝ٝذشָ
    㟓ִ׸ל㟓ִ׷קוծ،ؙإ
    أ׀הח؝ٝذش׾ⶴ׶䮶׷
    ךדر٦ةָ꧊ת׷תד儗꟦
    ַַָ׷

    View Slide

  21. h"UFBN *OD
    "#ذأز63-⽃⡘חٕ٦ذ؍ֶؚٝ״ןUBSHFUHSPVQ׾鷄⸇׃ג㹋倵
    倜"#ذأزך㹋倵铬겗
    "
    # 涫ꐮ歗꬗
    ذأز
    然钠歗꬗
    ذأز
    ؔٔآشٕ
    "
    "
    ؔٔآشٕ
    C
    B
    B
    B
    然钠歗꬗
    涫ꐮ歗꬗
    "#ذأزךٕ٦ذ؍ؚٝⴓ
    UBSHFUHSPVQ׾⡲䧭


    View Slide

  22. h"UFBN *OD
    "#ذأز㹋倵⚥חؔٔآشٕך㢌刿ָ涪欰ׅ׷
    倜"#ذأزך㹋倵铬겗
    "
    # 涫ꐮ歗꬗
    ذأز
    倜ؔٔآشٕ
    傊ؔٔآشٕ
    "
    "
    傊ؔٔآشٕ
    NBTUFSـٓٝث
    NFSHFEFQMPZ
    ؔٔآشָٕ
    〢ְתת


    ؔٔآشٕ UBSHFUHSPVQ
    "#ذأز UBSHFUHSPVQ

    View Slide

  23. h"UFBN *OD
    ؔٔآشٕך㢌刿׾ׅץגךؔٔآشٕח׮黝欽ׅ׷$*׾欽䠐
    倜"#ذأزך㹋倵铬겗
    "
    # 涫ꐮ歗꬗
    ذأز
    倜ؔٔآشٕ
    傊ؔٔآشٕ
    "
    " 傊ؔٔآشٕ
    EFQMPZ
    " 倜ؔٔآشٕ
    ؔٔآشٕ UBSHFUHSPVQ
    "#ذأز UBSHFUHSPVQ

    View Slide

  24. תה׭

    View Slide

  25. h"UFBN *OD
    תה׭
    p ؎ٝؿٓؒٝآص،ׄׯזֻג׮ֿֿתד〳腉濼陎ה׃גכ䗳銲
    p ⢪ֲ➂➙㔐כ⚺חؿٗٝزך铬겗׾荈׵䭪ְ何㊣׃גְֻ
    p ׅץגך铬겗ָ鍑寸דֹ׷ꋒך䔉⚵דכזְ醱侧ل٦آ׾騖ּ"#
    ָծ➙״׶㼰׃ד׮⵸פ
    p 嚂׃ְַծؙؙׅٙٙ׷ַ㣐✲
    p ➙㔐כֻ֮תד♧⢽ד֮׶ծת׌ת׌װ׶׋ְֿהָ֮׷ךדؒٝآ
    ص،٥رؠ؎ش٦瘝㉏׻׆ծ׈מؒ؎ث٦يח

    View Slide

  26. h"UFBN *OD
    ؎كٝز稱➜

    View Slide

  27. 穗㌀椚䙀
    չ׫׿זד䎊ׇחז׸׷⠓爡חׅ׷ֿהպ
    չ➙ַ׵䎃竲ֻ⠓爡חׅ׷ֿהպ

    View Slide