Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AB test infrastructure

chimame
November 04, 2018

AB test infrastructure

JAWS FESTA 2018 OSAKA

chimame

November 04, 2018
Tweet

More Decks by chimame

Other Decks in Technology

Transcript

  1. h"UFBN *OD  荈䊹稱➜ p/BNF 歊⿵ ⵃ㕼 ׋ת׋ ׶ה !SJUP

    p+PC TPGUXBSFEFWFMPQFS p4LJMMTFU 3VCZPO3BJMT 3FBDU (SBQI2- %PDLFS p0SHBOJ[FS 3BJMTGPMMPXVQ0TBLB p5XJUUFS !DIJNBNF@SU
  2. h"UFBN *OD  ٍؗحءؚٝ٥ؕ٦س ٗ٦ٝ筨さ嗚稊؟؎ز 穠㭴䒭㜥䞔㜠؟؎ز 䒷馉׃嫰鯰٥✮秈؟؎ز ׉ך➭ ـٓ؎ تٕꟼ鸬

    荈⹛鮦 ꟼ鸬 䒷馉׃ ꟼ鸬 ꆃ輐 ًر؍، 鮦叨㹀٥鮦顠《؟؎ز  㥍䚍ぢֽ欰椚傈✮䟝ֶ״ן⡤锃盖椚،فٔ  فؚٗٓوぢֽך䪮遭䞔㜠Ⱏ剣؟٦ؽأ  ♧䏝ךⰅ⸂ד剑㣐爡ך䒷馉׃噟罏ַ׵鋅琎׾《䖤  䲿䵿䒷馉׃噟罏侧כⰋ㕂爡⟃♳  噟歲ءؑ،زحفծ䒷馉׃ワ鴟؟٦ؽأ׮㢳侧㾜Ꟛ  ♧䏝ךⰅ⸂ד剑㣐爡ך顠《׶噟罏ַ׵鋅琎׾《䖤  ⿑姻ז㛇彊׾弫׋׃׋♧崧噟罏ה䲿䵿  噟歲ءؑ،زحف  Ⰻ㕂⟃♳ך穠㭴䒭㜥ַ׵⠓㜥׾׀稱➜  فٗך،سغ؎ؠ٦ָ䌢꽎ׅ׷㹋䏄莧׾Ⰻ㕂䏄莧㾜Ꟛ  չعشًِ㹀겘ؐؒر؍ؚٝպծչعشًِ1IPUPպ瘝׮㾜Ꟛ  ⿑鼅׃׋㣐䩛ꋒ遤ծ嶊顤罏ꆃ輐ך؟٦ؽأ׾䗡䎿嫰鯰  չشؽشؽٍؗحءؚٝպח⸇ִծչشؽشؽؙٖآحزؕ٦سպ չشؽشؽ⡝㸓ٗ٦ٝպչشؽشؽ'9պ׾㾜Ꟛ ٓ؎ؿأة؎ٕ؟ه٦ز✲噟 ⚺銲؟٦ؽأ
  3. h"UFBN *OD  شؽشؽ'9 ֻ׷ת%&-* ؿ؋؎ٝسفٗ ٓ؎ؿأة؎ٕ؟ه٦ز✲噟 倜鋉؟٦ؽأ -JGFٓ؎ؿسحز 4PMEJإر؍

    شؽشؽ⡝㸓ٗ٦ٝ ׅתְֲ׷ NJOPSJFىظٔؒ 荈⹛鮦ꟼ鸬 ꆃ輐ًر؍، ׉ך➭欰崞ꟼ鸬 ׉ך➭ٓ؎ؿؒٝر؍ؚٝ ꆃ輐ًر؍، ׉ך➭㹺鎘٥眍秈 ׉ך➭♶⹛欵 倜鋉؟٦ؽأ ׉ך➭قٕأ؛، չٕٓ٦ٝպ ِ٦ؠ٦ך㡮 ַ׵⡲׵׸׋ 㦄崞؟فٔ&$ ♶⹛欵㡰⽱䞔 㜠 嗚 稊 ؟ ؎ ز٥♶⹛欵♧ 䭍叨㹀؟؎ز ٓ ؎ ؿ ؒ ٝ ر؍ؚٝ䞔㜠 嗚稊؟؎ز ⚥〢荈⹛鮦ך ؎ٝة٦ طحز顋㡰 فָٗ侄ִ׷ ⡝ 㸓 ٗ ٦ ٝ 䞔㜠؟؎ز 鸐⥋顤٥㹺鎘 鋅湫׃؟؎ز ⴱ 䗰 罏 ך '9 ز ٖ ٦ س ؟ ه٦ز،فٔ ء ٗ ، ٔ ٥ عث꽀ꤐזו ⳿䓸鏝㉏؟٦ ؽأך嗚稊٥ ✮秈؟؎ز
  4. h"UFBN *OD  "#ذأزך㹋倵倯岀 ،فٔ ؛٦ءّٝ 3VCZPO3BJMT 4ZNGPOZ ىسٕؐؑ، /HJOY

    1VNB "QBDIF .Z42- 1PTUHSF42- 3FEJT .FNDBDIFE ؎ٝؿٓأ زؙٓثٍ طحزٙ٦ؙ إُؗٔذ؍ ⟎䟝⻉
  5. h"UFBN *OD  倜"#ذأزך㹋倵倯岀 ،فٔ ؛٦ءّٝ 3VCZPO3BJMT 4ZNGPOZ ىسٕؐؑ، /HJOY

    1VNB "QBDIF .Z42- 1PTUHSF42- 3FEJT .FNDBDIFE ؎ٝؿٓ أزؙٓثٍ طحزٙ٦ؙ إُؗٔذ؍ ⟎䟝⻉ ،فٔ؛٦ءّٝח⣛㶷׃זְ"#ذأز׾〳腉הׅ׷
  6. h"UFBN *OD  ⽃秪ח؝ٝذش׾㟓װׅה"#ذأزך鎘庠ח儗꟦ַַָ׷ 涫ꐮ歗꬗ 然钠歗꬗ 倜"#ذأزך㹋倵铬겗 " # B

    C 涫ꐮ歗꬗ ذأز 然钠歗꬗ ذأز ؔٔآشٕ B " "#ذأز׾遤ֲ؝ٝذشָ 㟓ִ׸ל㟓ִ׷קוծ،ؙإ أ׀הח؝ٝذش׾ⶴ׶䮶׷ ךדر٦ةָ꧊ת׷תד儗꟦ ַַָ׷
  7. h"UFBN *OD  "#ذأز63-⽃⡘חٕ٦ذ؍ֶؚٝ״ןUBSHFUHSPVQ׾鷄⸇׃ג㹋倵 倜"#ذأزך㹋倵铬겗 " # 涫ꐮ歗꬗ ذأز 然钠歗꬗

    ذأز ؔٔآشٕ " " ؔٔآشٕ C B B B 然钠歗꬗ 涫ꐮ歗꬗ "#ذأزךٕ٦ذ؍ؚٝⴓ UBSHFUHSPVQ׾⡲䧭  
  8. h"UFBN *OD  "#ذأز㹋倵⚥חؔٔآشٕך㢌刿ָ涪欰ׅ׷ 倜"#ذأزך㹋倵铬겗 " # 涫ꐮ歗꬗ ذأز 倜ؔٔآشٕ

    傊ؔٔآشٕ " " 傊ؔٔآشٕ NBTUFSـٓٝث NFSHFEFQMPZ ؔٔآشָٕ 〢ְתת   ؔٔآشٕ UBSHFUHSPVQ "#ذأز UBSHFUHSPVQ
  9. h"UFBN *OD  ؔٔآشٕך㢌刿׾ׅץגךؔٔآشٕח׮黝欽ׅ׷$*׾欽䠐 倜"#ذأزך㹋倵铬겗 " # 涫ꐮ歗꬗ ذأز 倜ؔٔآشٕ

    傊ؔٔآشٕ " " 傊ؔٔآشٕ EFQMPZ " 倜ؔٔآشٕ ؔٔآشٕ UBSHFUHSPVQ "#ذأز UBSHFUHSPVQ
  10. h"UFBN *OD  תה׭ p ؎ٝؿٓؒٝآص،ׄׯזֻג׮ֿֿתד〳腉濼陎ה׃גכ䗳銲 p ⢪ֲ➂➙㔐כ⚺חؿٗٝزך铬겗׾荈׵䭪ְ何㊣׃גְֻ p ׅץגך铬겗ָ鍑寸דֹ׷ꋒך䔉⚵דכזְ醱侧ل٦آ׾騖ּ"#

    ָծ➙״׶㼰׃ד׮⵸פ p 嚂׃ְַծؙؙׅٙٙ׷ַ㣐✲ p ➙㔐כֻ֮תד♧⢽ד֮׶ծת׌ת׌װ׶׋ְֿהָ֮׷ךדؒٝآ ص،٥رؠ؎ش٦瘝㉏׻׆ծ׈מؒ؎ث٦يח