Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AB test infrastructure

Avatar for chimame chimame
November 04, 2018

AB test infrastructure

JAWS FESTA 2018 OSAKA

Avatar for chimame

chimame

November 04, 2018
Tweet

More Decks by chimame

Other Decks in Technology

Transcript

  1. h"UFBN *OD  荈䊹稱➜ p/BNF 歊⿵ ⵃ㕼 ׋ת׋ ׶ה !SJUP

    p+PC TPGUXBSFEFWFMPQFS p4LJMMTFU 3VCZPO3BJMT 3FBDU (SBQI2- %PDLFS p0SHBOJ[FS 3BJMTGPMMPXVQ0TBLB p5XJUUFS !DIJNBNF@SU
  2. h"UFBN *OD  ٍؗحءؚٝ٥ؕ٦س ٗ٦ٝ筨さ嗚稊؟؎ز 穠㭴䒭㜥䞔㜠؟؎ز 䒷馉׃嫰鯰٥✮秈؟؎ز ׉ך➭ ـٓ؎ تٕꟼ鸬

    荈⹛鮦 ꟼ鸬 䒷馉׃ ꟼ鸬 ꆃ輐 ًر؍، 鮦叨㹀٥鮦顠《؟؎ز  㥍䚍ぢֽ欰椚傈✮䟝ֶ״ן⡤锃盖椚،فٔ  فؚٗٓوぢֽך䪮遭䞔㜠Ⱏ剣؟٦ؽأ  ♧䏝ךⰅ⸂ד剑㣐爡ך䒷馉׃噟罏ַ׵鋅琎׾《䖤  䲿䵿䒷馉׃噟罏侧כⰋ㕂爡⟃♳  噟歲ءؑ،زحفծ䒷馉׃ワ鴟؟٦ؽأ׮㢳侧㾜Ꟛ  ♧䏝ךⰅ⸂ד剑㣐爡ך顠《׶噟罏ַ׵鋅琎׾《䖤  ⿑姻ז㛇彊׾弫׋׃׋♧崧噟罏ה䲿䵿  噟歲ءؑ،زحف  Ⰻ㕂⟃♳ך穠㭴䒭㜥ַ׵⠓㜥׾׀稱➜  فٗך،سغ؎ؠ٦ָ䌢꽎ׅ׷㹋䏄莧׾Ⰻ㕂䏄莧㾜Ꟛ  չعشًِ㹀겘ؐؒر؍ؚٝպծչعشًِ1IPUPպ瘝׮㾜Ꟛ  ⿑鼅׃׋㣐䩛ꋒ遤ծ嶊顤罏ꆃ輐ך؟٦ؽأ׾䗡䎿嫰鯰  չشؽشؽٍؗحءؚٝպח⸇ִծչشؽشؽؙٖآحزؕ٦سպ չشؽشؽ⡝㸓ٗ٦ٝպչشؽشؽ'9պ׾㾜Ꟛ ٓ؎ؿأة؎ٕ؟ه٦ز✲噟 ⚺銲؟٦ؽأ
  3. h"UFBN *OD  شؽشؽ'9 ֻ׷ת%&-* ؿ؋؎ٝسفٗ ٓ؎ؿأة؎ٕ؟ه٦ز✲噟 倜鋉؟٦ؽأ -JGFٓ؎ؿسحز 4PMEJإر؍

    شؽشؽ⡝㸓ٗ٦ٝ ׅתְֲ׷ NJOPSJFىظٔؒ 荈⹛鮦ꟼ鸬 ꆃ輐ًر؍، ׉ך➭欰崞ꟼ鸬 ׉ך➭ٓ؎ؿؒٝر؍ؚٝ ꆃ輐ًر؍، ׉ך➭㹺鎘٥眍秈 ׉ך➭♶⹛欵 倜鋉؟٦ؽأ ׉ך➭قٕأ؛، չٕٓ٦ٝպ ِ٦ؠ٦ך㡮 ַ׵⡲׵׸׋ 㦄崞؟فٔ&$ ♶⹛欵㡰⽱䞔 㜠 嗚 稊 ؟ ؎ ز٥♶⹛欵♧ 䭍叨㹀؟؎ز ٓ ؎ ؿ ؒ ٝ ر؍ؚٝ䞔㜠 嗚稊؟؎ز ⚥〢荈⹛鮦ך ؎ٝة٦ طحز顋㡰 فָٗ侄ִ׷ ⡝ 㸓 ٗ ٦ ٝ 䞔㜠؟؎ز 鸐⥋顤٥㹺鎘 鋅湫׃؟؎ز ⴱ 䗰 罏 ך '9 ز ٖ ٦ س ؟ ه٦ز،فٔ ء ٗ ، ٔ ٥ عث꽀ꤐזו ⳿䓸鏝㉏؟٦ ؽأך嗚稊٥ ✮秈؟؎ز
  4. h"UFBN *OD  "#ذأزך㹋倵倯岀 ،فٔ ؛٦ءّٝ 3VCZPO3BJMT 4ZNGPOZ ىسٕؐؑ، /HJOY

    1VNB "QBDIF .Z42- 1PTUHSF42- 3FEJT .FNDBDIFE ؎ٝؿٓأ زؙٓثٍ طحزٙ٦ؙ إُؗٔذ؍ ⟎䟝⻉
  5. h"UFBN *OD  倜"#ذأزך㹋倵倯岀 ،فٔ ؛٦ءّٝ 3VCZPO3BJMT 4ZNGPOZ ىسٕؐؑ، /HJOY

    1VNB "QBDIF .Z42- 1PTUHSF42- 3FEJT .FNDBDIFE ؎ٝؿٓ أزؙٓثٍ طحزٙ٦ؙ إُؗٔذ؍ ⟎䟝⻉ ،فٔ؛٦ءّٝח⣛㶷׃זְ"#ذأز׾〳腉הׅ׷
  6. h"UFBN *OD  ⽃秪ח؝ٝذش׾㟓װׅה"#ذأزך鎘庠ח儗꟦ַַָ׷ 涫ꐮ歗꬗ 然钠歗꬗ 倜"#ذأزך㹋倵铬겗 " # B

    C 涫ꐮ歗꬗ ذأز 然钠歗꬗ ذأز ؔٔآشٕ B " "#ذأز׾遤ֲ؝ٝذشָ 㟓ִ׸ל㟓ִ׷קוծ،ؙإ أ׀הח؝ٝذش׾ⶴ׶䮶׷ ךדر٦ةָ꧊ת׷תד儗꟦ ַַָ׷
  7. h"UFBN *OD  "#ذأز63-⽃⡘חٕ٦ذ؍ֶؚٝ״ןUBSHFUHSPVQ׾鷄⸇׃ג㹋倵 倜"#ذأزך㹋倵铬겗 " # 涫ꐮ歗꬗ ذأز 然钠歗꬗

    ذأز ؔٔآشٕ " " ؔٔآشٕ C B B B 然钠歗꬗ 涫ꐮ歗꬗ "#ذأزךٕ٦ذ؍ؚٝⴓ UBSHFUHSPVQ׾⡲䧭  
  8. h"UFBN *OD  "#ذأز㹋倵⚥חؔٔآشٕך㢌刿ָ涪欰ׅ׷ 倜"#ذأزך㹋倵铬겗 " # 涫ꐮ歗꬗ ذأز 倜ؔٔآشٕ

    傊ؔٔآشٕ " " 傊ؔٔآشٕ NBTUFSـٓٝث NFSHFEFQMPZ ؔٔآشָٕ 〢ְתת   ؔٔآشٕ UBSHFUHSPVQ "#ذأز UBSHFUHSPVQ
  9. h"UFBN *OD  ؔٔآشٕך㢌刿׾ׅץגךؔٔآشٕח׮黝欽ׅ׷$*׾欽䠐 倜"#ذأزך㹋倵铬겗 " # 涫ꐮ歗꬗ ذأز 倜ؔٔآشٕ

    傊ؔٔآشٕ " " 傊ؔٔآشٕ EFQMPZ " 倜ؔٔآشٕ ؔٔآشٕ UBSHFUHSPVQ "#ذأز UBSHFUHSPVQ
  10. h"UFBN *OD  תה׭ p ؎ٝؿٓؒٝآص،ׄׯזֻג׮ֿֿתד〳腉濼陎ה׃גכ䗳銲 p ⢪ֲ➂➙㔐כ⚺חؿٗٝزך铬겗׾荈׵䭪ְ何㊣׃גְֻ p ׅץגך铬겗ָ鍑寸דֹ׷ꋒך䔉⚵דכזְ醱侧ل٦آ׾騖ּ"#

    ָծ➙״׶㼰׃ד׮⵸פ p 嚂׃ְַծؙؙׅٙٙ׷ַ㣐✲ p ➙㔐כֻ֮תד♧⢽ד֮׶ծת׌ת׌װ׶׋ְֿהָ֮׷ךדؒٝآ ص،٥رؠ؎ش٦瘝㉏׻׆ծ׈מؒ؎ث٦يח