AB test infrastructure

52f95e52a6d71b67361ac179fb8405f4?s=47 chimame
November 04, 2018

AB test infrastructure

JAWS FESTA 2018 OSAKA

52f95e52a6d71b67361ac179fb8405f4?s=128

chimame

November 04, 2018
Tweet

Transcript

  1. "#UFTUJOGSBTUSVDUVSF +"84'&45"04","  吳䒭⠓爡ؒ؎ث٦ي 歊⿵ ⵃ㕼

  2. 荈䊹稱➜

  3. h"UFBN *OD  荈䊹稱➜ p/BNF 歊⿵ ⵃ㕼 ׋ת׋ ׶ה !SJUP

    p+PC TPGUXBSFEFWFMPQFS p4LJMMTFU 3VCZPO3BJMT 3FBDU (SBQI2- %PDLFS p0SHBOJ[FS 3BJMTGPMMPXVQ0TBLB p5XJUUFS !DIJNBNF@SU
  4. ؟٦ؽأ稱➜

  5. h"UFBN *OD  ٍؗحءؚٝ٥ؕ٦س ٗ٦ٝ筨さ嗚稊؟؎ز 穠㭴䒭㜥䞔㜠؟؎ز 䒷馉׃嫰鯰٥✮秈؟؎ز ׉ך➭ ـٓ؎ تٕꟼ鸬

    荈⹛鮦 ꟼ鸬 䒷馉׃ ꟼ鸬 ꆃ輐 ًر؍، 鮦叨㹀٥鮦顠《؟؎ز  㥍䚍ぢֽ欰椚傈✮䟝ֶ״ן⡤锃盖椚،فٔ  فؚٗٓوぢֽך䪮遭䞔㜠Ⱏ剣؟٦ؽأ  ♧䏝ךⰅ⸂ד剑㣐爡ך䒷馉׃噟罏ַ׵鋅琎׾《䖤  䲿䵿䒷馉׃噟罏侧כⰋ㕂爡⟃♳  噟歲ءؑ،زحفծ䒷馉׃ワ鴟؟٦ؽأ׮㢳侧㾜Ꟛ  ♧䏝ךⰅ⸂ד剑㣐爡ך顠《׶噟罏ַ׵鋅琎׾《䖤  ⿑姻ז㛇彊׾弫׋׃׋♧崧噟罏ה䲿䵿  噟歲ءؑ،زحف  Ⰻ㕂⟃♳ך穠㭴䒭㜥ַ׵⠓㜥׾׀稱➜  فٗך،سغ؎ؠ٦ָ䌢꽎ׅ׷㹋䏄莧׾Ⰻ㕂䏄莧㾜Ꟛ  չعشًِ㹀겘ؐؒر؍ؚٝպծչعشًِ1IPUPպ瘝׮㾜Ꟛ  ⿑鼅׃׋㣐䩛ꋒ遤ծ嶊顤罏ꆃ輐ך؟٦ؽأ׾䗡䎿嫰鯰  չشؽشؽٍؗحءؚٝպח⸇ִծչشؽشؽؙٖآحزؕ٦سպ չشؽشؽ⡝㸓ٗ٦ٝպչشؽشؽ'9պ׾㾜Ꟛ ٓ؎ؿأة؎ٕ؟ه٦ز✲噟 ⚺銲؟٦ؽأ
  6. h"UFBN *OD  شؽشؽ'9 ֻ׷ת%&-* ؿ؋؎ٝسفٗ ٓ؎ؿأة؎ٕ؟ه٦ز✲噟 倜鋉؟٦ؽأ -JGFٓ؎ؿسحز 4PMEJإر؍

    شؽشؽ⡝㸓ٗ٦ٝ ׅתְֲ׷ NJOPSJFىظٔؒ 荈⹛鮦ꟼ鸬 ꆃ輐ًر؍، ׉ך➭欰崞ꟼ鸬 ׉ך➭ٓ؎ؿؒٝر؍ؚٝ ꆃ輐ًر؍، ׉ך➭㹺鎘٥眍秈 ׉ך➭♶⹛欵 倜鋉؟٦ؽأ ׉ך➭قٕأ؛، չٕٓ٦ٝպ ِ٦ؠ٦ך㡮 ַ׵⡲׵׸׋ 㦄崞؟فٔ&$ ♶⹛欵㡰⽱䞔 㜠 嗚 稊 ؟ ؎ ز٥♶⹛欵♧ 䭍叨㹀؟؎ز ٓ ؎ ؿ ؒ ٝ ر؍ؚٝ䞔㜠 嗚稊؟؎ز ⚥〢荈⹛鮦ך ؎ٝة٦ طحز顋㡰 فָٗ侄ִ׷ ⡝ 㸓 ٗ ٦ ٝ 䞔㜠؟؎ز 鸐⥋顤٥㹺鎘 鋅湫׃؟؎ز ⴱ 䗰 罏 ך '9 ز ٖ ٦ س ؟ ه٦ز،فٔ ء ٗ ، ٔ ٥ عث꽀ꤐזו ⳿䓸鏝㉏؟٦ ؽأך嗚稊٥ ✮秈؟؎ز
  7. ؟٦ؽأ何㊣铬겗

  8. h"UFBN *OD  ؟٦ؽأ何㊣ Ⰵ⸂׃חְֻؿؓ٦ي׾鏣縧ׅ׷הِ٦ؠךꨄ膴ָ涪欰

  9. h"UFBN *OD  ؟٦ؽأ何㊣ ⵃ欽罏ָ実׭׷䞔㜠װ鋅װְׅ٥Ⰵ⸂׃װְׅ؟٦ؽأה׃ג何㊣ׅ׷䗳 銲ָ֮׷

  10. h"UFBN *OD  "#ذأزخ٦ٕ "#ذأز׮♧鿇خ٦ٕ׾嗚鏾٥㼪Ⰵ׃ג㹋倵

  11. h"UFBN *OD  "#ذأزך㹋倵倯岀 知⽃ז"#ذأز㹋倵חכ⵸鶢ךخ٦ٕ׾⢪欽ׅ׷ָ%0.圓䧭ָ㣐ֹֻ㢌 ׻׷״ֲז"#ذأزכ爡ⰻדꟚ涪ׁ׸׋杝荈فؚٗٓيSVCZHFNT׾ ⢪欽׃ג㹋倵կ 鑩䔲ل٦آך"#ذأز׾㹋倵ׅ׷׋׭ח鑩䔲ل٦آךا٦أ׾"#㹋倵ر ؍ؙٖزٔח؝ؾ٦׃ג㹋倵կ 

    A B
  12. h"UFBN *OD  "#ذأزך㹋倵倯岀 ،فٔ ؛٦ءّٝ 3VCZPO3BJMT 4ZNGPOZ ىسٕؐؑ، /HJOY

    1VNB "QBDIF .Z42- 1PTUHSF42- 3FEJT .FNDBDIFE ؎ٝؿٓأ زؙٓثٍ طحزٙ٦ؙ إُؗٔذ؍ ⟎䟝⻉
  13. h"UFBN *OD  "#ذأزך㹋倵ח״׷铬겗 p ♧㹀ךٕ٦ٕדر؍ؙٖزٔ圓鸡׾⡲䧭׃זְהְֽזְָ➬圫ך椚 鍑ָꨇ׃ְ p ؝ؾلׁ׸׋؝٦سָ㣐ꆀח涪欰׃ծ⟎חⰋ⡤ח⥜姻ָ䗳銲הז׏׋ 㜥さח⥜姻ꆀָ花㣐חז׷

    p "#ذأز׾遤׏גְ׷ل٦آדず♧ךػ٦ءٍٕ׾⿫撑׃גְ׷㜥さ ח♶Ⱗさ׾饯ֿ׃װְׅ
  14. 鍑寸倯岀 GPS&$4

  15. h"UFBN *OD  倜"#ذأزך㹋倵倯岀 ،فٔ ؛٦ءّٝ 3VCZPO3BJMT 4ZNGPOZ ىسٕؐؑ، /HJOY

    1VNB "QBDIF .Z42- 1PTUHSF42- 3FEJT .FNDBDIFE ؎ٝؿٓ أزؙٓثٍ طحزٙ٦ؙ إُؗٔذ؍ ⟎䟝⻉ ،فٔ؛٦ءّٝח⣛㶷׃זְ"#ذأز׾〳腉הׅ׷
  16. h"UFBN *OD  倜"#ذأزך㹋倵倯岀 ؝ٝذش׾⢪׏ג،فٔ؛٦ءّٝ׀ה"#ذأز׾㹋倵 " #

  17. h"UFBN *OD  倜"#ذأزך㹋倵倯岀 ـٓٝث׀הח؝ٝذش׾⡲䧭׃ծرفٗ؎׾遤ֲ .BTUFSـٓٝث NFSHFEFQMPZ "#欽ـٓٝث NFSHFEFQMPZ

  18. h"UFBN *OD  倜"#ذأزך㹋倵铬겗 ،ؙإأ鿪䏝חل٦آָ㢌׻׏ג׃תֲ " #

  19. h"UFBN *OD  倜"#ذأزך㹋倵铬겗 䌢חずׄ؝ٝذشַ׵ؙٔؒأز׾鵤׃ג׮׵ֲ # ず♧؝ٝذشפךؙٔؒأز׾笝䭯 ׅ׷׋׭חכ"-#ךUBSHFUHSPVQך 4UJDLZ4FTTJPO׾剣⸬חׅ׷

  20. h"UFBN *OD  ⽃秪ח؝ٝذش׾㟓װׅה"#ذأزך鎘庠ח儗꟦ַַָ׷ 涫ꐮ歗꬗ 然钠歗꬗ 倜"#ذأزך㹋倵铬겗 " # B

    C 涫ꐮ歗꬗ ذأز 然钠歗꬗ ذأز ؔٔآشٕ B " "#ذأز׾遤ֲ؝ٝذشָ 㟓ִ׸ל㟓ִ׷קוծ،ؙإ أ׀הח؝ٝذش׾ⶴ׶䮶׷ ךדر٦ةָ꧊ת׷תד儗꟦ ַַָ׷
  21. h"UFBN *OD  "#ذأز63-⽃⡘חٕ٦ذ؍ֶؚٝ״ןUBSHFUHSPVQ׾鷄⸇׃ג㹋倵 倜"#ذأزך㹋倵铬겗 " # 涫ꐮ歗꬗ ذأز 然钠歗꬗

    ذأز ؔٔآشٕ " " ؔٔآشٕ C B B B 然钠歗꬗ 涫ꐮ歗꬗ "#ذأزךٕ٦ذ؍ؚٝⴓ UBSHFUHSPVQ׾⡲䧭  
  22. h"UFBN *OD  "#ذأز㹋倵⚥חؔٔآشٕך㢌刿ָ涪欰ׅ׷ 倜"#ذأزך㹋倵铬겗 " # 涫ꐮ歗꬗ ذأز 倜ؔٔآشٕ

    傊ؔٔآشٕ " " 傊ؔٔآشٕ NBTUFSـٓٝث NFSHFEFQMPZ ؔٔآشָٕ 〢ְתת   ؔٔآشٕ UBSHFUHSPVQ "#ذأز UBSHFUHSPVQ
  23. h"UFBN *OD  ؔٔآشٕך㢌刿׾ׅץגךؔٔآشٕח׮黝欽ׅ׷$*׾欽䠐 倜"#ذأزך㹋倵铬겗 " # 涫ꐮ歗꬗ ذأز 倜ؔٔآشٕ

    傊ؔٔآشٕ " " 傊ؔٔآشٕ EFQMPZ " 倜ؔٔآشٕ ؔٔآشٕ UBSHFUHSPVQ "#ذأز UBSHFUHSPVQ
  24. תה׭

  25. h"UFBN *OD  תה׭ p ؎ٝؿٓؒٝآص،ׄׯזֻג׮ֿֿתד〳腉濼陎ה׃גכ䗳銲 p ⢪ֲ➂➙㔐כ⚺חؿٗٝزך铬겗׾荈׵䭪ְ何㊣׃גְֻ p ׅץגך铬겗ָ鍑寸דֹ׷ꋒך䔉⚵דכזְ醱侧ل٦آ׾騖ּ"#

    ָծ➙״׶㼰׃ד׮⵸פ p 嚂׃ְַծؙؙׅٙٙ׷ַ㣐✲ p ➙㔐כֻ֮תד♧⢽ד֮׶ծת׌ת׌װ׶׋ְֿהָ֮׷ךדؒٝآ ص،٥رؠ؎ش٦瘝㉏׻׆ծ׈מؒ؎ث٦يח
  26. h"UFBN *OD  ؎كٝز稱➜

  27. 穗㌀椚䙀 չ׫׿זד䎊ׇחז׸׷⠓爡חׅ׷ֿהպ չ➙ַ׵䎃竲ֻ⠓爡חׅ׷ֿהպ