Fluentd, mongoDB, Kibanaを利用したはてなブログABテストの事例

「Hatena Engineer Seminar #2」で発表した資料です

Transcript

1. 'MVFOUE
   NPOHP%#
   ,JCBOB Λར༻ͨ͠
   ͸ͯͳϒϩά"#ςετͷࣄྫ )BUFOB
   &OHJOFFS
   4FNJOBS
    JETIJCB@ZV
2. ຊ೔ͷΞδΣϯμ w͸ͯͳϒϩάͰͷ"#ςετࣄྫ঺հ w؆୯ʹ"#ςετΛߦ͏࢓૊Έ঺հ
3. ࣗݾ঺հ wJETIJCB@ZV !TIJCB@ZV wΞϓϦέʔγϣϯΤϯδχΞ ೥ೖࣾ
4. ࣗݾ঺հ w͸ͯͳϒϩάνʔϜ wཪଆΑΓͷ։ൃ͕ଟ͍ wσʔλूܭ wσϓϩΠपΓ wͦͷଞ৭ʑ
5. ͸ͯͳϒϩάͰͷ "#ςετࣄྫ
6. ͸ͯͳϒϩάͰ͸ wαʔϏεͷվળʹ"#ςετΛಋೖ wϥϯσΟϯάϖʔδͷվળ wฤूը໘ͷվળ
7. ͸ͯͳϒϩάͰ͸ wαʔϏεͷվળʹ"#ςετΛಋೖ wϥϯσΟϯάϖʔδͷվળ wฤूը໘ͷվળ
8. ϥϯσΟϯάϖʔδվળ wొ࿥ϘλϯͷΫϦοΫ਺Λ৳͹͍ͨ͠ wϘλϯͱϑΥʔϜͲͪΒ͕ΫϦοΫ͞ ΕΔ
10. ςετ݁Ռ   ΫϦοΫ཰্͕ঢ
11. "#ςετΛ͢Δͱ wਪଌͰ͸ͳ͘ɺ਺ࣈΛݕূ͠ͳ͕Βվ ળ͕ग़དྷΔ w։ൃऀͷצͰ࡞Βͳ͍ wࣦഊͯ͠΋ৼΓฦΓ͕ग़དྷΔ
12. ೔ʑ"#ςετΛ͢Δʹ͸ w͸ͯͳϒϩάͰ͸೔ʑ"#ςετΛͯ͠ ͍Δ w࣮ࢪ͕ͱʹ͔͘؆୯Ͱͳ͍ͱବ໨ w೉͍͠ͱ΍Βͳ͍
13. ؆୯ʹ"#ςετΛ ͢Δ࢓૊Έ
14. ϥϯσΟϯάϖʔδͷ "#ςετखॱ wදࣔΛग़͠෼͚Δ wϘλϯΛΫϦοΫͨ࣌͠ʹϩάΛه࿥ wू·ͬͨϩάΛूܭɾάϥϑԽ͢Δ
15. දࣔͷ੾ସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫ৘ใͷه࿥ ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ
16. wදࣔΛग़͠෼͚ wDIBOLPͷΑ͏ͳ࢓૊Έ wΫϦοΫϩάͷه࿥ w+4ͰͷΠϕϯτૹ৴ɾ'MVFOUE wूܭɾάϥϑԽ w.POHP%#ɺ,JCBOBͷར༻ Ͱ͖Δ͚ͩ؆୯ʹ
17. wදࣔΛग़͠෼͚ wDIBOLPͷΑ͏ͳ࢓૊Έ wΫϦοΫϩάͷه࿥ w+4ͰͷΠϕϯτૹ৴ɾ'MVFOUE wूܭɾάϥϑԽ wNPOHP%#ɺ,JCBOBͷར༻ Ͱ͖Δ͚ͩ؆୯ʹ
19. දࣔͷग़͠Θ͚
20. දࣔͷ੾ସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫ৘ใͷه࿥ ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ
21. දࣔͷग़͠Θ͚ wࠓճ͸͋·Γ࿩͠·ͤΜ w͜ͷ෦෼͸ग़དྷ͍ͯΔͱ͍͏લఏͰ࿩ ͠·͢
22. දࣔͷग़͠Θ͚ w*1ΞυϨεͳͲͰग़͠Θ͚ wSVCZͷDIBOLP wDPPLQBEࣾ੡ w͸ͯͳϒϩάͰ͸ಉ͡Α͏ͳ࢓૊ΈΛ ࡞͍ͬͯΔ
24. ΫϦοΫϩάͷه࿥
25. දࣔͷ੾ସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫ৘ใͷه࿥ ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ
26. ؆୯ͳྲྀΕ wΫϦοΫ͞ΕͨΒΠϕϯτ͕αʔόʹ ૹ৴ wαʔόͰ͸"#ςετ৘ใͱڞʹ+40/ Ͱه࿥ w+40/ͷϩά͸'MVFOUEͰதԝαʔόʹ ूΊΔ
27. ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰ৘ใ෇͚ͯ +40/ʹه࿥ ϩάΛ'MVFOUEͰ தԝʹ
28. ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰ৘ใ෇͚ͯ +40/ʹه࿥ ϩάΛ'MVFOUEͰ தԝʹ
29. +4ͰΠϕϯτૹ৴ wEBUBFWFOUOBNFଐੑΛॻ͍͓ͯͩ͘ ͚Ͱૹ৴ wσβΠφͳͲͰ΋؆୯ʹग़དྷΔΑ͏ʹ <button data-event-name="guide-register-button"> ͸ͯͳϒϩάΛ࢝ΊΔ(ແྉ) </button>
30. ࣮૷ wΫϦοΫΠϕϯτΛ؂ࢹ wEBUBFWFOUOBNF͕͋ΔཁૉͳΒ"1* ʹΞΫηε
31. ࣮૷ྫ $(document).on('mousedown', '[data-event-name]', function(e) { $.ajax({ url: "/api/event", type: "POST", data: { event_name: $(this).attr('data-event-name') } }); });
32. ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰ৘ใ෇͚ͯ +40/ʹه࿥ ϩάΛ'MVFOUEͰ தԝʹ
33. αʔόͰ+40/ϩά wΞΫηεʹ߹Θͤͯ+40/ΛϑΝΠϧग़ ྗ͢Δ͚ͩ wBQJFWFOU FWFOU@OBNF w"#ςετ৘ใ΋ڞʹه࿥ wͲͪΒͷύλʔϯ͔ wͲͷϘλϯ FWFOU ͔
34. FLUENT({ tag => "event", event_name => "guide-register-button", }) { "tag" : "event", "event_name" : "guide-register-button", "units" : { "ABUnit::GuideForm" : 1, "ABUnit::EditorColor" : 0, }, "time" : 1384844834 }
35. ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰ৘ใ෇͚ͯ +40/ʹه࿥ ϩάΛ'MVFOUEͰ தԝʹ
36. ϩάΛதԝʹूΊΔ wͲ͜Ͱه࿥͞Εͯ΋͏·͘ूΊΒΕͨ ͍ wूܭ΍ՄࢹԽͷͨΊʹσʔλΛม׵͠ ͍ͨ
37. ϩάΛதԝʹूΊΔ wͲ͜Ͱه࿥͞Εͯ΋͏·͘ूΊΒΕͨ ͍ wूܭ΍ՄࢹԽͷͨΊʹσʔλΛม׵͠ ͍ͨ
38. 'MVFOUE
39. 'MVFOUE wγεςϜͷ֤ॴ͔Βग़ྗ͞ΕΔϩάΛ ഑ૹɾू໿͢ΔͨΊͷϛυϧ΢ΣΞ wϓϥάΠϯʹΑͬͯूΊͨϩάΛଞͷ σʔλʹม׵΋ग़དྷΔ
40. ϩάΛूΊΔߏ੒ w֤αʔό͸ϩάϑΝΠϧʹɺϩάߦ ͷ+40/ͱͯ͠ग़ྗ w֤αʔόͷϑΝΠϧΛUBJMͯ͠ɺBHFOU ͕தԝʹૹΔ͚ͩ
42. ͜͜·Ͱͷ·ͱΊ wಠࣗσʔλଐੑΛ෇͚Δ͚ͩͰΫϦο Ϋϩά͕ग़དྷΔΑ͏ʹ wϩά͸ࣗಈͰ'MVFOUEͷதԝʹू໿͞Ε ΔΑ͏ʹ ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰ৘ใ෇͚ͯ +40/ʹه࿥ ϩάΛ'MVFOUEͰ தԝʹ
44. ूܭɾάϥϑԽ
45. දࣔͷ੾ସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫ৘ใͷه࿥ ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ
46. ू໿͞Εͨϩά w+40/͕'MVFOUEͰू·͍ͬͯΔ͚ͩ w͜ͷ··Ͱ͸Կ΋Θ͔Βͳ͍
47. ूܭɾՄࢹԽՄೳʹ w"͸˓ΫϦοΫɺ#͸˚ΫϦοΫͱ͍͏ ৘ใΛूܭ wද΍άϥϑʹΑΔՄࢹԽ
48. ख๏ wॊೈʹूܭ
    
    NPOHP%#Λར༻ w؆୯ʹάϥϑԽ
    
    ,JCBOBΛར༻
49. ϩάΛूܭՄೳʹ wNPOHP%#Λར༻ w'MVFOUEͷ+40/
    
    NPOHP%#ʹ w+BWB4DSJQUΛ࢖ͬͯूܭ
50. 'MVFOUE
    
    NPOHP%# w'MVFOUEͰू໿͞Εͨϩά͕NPOHP%# ʹೖΔΑ͏ʹ
51. 'MVFOUE
    
    NPOHP%# wqVFOUQMVHJONPOHP
    
    ਺ߦͷઃఆ w'MVFOUEͷUBH͝ͱͷDPMMFDUJPO͕ग़དྷ Δ
52. 'MVFOUEઃఆྫ <match **> type mongo database fluent collection debug host mongohost port 10000 capped capped_size 100m </match>
53. NPOHP%#Ͱूܭ w+BWB4DSJQUͰΫΤϦΛ࡞ΓɺूܭՄೳ
54. { "tag" : "event", "event_name" : "guide-register-button", "units" : { "ABUnit::GuideForm" : 1, }, } ϩάͷܗࣜ "ύλʔϯͷΫϦοΫ਺ूܭ db.event.find({ "event_name":"guide-register-button", "units.ABUnit::GuideForm":1 }).count();
55. NPOHP%#Ͱूܭ wΫΤϦΛ࡞Ε͹ूܭՄೳʹ wΫΤϦΛ޻෉͢Ε͹ॊೈʹूܭͰ͖Δ
56. άϥϑԽՄೳʹ wॊೈʹूܭग़དྷΔ͚ͩͰ໨త͸ୡ੒ wͨͩ͠ຖճΫΤϦΛ࡞Δͷ͸େม wͬ͟ͱ"#ςετঢ়گΛάϥϑԽ͍ͨ͠
57. άϥϑԽՄೳʹ w&MBTUJD4FBSDI
    
    ,JCBOBΛར༻
58. ,JCBOB w&MBTUJDTFBSDIͷΠϯσοΫεΛར༻͠ ͯXFC্Ͱ༷ʑͳάϥϑΛ࡞Δπʔϧ w&MBTUJDTFBSDI͸4PMSͷΑ͏ͳશจݕࡧ γεςϜ
59. 'MVFOUE
    
    ,JCBOB w'MVFOUEͰू໿͞ΕͨϩάΛ &MBTUJDTFBSDIʹ wqVFOUQMVHJOFMBTUJDTFBSDI
    
    ਺ߦͷ ઃఆ w,JCBOB͸&MBTUJDTFBSDIΛݟΔΑ͏ʹ
60. 'MVFOUE
    
    ,JCBOB
61. 'MVFOUEઃఆྫ <match **> type elasticsearch host fluenthost port 9200 logstash_format true logstash_prefix service type_name service </store>
62. ,JCBOBͰάϥϑԽ w'MVFOUEͷσʔλ͸&MBTUJDTFBSDIʹೖ ΔΑ͏ʹͳͬͨ wXFC্Ͱ,JCBOBΛར༻ͯ͠άϥϑԽͰ ͖Δ
64. άϥϑԽσϞ wશσʔλΛදͰදࣔ͠ wΫϦοΫσʔλͷΈʹݕࡧͰߜΓࠐΈ w"#ςετঢ়گ͕࣠ͷ๮άϥϑΛ௥Ճ
65. άϥϑԽσϞ { "tag" : "event", "event_name" : "guide-register-button", "units" : { "ABUnit::GuideForm" : 1, }, } ΫϦοΫσʔλΛݕࡧ "#ঢ়ଶΛ࣠ʹ๮άϥϑ
67. ͜͜·Ͱͷ·ͱΊ w'MVFOUEͷϩά͸NPOHP%#
    &MBTUJDTFBSDIʹ wNPOHP%#ͰॊೈʹूܭՄೳʹ w,JCBOBͰ؆୯ʹάϥϑԽՄೳʹ
68. ࠷ऴతͳ࢓૊Έ
70. ؆୯ͳ"#ςετखॱ wϘλϯʹEBUBFWFOUOBNFͱ͍͏ଐੑ ௥Ճ wϦϦʔεޙ,JCBOBͰάϥϑԽ w࠷ऴతʹNPOHP%#Ͱूܭ͠ɺ݁ՌΛ ֬ೝ
71. ·ͱΊ w͸ͯͳϒϩάͰ͸"#ςετ΍σʔλղ ੳͳͲͷ࢓ࣄ΋΍͍ͬͯ·͢ w'MVFOUE
    NPOHP%#
    ,JCBOBΛར༻͠ ͨ"#ςετΛ؆୯ʹߦ͏࢓૊ΈΛ঺հ wࣗࣾͰ࢓૊ΈΛ࡞ͬͨͷͰɺ΋ͬͱෳ ࡶͳ"#ςετ΋ग़དྷ·͢
72. ͝ਗ਼ௌ͋Γ͕ͱ͏ ͍͟͝·ͨ͠