Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Fluentd, mongoDB, Kibanaを利用したはてなブログABテストの事例
shibayu36
November 25, 2013
Programming
30
10k
Fluentd, mongoDB, Kibanaを利用したはてなブログABテストの事例
「Hatena Engineer Seminar #2」で発表した資料です
shibayu36
November 25, 2013
Tweet
Share
More Decks by shibayu36
See All by shibayu36
shibayu36
26
8.9k
shibayu36
7
5k
shibayu36
6
16k
shibayu36
8
29k
shibayu36
17
25k
shibayu36
1
5.2k
shibayu36
144
73k
shibayu36
2
1.7k
shibayu36
2
930
Other Decks in Programming
See All in Programming
pco2699
0
110
samjulien
0
470
bkuhlmann
6
710
wafuwafu13
1
180
hr01
0
2k
o0h
PRO
3
1.6k
doyaaaaaken
1
340
seike460
8
2.3k
fancyweb
1
100
fr0gger
2
2.9k
tkmnzm
1
270
bamboooo
1
140
Featured
See All Featured
caitiem20
311
17k
robhawkes
53
2.9k
eileencodes
114
25k
keithpitt
402
20k
zenorocha
298
40k
philhawksworth
193
8.9k
tammielis
237
23k
ddemaree
273
31k
revolveconf
201
9.7k
qrush
285
19k
holman
288
130k
reverentgeek
167
7.4k
Transcript
'MVFOUE NPOHP%# ,JCBOB Λར༻ͨ͠ ͯͳϒϩά"#ςετͷࣄྫ )BUFOB&OHJOFFS4FNJOBS JETIJCB@ZV
ຊͷΞδΣϯμ wͯͳϒϩάͰͷ"#ςετࣄྫհ w؆୯ʹ"#ςετΛߦ͏Έհ
ࣗݾհ wJETIJCB@ZV !TIJCB@ZV wΞϓϦέʔγϣϯΤϯδχΞ ೖࣾ
ࣗݾհ wͯͳϒϩάνʔϜ wཪଆΑΓͷ։ൃ͕ଟ͍ wσʔλूܭ wσϓϩΠपΓ wͦͷଞ৭ʑ
ͯͳϒϩάͰͷ "#ςετࣄྫ
ͯͳϒϩάͰ wαʔϏεͷվળʹ"#ςετΛಋೖ wϥϯσΟϯάϖʔδͷվળ wฤूը໘ͷվળ
ͯͳϒϩάͰ wαʔϏεͷվળʹ"#ςετΛಋೖ wϥϯσΟϯάϖʔδͷվળ wฤूը໘ͷվળ
ϥϯσΟϯάϖʔδվળ wొϘλϯͷΫϦοΫΛ৳͍ͨ͠ wϘλϯͱϑΥʔϜͲͪΒ͕ΫϦοΫ͞ ΕΔ
None
ςετ݁Ռ ΫϦοΫ্͕ঢ
"#ςετΛ͢Δͱ wਪଌͰͳ͘ɺࣈΛݕূ͠ͳ͕Βվ ળ͕ग़དྷΔ w։ൃऀͷצͰ࡞Βͳ͍ wࣦഊͯ͠ৼΓฦΓ͕ग़དྷΔ
ʑ"#ςετΛ͢Δʹ wͯͳϒϩάͰʑ"#ςετΛͯ͠ ͍Δ w࣮ࢪ͕ͱʹ͔͘؆୯Ͱͳ͍ͱବ w͍͠ͱΒͳ͍
؆୯ʹ"#ςετΛ ͢ΔΈ
ϥϯσΟϯάϖʔδͷ "#ςετखॱ wදࣔΛग़͚͠Δ wϘλϯΛΫϦοΫͨ࣌͠ʹϩάΛه wू·ͬͨϩάΛूܭɾάϥϑԽ͢Δ
දࣔͷସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫใͷه ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ
wදࣔΛग़͚͠ wDIBOLPͷΑ͏ͳΈ wΫϦοΫϩάͷه w+4ͰͷΠϕϯτૹ৴ɾ'MVFOUE wूܭɾάϥϑԽ w.POHP%#ɺ,JCBOBͷར༻ Ͱ͖Δ͚ͩ؆୯ʹ
wදࣔΛग़͚͠ wDIBOLPͷΑ͏ͳΈ wΫϦοΫϩάͷه w+4ͰͷΠϕϯτૹ৴ɾ'MVFOUE wूܭɾάϥϑԽ wNPOHP%#ɺ,JCBOBͷར༻ Ͱ͖Δ͚ͩ؆୯ʹ
None
දࣔͷग़͠Θ͚
දࣔͷସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫใͷه ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ
දࣔͷग़͠Θ͚ wࠓճ͋·Γ͠·ͤΜ w͜ͷ෦ग़དྷ͍ͯΔͱ͍͏લఏͰ ͠·͢
දࣔͷग़͠Θ͚ w*1ΞυϨεͳͲͰग़͠Θ͚ wSVCZͷDIBOLP wDPPLQBEࣾ wͯͳϒϩάͰಉ͡Α͏ͳΈΛ ࡞͍ͬͯΔ
None
ΫϦοΫϩάͷه
දࣔͷସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫใͷه ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ
؆୯ͳྲྀΕ wΫϦοΫ͞ΕͨΒΠϕϯτ͕αʔόʹ ૹ৴ wαʔόͰ"#ςετใͱڞʹ+40/ Ͱه w+40/ͷϩά'MVFOUEͰதԝαʔόʹ ूΊΔ
ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰใ͚ͯ +40/ʹه ϩάΛ'MVFOUEͰ தԝʹ
ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰใ͚ͯ +40/ʹه ϩάΛ'MVFOUEͰ தԝʹ
+4ͰΠϕϯτૹ৴ wEBUBFWFOUOBNFଐੑΛॻ͍͓ͯͩ͘ ͚Ͱૹ৴ wσβΠφͳͲͰ؆୯ʹग़དྷΔΑ͏ʹ <button data-event-name="guide-register-button"> ͯͳϒϩάΛ࢝ΊΔ(ແྉ) </button>
࣮ wΫϦοΫΠϕϯτΛࢹ wEBUBFWFOUOBNF͕͋ΔཁૉͳΒ"1* ʹΞΫηε
࣮ྫ $(document).on('mousedown', '[data-event-name]', function(e) { $.ajax({ url: "/api/event", type: "POST",
data: { event_name: $(this).attr('data-event-name') } }); });
ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰใ͚ͯ +40/ʹه ϩάΛ'MVFOUEͰ தԝʹ
αʔόͰ+40/ϩά wΞΫηεʹ߹Θͤͯ+40/ΛϑΝΠϧग़ ྗ͢Δ͚ͩ wBQJFWFOU FWFOU@OBNF w"#ςετใڞʹه wͲͪΒͷύλʔϯ͔ wͲͷϘλϯ FWFOU ͔
FLUENT({ tag => "event", event_name => "guide-register-button", }) { "tag"
: "event", "event_name" : "guide-register-button", "units" : { "ABUnit::GuideForm" : 1, "ABUnit::EditorColor" : 0, }, "time" : 1384844834 }
ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰใ͚ͯ +40/ʹه ϩάΛ'MVFOUEͰ தԝʹ
ϩάΛதԝʹूΊΔ wͲ͜Ͱه͞Εͯ͏·͘ूΊΒΕͨ ͍ wूܭՄࢹԽͷͨΊʹσʔλΛม͠ ͍ͨ
ϩάΛதԝʹूΊΔ wͲ͜Ͱه͞Εͯ͏·͘ूΊΒΕͨ ͍ wूܭՄࢹԽͷͨΊʹσʔλΛม͠ ͍ͨ
'MVFOUE
'MVFOUE wγεςϜͷ֤ॴ͔Βग़ྗ͞ΕΔϩάΛ ૹɾू͢ΔͨΊͷϛυϧΣΞ wϓϥάΠϯʹΑͬͯूΊͨϩάΛଞͷ σʔλʹมग़དྷΔ
ϩάΛूΊΔߏ w֤αʔόϩάϑΝΠϧʹɺϩάߦ ͷ+40/ͱͯ͠ग़ྗ w֤αʔόͷϑΝΠϧΛUBJMͯ͠ɺBHFOU ͕தԝʹૹΔ͚ͩ
None
͜͜·Ͱͷ·ͱΊ wಠࣗσʔλଐੑΛ͚Δ͚ͩͰΫϦο Ϋϩά͕ग़དྷΔΑ͏ʹ wϩάࣗಈͰ'MVFOUEͷதԝʹू͞Ε ΔΑ͏ʹ ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰใ͚ͯ +40/ʹه
ϩάΛ'MVFOUEͰ தԝʹ
None
ूܭɾάϥϑԽ
දࣔͷସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫใͷه ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ
ू͞Εͨϩά w+40/͕'MVFOUEͰू·͍ͬͯΔ͚ͩ w͜ͷ··ͰԿΘ͔Βͳ͍
ूܭɾՄࢹԽՄೳʹ w"˓ΫϦοΫɺ#˚ΫϦοΫͱ͍͏ ใΛूܭ wදάϥϑʹΑΔՄࢹԽ
ख๏ wॊೈʹूܭNPOHP%#Λར༻ w؆୯ʹάϥϑԽ,JCBOBΛར༻
ϩάΛूܭՄೳʹ wNPOHP%#Λར༻ w'MVFOUEͷ+40/NPOHP%#ʹ w+BWB4DSJQUΛͬͯूܭ
'MVFOUENPOHP%# w'MVFOUEͰू͞Εͨϩά͕NPOHP%# ʹೖΔΑ͏ʹ
'MVFOUENPOHP%# wqVFOUQMVHJONPOHP ߦͷઃఆ w'MVFOUEͷUBH͝ͱͷDPMMFDUJPO͕ग़དྷ Δ
'MVFOUEઃఆྫ <match **> type mongo database fluent collection debug host
mongohost port 10000 capped capped_size 100m </match>
NPOHP%#Ͱूܭ w+BWB4DSJQUͰΫΤϦΛ࡞ΓɺूܭՄೳ
{ "tag" : "event", "event_name" : "guide-register-button", "units" : {
"ABUnit::GuideForm" : 1, }, } ϩάͷܗࣜ "ύλʔϯͷΫϦοΫूܭ db.event.find({ "event_name":"guide-register-button", "units.ABUnit::GuideForm":1 }).count();
NPOHP%#Ͱूܭ wΫΤϦΛ࡞ΕूܭՄೳʹ wΫΤϦΛ͢ΕॊೈʹूܭͰ͖Δ
άϥϑԽՄೳʹ wॊೈʹूܭग़དྷΔ͚ͩͰతୡ wͨͩ͠ຖճΫΤϦΛ࡞Δͷେม wͬ͟ͱ"#ςετঢ়گΛάϥϑԽ͍ͨ͠
άϥϑԽՄೳʹ w&MBTUJD4FBSDI ,JCBOBΛར༻
,JCBOB w&MBTUJDTFBSDIͷΠϯσοΫεΛར༻͠ ͯXFC্Ͱ༷ʑͳάϥϑΛ࡞Δπʔϧ w&MBTUJDTFBSDI4PMSͷΑ͏ͳશจݕࡧ γεςϜ
'MVFOUE,JCBOB w'MVFOUEͰू͞ΕͨϩάΛ &MBTUJDTFBSDIʹ wqVFOUQMVHJOFMBTUJDTFBSDI ߦͷ ઃఆ w,JCBOB&MBTUJDTFBSDIΛݟΔΑ͏ʹ
'MVFOUE,JCBOB
'MVFOUEઃఆྫ <match **> type elasticsearch host fluenthost port 9200 logstash_format
true logstash_prefix service type_name service </store>
,JCBOBͰάϥϑԽ w'MVFOUEͷσʔλ&MBTUJDTFBSDIʹೖ ΔΑ͏ʹͳͬͨ wXFC্Ͱ,JCBOBΛར༻ͯ͠άϥϑԽͰ ͖Δ
None
άϥϑԽσϞ wશσʔλΛදͰදࣔ͠ wΫϦοΫσʔλͷΈʹݕࡧͰߜΓࠐΈ w"#ςετঢ়گ͕࣠ͷάϥϑΛՃ
άϥϑԽσϞ { "tag" : "event", "event_name" : "guide-register-button", "units" :
{ "ABUnit::GuideForm" : 1, }, } ΫϦοΫσʔλΛݕࡧ "#ঢ়ଶΛ࣠ʹάϥϑ
None
͜͜·Ͱͷ·ͱΊ w'MVFOUEͷϩάNPOHP%# &MBTUJDTFBSDIʹ wNPOHP%#ͰॊೈʹूܭՄೳʹ w,JCBOBͰ؆୯ʹάϥϑԽՄೳʹ
࠷ऴతͳΈ
None
؆୯ͳ"#ςετखॱ wϘλϯʹEBUBFWFOUOBNFͱ͍͏ଐੑ Ճ wϦϦʔεޙ,JCBOBͰάϥϑԽ w࠷ऴతʹNPOHP%#Ͱूܭ͠ɺ݁ՌΛ ֬ೝ
·ͱΊ wͯͳϒϩάͰ"#ςετσʔλղ ੳͳͲͷࣄ͍ͬͯ·͢ w'MVFOUE NPOHP%# ,JCBOBΛར༻͠ ͨ"#ςετΛ؆୯ʹߦ͏ΈΛհ wࣗࣾͰΈΛ࡞ͬͨͷͰɺͬͱෳ ࡶͳ"#ςετग़དྷ·͢
͝ਗ਼ௌ͋Γ͕ͱ͏ ͍͟͝·ͨ͠