Fluentd, mongoDB, Kibanaを利用したはてなブログABテストの事例

0ac642c69b7f699a69e3ae3372244dc6?s=47 shibayu36
November 25, 2013

Fluentd, mongoDB, Kibanaを利用したはてなブログABテストの事例

「Hatena Engineer Seminar #2」で発表した資料です

0ac642c69b7f699a69e3ae3372244dc6?s=128

shibayu36

November 25, 2013
Tweet

Transcript

  1. 'MVFOUE NPOHP%# ,JCBOB Λར༻ͨ͠ ͸ͯͳϒϩά"#ςετͷࣄྫ )BUFOB&OHJOFFS4FNJOBS JETIJCB@ZV

  2. ຊ೔ͷΞδΣϯμ w͸ͯͳϒϩάͰͷ"#ςετࣄྫ঺հ w؆୯ʹ"#ςετΛߦ͏࢓૊Έ঺հ

  3. ࣗݾ঺հ wJETIJCB@ZV !TIJCB@ZV wΞϓϦέʔγϣϯΤϯδχΞ ೥ೖࣾ

  4. ࣗݾ঺հ w͸ͯͳϒϩάνʔϜ wཪଆΑΓͷ։ൃ͕ଟ͍ wσʔλूܭ wσϓϩΠपΓ wͦͷଞ৭ʑ

  5. ͸ͯͳϒϩάͰͷ "#ςετࣄྫ

  6. ͸ͯͳϒϩάͰ͸ wαʔϏεͷվળʹ"#ςετΛಋೖ wϥϯσΟϯάϖʔδͷվળ wฤूը໘ͷվળ

  7. ͸ͯͳϒϩάͰ͸ wαʔϏεͷվળʹ"#ςετΛಋೖ wϥϯσΟϯάϖʔδͷվળ wฤूը໘ͷվળ

  8. ϥϯσΟϯάϖʔδվળ wొ࿥ϘλϯͷΫϦοΫ਺Λ৳͹͍ͨ͠ wϘλϯͱϑΥʔϜͲͪΒ͕ΫϦοΫ͞ ΕΔ

  9. None
  10. ςετ݁Ռ   ΫϦοΫ཰্͕ঢ

  11. "#ςετΛ͢Δͱ wਪଌͰ͸ͳ͘ɺ਺ࣈΛݕূ͠ͳ͕Βվ ળ͕ग़དྷΔ w։ൃऀͷצͰ࡞Βͳ͍ wࣦഊͯ͠΋ৼΓฦΓ͕ग़དྷΔ

  12. ೔ʑ"#ςετΛ͢Δʹ͸ w͸ͯͳϒϩάͰ͸೔ʑ"#ςετΛͯ͠ ͍Δ w࣮ࢪ͕ͱʹ͔͘؆୯Ͱͳ͍ͱବ໨ w೉͍͠ͱ΍Βͳ͍

  13. ؆୯ʹ"#ςετΛ ͢Δ࢓૊Έ

  14. ϥϯσΟϯάϖʔδͷ "#ςετखॱ wදࣔΛग़͠෼͚Δ wϘλϯΛΫϦοΫͨ࣌͠ʹϩάΛه࿥ wू·ͬͨϩάΛूܭɾάϥϑԽ͢Δ

  15. දࣔͷ੾ସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫ৘ใͷه࿥ ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ

  16. wදࣔΛग़͠෼͚ wDIBOLPͷΑ͏ͳ࢓૊Έ wΫϦοΫϩάͷه࿥ w+4ͰͷΠϕϯτૹ৴ɾ'MVFOUE wूܭɾάϥϑԽ w.POHP%#ɺ,JCBOBͷར༻ Ͱ͖Δ͚ͩ؆୯ʹ

  17. wදࣔΛग़͠෼͚ wDIBOLPͷΑ͏ͳ࢓૊Έ wΫϦοΫϩάͷه࿥ w+4ͰͷΠϕϯτૹ৴ɾ'MVFOUE wूܭɾάϥϑԽ wNPOHP%#ɺ,JCBOBͷར༻ Ͱ͖Δ͚ͩ؆୯ʹ

  18. None
  19. දࣔͷग़͠Θ͚

  20. දࣔͷ੾ସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫ৘ใͷه࿥ ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ

  21. දࣔͷग़͠Θ͚ wࠓճ͸͋·Γ࿩͠·ͤΜ w͜ͷ෦෼͸ग़དྷ͍ͯΔͱ͍͏લఏͰ࿩ ͠·͢

  22. දࣔͷग़͠Θ͚ w*1ΞυϨεͳͲͰग़͠Θ͚ wSVCZͷDIBOLP wDPPLQBEࣾ੡ w͸ͯͳϒϩάͰ͸ಉ͡Α͏ͳ࢓૊ΈΛ ࡞͍ͬͯΔ

  23. None
  24. ΫϦοΫϩάͷه࿥

  25. දࣔͷ੾ସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫ৘ใͷه࿥ ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ

  26. ؆୯ͳྲྀΕ wΫϦοΫ͞ΕͨΒΠϕϯτ͕αʔόʹ ૹ৴ wαʔόͰ͸"#ςετ৘ใͱڞʹ+40/ Ͱه࿥ w+40/ͷϩά͸'MVFOUEͰதԝαʔόʹ ूΊΔ

  27. ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰ৘ใ෇͚ͯ +40/ʹه࿥ ϩάΛ'MVFOUEͰ தԝʹ

  28. ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰ৘ใ෇͚ͯ +40/ʹه࿥ ϩάΛ'MVFOUEͰ தԝʹ

  29. +4ͰΠϕϯτૹ৴ wEBUBFWFOUOBNFଐੑΛॻ͍͓ͯͩ͘ ͚Ͱૹ৴ wσβΠφͳͲͰ΋؆୯ʹग़དྷΔΑ͏ʹ <button data-event-name="guide-register-button"> ͸ͯͳϒϩάΛ࢝ΊΔ(ແྉ) </button>

  30. ࣮૷ wΫϦοΫΠϕϯτΛ؂ࢹ wEBUBFWFOUOBNF͕͋ΔཁૉͳΒ"1* ʹΞΫηε

  31. ࣮૷ྫ $(document).on('mousedown', '[data-event-name]', function(e) { $.ajax({ url: "/api/event", type: "POST",

    data: { event_name: $(this).attr('data-event-name') } }); });
  32. ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰ৘ใ෇͚ͯ +40/ʹه࿥ ϩάΛ'MVFOUEͰ தԝʹ

  33. αʔόͰ+40/ϩά wΞΫηεʹ߹Θͤͯ+40/ΛϑΝΠϧग़ ྗ͢Δ͚ͩ wBQJFWFOU FWFOU@OBNF w"#ςετ৘ใ΋ڞʹه࿥ wͲͪΒͷύλʔϯ͔ wͲͷϘλϯ FWFOU ͔

  34. FLUENT({ tag => "event", event_name => "guide-register-button", }) { "tag"

    : "event", "event_name" : "guide-register-button", "units" : { "ABUnit::GuideForm" : 1, "ABUnit::EditorColor" : 0, }, "time" : 1384844834 }
  35. ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰ৘ใ෇͚ͯ +40/ʹه࿥ ϩάΛ'MVFOUEͰ தԝʹ

  36. ϩάΛதԝʹूΊΔ wͲ͜Ͱه࿥͞Εͯ΋͏·͘ूΊΒΕͨ ͍ wूܭ΍ՄࢹԽͷͨΊʹσʔλΛม׵͠ ͍ͨ

  37. ϩάΛதԝʹूΊΔ wͲ͜Ͱه࿥͞Εͯ΋͏·͘ूΊΒΕͨ ͍ wूܭ΍ՄࢹԽͷͨΊʹσʔλΛม׵͠ ͍ͨ

  38. 'MVFOUE

  39. 'MVFOUE wγεςϜͷ֤ॴ͔Βग़ྗ͞ΕΔϩάΛ ഑ૹɾू໿͢ΔͨΊͷϛυϧ΢ΣΞ wϓϥάΠϯʹΑͬͯूΊͨϩάΛଞͷ σʔλʹม׵΋ग़དྷΔ

  40. ϩάΛूΊΔߏ੒ w֤αʔό͸ϩάϑΝΠϧʹɺϩάߦ ͷ+40/ͱͯ͠ग़ྗ w֤αʔόͷϑΝΠϧΛUBJMͯ͠ɺBHFOU ͕தԝʹૹΔ͚ͩ

  41. None
  42. ͜͜·Ͱͷ·ͱΊ wಠࣗσʔλଐੑΛ෇͚Δ͚ͩͰΫϦο Ϋϩά͕ग़དྷΔΑ͏ʹ wϩά͸ࣗಈͰ'MVFOUEͷதԝʹू໿͞Ε ΔΑ͏ʹ ΫϦοΫ +4ͰΠϕϯτ ૹ৴ αʔόͰ৘ใ෇͚ͯ +40/ʹه࿥

    ϩάΛ'MVFOUEͰ தԝʹ
  43. None
  44. ूܭɾάϥϑԽ

  45. දࣔͷ੾ସ ू·ͬͨϩάͷ ूܭɾάϥϑԽ ΫϦοΫ৘ใͷه࿥ ɾͲͪΒͷύλʔϯʁ ɾͲͷϘλϯʁ

  46. ू໿͞Εͨϩά w+40/͕'MVFOUEͰू·͍ͬͯΔ͚ͩ w͜ͷ··Ͱ͸Կ΋Θ͔Βͳ͍

  47. ूܭɾՄࢹԽՄೳʹ w"͸˓ΫϦοΫɺ#͸˚ΫϦοΫͱ͍͏ ৘ใΛूܭ wද΍άϥϑʹΑΔՄࢹԽ

  48. ख๏ wॊೈʹूܭNPOHP%#Λར༻ w؆୯ʹάϥϑԽ,JCBOBΛར༻

  49. ϩάΛूܭՄೳʹ wNPOHP%#Λར༻ w'MVFOUEͷ+40/NPOHP%#ʹ w+BWB4DSJQUΛ࢖ͬͯूܭ

  50. 'MVFOUENPOHP%# w'MVFOUEͰू໿͞Εͨϩά͕NPOHP%# ʹೖΔΑ͏ʹ

  51. 'MVFOUENPOHP%# wqVFOUQMVHJONPOHP ਺ߦͷઃఆ w'MVFOUEͷUBH͝ͱͷDPMMFDUJPO͕ग़དྷ Δ

  52. 'MVFOUEઃఆྫ <match **> type mongo database fluent collection debug host

    mongohost port 10000 capped capped_size 100m </match>
  53. NPOHP%#Ͱूܭ w+BWB4DSJQUͰΫΤϦΛ࡞ΓɺूܭՄೳ

  54. { "tag" : "event", "event_name" : "guide-register-button", "units" : {

    "ABUnit::GuideForm" : 1, }, } ϩάͷܗࣜ "ύλʔϯͷΫϦοΫ਺ूܭ db.event.find({ "event_name":"guide-register-button", "units.ABUnit::GuideForm":1 }).count();
  55. NPOHP%#Ͱूܭ wΫΤϦΛ࡞Ε͹ूܭՄೳʹ wΫΤϦΛ޻෉͢Ε͹ॊೈʹूܭͰ͖Δ

  56. άϥϑԽՄೳʹ wॊೈʹूܭग़དྷΔ͚ͩͰ໨త͸ୡ੒ wͨͩ͠ຖճΫΤϦΛ࡞Δͷ͸େม wͬ͟ͱ"#ςετঢ়گΛάϥϑԽ͍ͨ͠

  57. άϥϑԽՄೳʹ w&MBTUJD4FBSDI ,JCBOBΛར༻

  58. ,JCBOB w&MBTUJDTFBSDIͷΠϯσοΫεΛར༻͠ ͯXFC্Ͱ༷ʑͳάϥϑΛ࡞Δπʔϧ w&MBTUJDTFBSDI͸4PMSͷΑ͏ͳશจݕࡧ γεςϜ

  59. 'MVFOUE,JCBOB w'MVFOUEͰू໿͞ΕͨϩάΛ &MBTUJDTFBSDIʹ wqVFOUQMVHJOFMBTUJDTFBSDI ਺ߦͷ ઃఆ w,JCBOB͸&MBTUJDTFBSDIΛݟΔΑ͏ʹ

  60. 'MVFOUE,JCBOB

  61. 'MVFOUEઃఆྫ <match **> type elasticsearch host fluenthost port 9200 logstash_format

    true logstash_prefix service type_name service </store>
  62. ,JCBOBͰάϥϑԽ w'MVFOUEͷσʔλ͸&MBTUJDTFBSDIʹೖ ΔΑ͏ʹͳͬͨ wXFC্Ͱ,JCBOBΛར༻ͯ͠άϥϑԽͰ ͖Δ

  63. None
  64. άϥϑԽσϞ wશσʔλΛදͰදࣔ͠ wΫϦοΫσʔλͷΈʹݕࡧͰߜΓࠐΈ w"#ςετঢ়گ͕࣠ͷ๮άϥϑΛ௥Ճ

  65. άϥϑԽσϞ { "tag" : "event", "event_name" : "guide-register-button", "units" :

    { "ABUnit::GuideForm" : 1, }, } ΫϦοΫσʔλΛݕࡧ "#ঢ়ଶΛ࣠ʹ๮άϥϑ
  66. None
  67. ͜͜·Ͱͷ·ͱΊ w'MVFOUEͷϩά͸NPOHP%#  &MBTUJDTFBSDIʹ wNPOHP%#ͰॊೈʹूܭՄೳʹ w,JCBOBͰ؆୯ʹάϥϑԽՄೳʹ

  68. ࠷ऴతͳ࢓૊Έ

  69. None
  70. ؆୯ͳ"#ςετखॱ wϘλϯʹEBUBFWFOUOBNFͱ͍͏ଐੑ ௥Ճ wϦϦʔεޙ,JCBOBͰάϥϑԽ w࠷ऴతʹNPOHP%#Ͱूܭ͠ɺ݁ՌΛ ֬ೝ

  71. ·ͱΊ w͸ͯͳϒϩάͰ͸"#ςετ΍σʔλղ ੳͳͲͷ࢓ࣄ΋΍͍ͬͯ·͢ w'MVFOUE NPOHP%# ,JCBOBΛར༻͠ ͨ"#ςετΛ؆୯ʹߦ͏࢓૊ΈΛ঺հ wࣗࣾͰ࢓૊ΈΛ࡞ͬͨͷͰɺ΋ͬͱෳ ࡶͳ"#ςετ΋ग़དྷ·͢

  72. ͝ਗ਼ௌ͋Γ͕ͱ͏ ͍͟͝·ͨ͠