Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fluentd, mongoDB, Kibanaを利用したはてなブログABテストの事例

shibayu36
November 25, 2013

Fluentd, mongoDB, Kibanaを利用したはてなブログABテストの事例

「Hatena Engineer Seminar #2」で発表した資料です

shibayu36

November 25, 2013
Tweet

More Decks by shibayu36

Other Decks in Programming

Transcript

  1. 'MVFOUE NPOHP%# ,JCBOB
    Λར༻ͨ͠
    ͸ͯͳϒϩά"#ςετͷࣄྫ
    )BUFOB&OHJOFFS4FNJOBS
    JETIJCB@ZV

    View Slide

  2. ຊ೔ͷΞδΣϯμ
    w͸ͯͳϒϩάͰͷ"#ςετࣄྫ঺հ
    w؆୯ʹ"#ςετΛߦ͏࢓૊Έ঺հ

    View Slide

  3. ࣗݾ঺հ
    wJETIJCB@ZV !TIJCB@ZV

    wΞϓϦέʔγϣϯΤϯδχΞ
    ೥ೖࣾ

    View Slide

  4. ࣗݾ঺հ
    w͸ͯͳϒϩάνʔϜ
    wཪଆΑΓͷ։ൃ͕ଟ͍
    wσʔλूܭ
    wσϓϩΠपΓ
    wͦͷଞ৭ʑ

    View Slide

  5. ͸ͯͳϒϩάͰͷ
    "#ςετࣄྫ

    View Slide

  6. ͸ͯͳϒϩάͰ͸
    wαʔϏεͷվળʹ"#ςετΛಋೖ
    wϥϯσΟϯάϖʔδͷվળ
    wฤूը໘ͷվળ

    View Slide

  7. ͸ͯͳϒϩάͰ͸
    wαʔϏεͷվળʹ"#ςετΛಋೖ
    wϥϯσΟϯάϖʔδͷվળ
    wฤूը໘ͷվળ

    View Slide

  8. ϥϯσΟϯάϖʔδվળ
    wొ࿥ϘλϯͷΫϦοΫ਺Λ৳͹͍ͨ͠
    wϘλϯͱϑΥʔϜͲͪΒ͕ΫϦοΫ͞
    ΕΔ

    View Slide

  9. View Slide

  10. ςετ݁Ռ

    ΫϦοΫ཰্͕ঢ

    View Slide

  11. "#ςετΛ͢Δͱ
    wਪଌͰ͸ͳ͘ɺ਺ࣈΛݕূ͠ͳ͕Βվ
    ળ͕ग़དྷΔ
    w։ൃऀͷצͰ࡞Βͳ͍
    wࣦഊͯ͠΋ৼΓฦΓ͕ग़དྷΔ

    View Slide

  12. ೔ʑ"#ςετΛ͢Δʹ͸
    w͸ͯͳϒϩάͰ͸೔ʑ"#ςετΛͯ͠
    ͍Δ
    w࣮ࢪ͕ͱʹ͔͘؆୯Ͱͳ͍ͱବ໨
    w೉͍͠ͱ΍Βͳ͍

    View Slide

  13. ؆୯ʹ"#ςετΛ
    ͢Δ࢓૊Έ

    View Slide

  14. ϥϯσΟϯάϖʔδͷ
    "#ςετखॱ
    wදࣔΛग़͠෼͚Δ
    wϘλϯΛΫϦοΫͨ࣌͠ʹϩάΛه࿥
    wू·ͬͨϩάΛूܭɾάϥϑԽ͢Δ

    View Slide

  15. දࣔͷ੾ସ
    ू·ͬͨϩάͷ
    ूܭɾάϥϑԽ
    ΫϦοΫ৘ใͷه࿥
    ɾͲͪΒͷύλʔϯʁ
    ɾͲͷϘλϯʁ

    View Slide

  16. wදࣔΛग़͠෼͚
    wDIBOLPͷΑ͏ͳ࢓૊Έ
    wΫϦοΫϩάͷه࿥
    w+4ͰͷΠϕϯτૹ৴ɾ'MVFOUE
    wूܭɾάϥϑԽ
    w.POHP%#ɺ,JCBOBͷར༻
    Ͱ͖Δ͚ͩ؆୯ʹ

    View Slide

  17. wදࣔΛग़͠෼͚
    wDIBOLPͷΑ͏ͳ࢓૊Έ
    wΫϦοΫϩάͷه࿥
    w+4ͰͷΠϕϯτૹ৴ɾ'MVFOUE
    wूܭɾάϥϑԽ
    wNPOHP%#ɺ,JCBOBͷར༻
    Ͱ͖Δ͚ͩ؆୯ʹ

    View Slide

  18. View Slide

  19. දࣔͷग़͠Θ͚

    View Slide

  20. දࣔͷ੾ସ
    ू·ͬͨϩάͷ
    ूܭɾάϥϑԽ
    ΫϦοΫ৘ใͷه࿥
    ɾͲͪΒͷύλʔϯʁ
    ɾͲͷϘλϯʁ

    View Slide

  21. දࣔͷग़͠Θ͚
    wࠓճ͸͋·Γ࿩͠·ͤΜ
    w͜ͷ෦෼͸ग़དྷ͍ͯΔͱ͍͏લఏͰ࿩
    ͠·͢

    View Slide

  22. දࣔͷग़͠Θ͚
    w*1ΞυϨεͳͲͰग़͠Θ͚
    wSVCZͷDIBOLP
    wDPPLQBEࣾ੡
    w͸ͯͳϒϩάͰ͸ಉ͡Α͏ͳ࢓૊ΈΛ
    ࡞͍ͬͯΔ

    View Slide

  23. View Slide

  24. ΫϦοΫϩάͷه࿥

    View Slide

  25. දࣔͷ੾ସ
    ू·ͬͨϩάͷ
    ूܭɾάϥϑԽ
    ΫϦοΫ৘ใͷه࿥
    ɾͲͪΒͷύλʔϯʁ
    ɾͲͷϘλϯʁ

    View Slide

  26. ؆୯ͳྲྀΕ
    wΫϦοΫ͞ΕͨΒΠϕϯτ͕αʔόʹ
    ૹ৴
    wαʔόͰ͸"#ςετ৘ใͱڞʹ+40/
    Ͱه࿥
    w+40/ͷϩά͸'MVFOUEͰதԝαʔόʹ
    ूΊΔ

    View Slide

  27. ΫϦοΫ
    +4ͰΠϕϯτ
    ૹ৴
    αʔόͰ৘ใ෇͚ͯ
    +40/ʹه࿥
    ϩάΛ'MVFOUEͰ
    தԝʹ

    View Slide

  28. ΫϦοΫ
    +4ͰΠϕϯτ
    ૹ৴
    αʔόͰ৘ใ෇͚ͯ
    +40/ʹه࿥
    ϩάΛ'MVFOUEͰ
    தԝʹ

    View Slide

  29. +4ͰΠϕϯτૹ৴
    wEBUBFWFOUOBNFଐੑΛॻ͍͓ͯͩ͘
    ͚Ͱૹ৴
    wσβΠφͳͲͰ΋؆୯ʹग़དྷΔΑ͏ʹ

    ͸ͯͳϒϩάΛ࢝ΊΔ(ແྉ)

    View Slide

  30. ࣮૷
    wΫϦοΫΠϕϯτΛ؂ࢹ
    wEBUBFWFOUOBNF͕͋ΔཁૉͳΒ"1*
    ʹΞΫηε

    View Slide

  31. ࣮૷ྫ
    $(document).on('mousedown', '[data-event-name]', function(e) {
    $.ajax({
    url: "/api/event", type: "POST",
    data: {
    event_name: $(this).attr('data-event-name')
    }
    });
    });

    View Slide

  32. ΫϦοΫ
    +4ͰΠϕϯτ
    ૹ৴
    αʔόͰ৘ใ෇͚ͯ
    +40/ʹه࿥
    ϩάΛ'MVFOUEͰ
    தԝʹ

    View Slide

  33. αʔόͰ+40/ϩά
    wΞΫηεʹ߹Θͤͯ+40/ΛϑΝΠϧग़
    ྗ͢Δ͚ͩ
    wBQJFWFOU FWFOU@OBNF
    w"#ςετ৘ใ΋ڞʹه࿥
    wͲͪΒͷύλʔϯ͔
    wͲͷϘλϯ FWFOU
    ͔

    View Slide

  34. FLUENT({
    tag => "event",
    event_name => "guide-register-button",
    })
    {
    "tag" : "event",
    "event_name" : "guide-register-button",
    "units" : {
    "ABUnit::GuideForm" : 1,
    "ABUnit::EditorColor" : 0,
    },
    "time" : 1384844834
    }

    View Slide

  35. ΫϦοΫ
    +4ͰΠϕϯτ
    ૹ৴
    αʔόͰ৘ใ෇͚ͯ
    +40/ʹه࿥
    ϩάΛ'MVFOUEͰ
    தԝʹ

    View Slide

  36. ϩάΛதԝʹूΊΔ
    wͲ͜Ͱه࿥͞Εͯ΋͏·͘ूΊΒΕͨ
    ͍
    wूܭ΍ՄࢹԽͷͨΊʹσʔλΛม׵͠
    ͍ͨ

    View Slide

  37. ϩάΛதԝʹूΊΔ
    wͲ͜Ͱه࿥͞Εͯ΋͏·͘ूΊΒΕͨ
    ͍
    wूܭ΍ՄࢹԽͷͨΊʹσʔλΛม׵͠
    ͍ͨ

    View Slide

  38. 'MVFOUE

    View Slide

  39. 'MVFOUE
    wγεςϜͷ֤ॴ͔Βग़ྗ͞ΕΔϩάΛ
    ഑ૹɾू໿͢ΔͨΊͷϛυϧ΢ΣΞ
    wϓϥάΠϯʹΑͬͯूΊͨϩάΛଞͷ
    σʔλʹม׵΋ग़དྷΔ

    View Slide

  40. ϩάΛूΊΔߏ੒
    w֤αʔό͸ϩάϑΝΠϧʹɺϩάߦ
    ͷ+40/ͱͯ͠ग़ྗ
    w֤αʔόͷϑΝΠϧΛUBJMͯ͠ɺBHFOU
    ͕தԝʹૹΔ͚ͩ

    View Slide

  41. View Slide

  42. ͜͜·Ͱͷ·ͱΊ
    wಠࣗσʔλଐੑΛ෇͚Δ͚ͩͰΫϦο
    Ϋϩά͕ग़དྷΔΑ͏ʹ
    wϩά͸ࣗಈͰ'MVFOUEͷதԝʹू໿͞Ε
    ΔΑ͏ʹ
    ΫϦοΫ
    +4ͰΠϕϯτ
    ૹ৴
    αʔόͰ৘ใ෇͚ͯ
    +40/ʹه࿥
    ϩάΛ'MVFOUEͰ
    தԝʹ

    View Slide

  43. View Slide

  44. ूܭɾάϥϑԽ

    View Slide

  45. දࣔͷ੾ସ
    ू·ͬͨϩάͷ
    ूܭɾάϥϑԽ
    ΫϦοΫ৘ใͷه࿥
    ɾͲͪΒͷύλʔϯʁ
    ɾͲͷϘλϯʁ

    View Slide

  46. ू໿͞Εͨϩά
    w+40/͕'MVFOUEͰू·͍ͬͯΔ͚ͩ
    w͜ͷ··Ͱ͸Կ΋Θ͔Βͳ͍

    View Slide

  47. ूܭɾՄࢹԽՄೳʹ
    w"͸˓ΫϦοΫɺ#͸˚ΫϦοΫͱ͍͏
    ৘ใΛूܭ
    wද΍άϥϑʹΑΔՄࢹԽ

    View Slide

  48. ख๏
    wॊೈʹूܭNPOHP%#Λར༻
    w؆୯ʹάϥϑԽ,JCBOBΛར༻

    View Slide

  49. ϩάΛूܭՄೳʹ
    wNPOHP%#Λར༻
    w'MVFOUEͷ+40/NPOHP%#ʹ
    w+BWB4DSJQUΛ࢖ͬͯूܭ

    View Slide

  50. 'MVFOUENPOHP%#
    w'MVFOUEͰू໿͞Εͨϩά͕NPOHP%#
    ʹೖΔΑ͏ʹ

    View Slide

  51. 'MVFOUENPOHP%#
    wqVFOUQMVHJONPOHP਺ߦͷઃఆ
    w'MVFOUEͷUBH͝ͱͷDPMMFDUJPO͕ग़དྷ
    Δ

    View Slide

  52. 'MVFOUEઃఆྫ

    type mongo
    database fluent
    collection debug
    host mongohost
    port 10000
    capped
    capped_size 100m

    View Slide

  53. NPOHP%#Ͱूܭ
    w+BWB4DSJQUͰΫΤϦΛ࡞ΓɺूܭՄೳ

    View Slide

  54. {
    "tag" : "event",
    "event_name" : "guide-register-button",
    "units" : {
    "ABUnit::GuideForm" : 1,
    },
    }
    ϩάͷܗࣜ
    "ύλʔϯͷΫϦοΫ਺ूܭ
    db.event.find({
    "event_name":"guide-register-button",
    "units.ABUnit::GuideForm":1
    }).count();

    View Slide

  55. NPOHP%#Ͱूܭ
    wΫΤϦΛ࡞Ε͹ूܭՄೳʹ
    wΫΤϦΛ޻෉͢Ε͹ॊೈʹूܭͰ͖Δ

    View Slide

  56. άϥϑԽՄೳʹ
    wॊೈʹूܭग़དྷΔ͚ͩͰ໨త͸ୡ੒
    wͨͩ͠ຖճΫΤϦΛ࡞Δͷ͸େม
    wͬ͟ͱ"#ςετঢ়گΛάϥϑԽ͍ͨ͠

    View Slide

  57. άϥϑԽՄೳʹ
    w&MBTUJD4FBSDI,JCBOBΛར༻

    View Slide

  58. ,JCBOB
    w&MBTUJDTFBSDIͷΠϯσοΫεΛར༻͠
    ͯXFC্Ͱ༷ʑͳάϥϑΛ࡞Δπʔϧ
    w&MBTUJDTFBSDI͸4PMSͷΑ͏ͳશจݕࡧ
    γεςϜ

    View Slide

  59. 'MVFOUE,JCBOB
    w'MVFOUEͰू໿͞ΕͨϩάΛ
    &MBTUJDTFBSDIʹ
    wqVFOUQMVHJOFMBTUJDTFBSDI਺ߦͷ
    ઃఆ
    w,JCBOB͸&MBTUJDTFBSDIΛݟΔΑ͏ʹ

    View Slide

  60. 'MVFOUE,JCBOB

    View Slide

  61. 'MVFOUEઃఆྫ

    type elasticsearch
    host fluenthost
    port 9200
    logstash_format true
    logstash_prefix service
    type_name service

    View Slide

  62. ,JCBOBͰάϥϑԽ
    w'MVFOUEͷσʔλ͸&MBTUJDTFBSDIʹೖ
    ΔΑ͏ʹͳͬͨ
    wXFC্Ͱ,JCBOBΛར༻ͯ͠άϥϑԽͰ
    ͖Δ

    View Slide

  63. View Slide

  64. άϥϑԽσϞ
    wશσʔλΛදͰදࣔ͠
    wΫϦοΫσʔλͷΈʹݕࡧͰߜΓࠐΈ
    w"#ςετঢ়گ͕࣠ͷ๮άϥϑΛ௥Ճ

    View Slide

  65. άϥϑԽσϞ
    {
    "tag" : "event",
    "event_name" : "guide-register-button",
    "units" : {
    "ABUnit::GuideForm" : 1,
    },
    }
    ΫϦοΫσʔλΛݕࡧ
    "#ঢ়ଶΛ࣠ʹ๮άϥϑ

    View Slide

  66. View Slide

  67. ͜͜·Ͱͷ·ͱΊ
    w'MVFOUEͷϩά͸NPOHP%#
    &MBTUJDTFBSDIʹ
    wNPOHP%#ͰॊೈʹूܭՄೳʹ
    w,JCBOBͰ؆୯ʹάϥϑԽՄೳʹ

    View Slide

  68. ࠷ऴతͳ࢓૊Έ

    View Slide

  69. View Slide

  70. ؆୯ͳ"#ςετखॱ
    wϘλϯʹEBUBFWFOUOBNFͱ͍͏ଐੑ
    ௥Ճ
    wϦϦʔεޙ,JCBOBͰάϥϑԽ
    w࠷ऴతʹNPOHP%#Ͱूܭ͠ɺ݁ՌΛ
    ֬ೝ

    View Slide

  71. ·ͱΊ
    w͸ͯͳϒϩάͰ͸"#ςετ΍σʔλղ
    ੳͳͲͷ࢓ࣄ΋΍͍ͬͯ·͢
    w'MVFOUE NPOHP%# ,JCBOBΛར༻͠
    ͨ"#ςετΛ؆୯ʹߦ͏࢓૊ΈΛ঺հ
    wࣗࣾͰ࢓૊ΈΛ࡞ͬͨͷͰɺ΋ͬͱෳ
    ࡶͳ"#ςετ΋ग़དྷ·͢

    View Slide

  72. ͝ਗ਼ௌ͋Γ͕ͱ͏
    ͍͟͝·ͨ͠

    View Slide