Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fluentd, mongoDB, Kibanaを利用したはてなブログABテストの事例

shibayu36
November 25, 2013

Fluentd, mongoDB, Kibanaを利用したはてなブログABテストの事例

「Hatena Engineer Seminar #2」で発表した資料です

shibayu36

November 25, 2013
Tweet

More Decks by shibayu36

Other Decks in Programming

Transcript

  1. 'MVFOUE NPOHP%# ,JCBOB
    Λར༻ͨ͠
    ͸ͯͳϒϩά"#ςετͷࣄྫ
    )BUFOB&OHJOFFS4FNJOBS
    JETIJCB@ZV

    View full-size slide

  2. ຊ೔ͷΞδΣϯμ
    w͸ͯͳϒϩάͰͷ"#ςετࣄྫ঺հ
    w؆୯ʹ"#ςετΛߦ͏࢓૊Έ঺հ

    View full-size slide

  3. ࣗݾ঺հ
    wJETIJCB@ZV !TIJCB@ZV

    wΞϓϦέʔγϣϯΤϯδχΞ
    ೥ೖࣾ

    View full-size slide

  4. ࣗݾ঺հ
    w͸ͯͳϒϩάνʔϜ
    wཪଆΑΓͷ։ൃ͕ଟ͍
    wσʔλूܭ
    wσϓϩΠपΓ
    wͦͷଞ৭ʑ

    View full-size slide

  5. ͸ͯͳϒϩάͰͷ
    "#ςετࣄྫ

    View full-size slide

  6. ͸ͯͳϒϩάͰ͸
    wαʔϏεͷվળʹ"#ςετΛಋೖ
    wϥϯσΟϯάϖʔδͷվળ
    wฤूը໘ͷվળ

    View full-size slide

  7. ͸ͯͳϒϩάͰ͸
    wαʔϏεͷվળʹ"#ςετΛಋೖ
    wϥϯσΟϯάϖʔδͷվળ
    wฤूը໘ͷվળ

    View full-size slide

  8. ϥϯσΟϯάϖʔδվળ
    wొ࿥ϘλϯͷΫϦοΫ਺Λ৳͹͍ͨ͠
    wϘλϯͱϑΥʔϜͲͪΒ͕ΫϦοΫ͞
    ΕΔ

    View full-size slide

  9. ςετ݁Ռ

    ΫϦοΫ཰্͕ঢ

    View full-size slide

  10. "#ςετΛ͢Δͱ
    wਪଌͰ͸ͳ͘ɺ਺ࣈΛݕূ͠ͳ͕Βվ
    ળ͕ग़དྷΔ
    w։ൃऀͷצͰ࡞Βͳ͍
    wࣦഊͯ͠΋ৼΓฦΓ͕ग़དྷΔ

    View full-size slide

  11. ೔ʑ"#ςετΛ͢Δʹ͸
    w͸ͯͳϒϩάͰ͸೔ʑ"#ςετΛͯ͠
    ͍Δ
    w࣮ࢪ͕ͱʹ͔͘؆୯Ͱͳ͍ͱବ໨
    w೉͍͠ͱ΍Βͳ͍

    View full-size slide

  12. ؆୯ʹ"#ςετΛ
    ͢Δ࢓૊Έ

    View full-size slide

  13. ϥϯσΟϯάϖʔδͷ
    "#ςετखॱ
    wදࣔΛग़͠෼͚Δ
    wϘλϯΛΫϦοΫͨ࣌͠ʹϩάΛه࿥
    wू·ͬͨϩάΛूܭɾάϥϑԽ͢Δ

    View full-size slide

  14. දࣔͷ੾ସ
    ू·ͬͨϩάͷ
    ूܭɾάϥϑԽ
    ΫϦοΫ৘ใͷه࿥
    ɾͲͪΒͷύλʔϯʁ
    ɾͲͷϘλϯʁ

    View full-size slide

  15. wදࣔΛग़͠෼͚
    wDIBOLPͷΑ͏ͳ࢓૊Έ
    wΫϦοΫϩάͷه࿥
    w+4ͰͷΠϕϯτૹ৴ɾ'MVFOUE
    wूܭɾάϥϑԽ
    w.POHP%#ɺ,JCBOBͷར༻
    Ͱ͖Δ͚ͩ؆୯ʹ

    View full-size slide

  16. wදࣔΛग़͠෼͚
    wDIBOLPͷΑ͏ͳ࢓૊Έ
    wΫϦοΫϩάͷه࿥
    w+4ͰͷΠϕϯτૹ৴ɾ'MVFOUE
    wूܭɾάϥϑԽ
    wNPOHP%#ɺ,JCBOBͷར༻
    Ͱ͖Δ͚ͩ؆୯ʹ

    View full-size slide

  17. දࣔͷग़͠Θ͚

    View full-size slide

  18. දࣔͷ੾ସ
    ू·ͬͨϩάͷ
    ूܭɾάϥϑԽ
    ΫϦοΫ৘ใͷه࿥
    ɾͲͪΒͷύλʔϯʁ
    ɾͲͷϘλϯʁ

    View full-size slide

  19. දࣔͷग़͠Θ͚
    wࠓճ͸͋·Γ࿩͠·ͤΜ
    w͜ͷ෦෼͸ग़དྷ͍ͯΔͱ͍͏લఏͰ࿩
    ͠·͢

    View full-size slide

  20. දࣔͷग़͠Θ͚
    w*1ΞυϨεͳͲͰग़͠Θ͚
    wSVCZͷDIBOLP
    wDPPLQBEࣾ੡
    w͸ͯͳϒϩάͰ͸ಉ͡Α͏ͳ࢓૊ΈΛ
    ࡞͍ͬͯΔ

    View full-size slide

  21. ΫϦοΫϩάͷه࿥

    View full-size slide

  22. දࣔͷ੾ସ
    ू·ͬͨϩάͷ
    ूܭɾάϥϑԽ
    ΫϦοΫ৘ใͷه࿥
    ɾͲͪΒͷύλʔϯʁ
    ɾͲͷϘλϯʁ

    View full-size slide

  23. ؆୯ͳྲྀΕ
    wΫϦοΫ͞ΕͨΒΠϕϯτ͕αʔόʹ
    ૹ৴
    wαʔόͰ͸"#ςετ৘ใͱڞʹ+40/
    Ͱه࿥
    w+40/ͷϩά͸'MVFOUEͰதԝαʔόʹ
    ूΊΔ

    View full-size slide

  24. ΫϦοΫ
    +4ͰΠϕϯτ
    ૹ৴
    αʔόͰ৘ใ෇͚ͯ
    +40/ʹه࿥
    ϩάΛ'MVFOUEͰ
    தԝʹ

    View full-size slide

  25. ΫϦοΫ
    +4ͰΠϕϯτ
    ૹ৴
    αʔόͰ৘ใ෇͚ͯ
    +40/ʹه࿥
    ϩάΛ'MVFOUEͰ
    தԝʹ

    View full-size slide

  26. +4ͰΠϕϯτૹ৴
    wEBUBFWFOUOBNFଐੑΛॻ͍͓ͯͩ͘
    ͚Ͱૹ৴
    wσβΠφͳͲͰ΋؆୯ʹग़དྷΔΑ͏ʹ

    ͸ͯͳϒϩάΛ࢝ΊΔ(ແྉ)

    View full-size slide

  27. ࣮૷
    wΫϦοΫΠϕϯτΛ؂ࢹ
    wEBUBFWFOUOBNF͕͋ΔཁૉͳΒ"1*
    ʹΞΫηε

    View full-size slide

  28. ࣮૷ྫ
    $(document).on('mousedown', '[data-event-name]', function(e) {
    $.ajax({
    url: "/api/event", type: "POST",
    data: {
    event_name: $(this).attr('data-event-name')
    }
    });
    });

    View full-size slide

  29. ΫϦοΫ
    +4ͰΠϕϯτ
    ૹ৴
    αʔόͰ৘ใ෇͚ͯ
    +40/ʹه࿥
    ϩάΛ'MVFOUEͰ
    தԝʹ

    View full-size slide

  30. αʔόͰ+40/ϩά
    wΞΫηεʹ߹Θͤͯ+40/ΛϑΝΠϧग़
    ྗ͢Δ͚ͩ
    wBQJFWFOU FWFOU@OBNF
    w"#ςετ৘ใ΋ڞʹه࿥
    wͲͪΒͷύλʔϯ͔
    wͲͷϘλϯ FWFOU
    ͔

    View full-size slide

  31. FLUENT({
    tag => "event",
    event_name => "guide-register-button",
    })
    {
    "tag" : "event",
    "event_name" : "guide-register-button",
    "units" : {
    "ABUnit::GuideForm" : 1,
    "ABUnit::EditorColor" : 0,
    },
    "time" : 1384844834
    }

    View full-size slide

  32. ΫϦοΫ
    +4ͰΠϕϯτ
    ૹ৴
    αʔόͰ৘ใ෇͚ͯ
    +40/ʹه࿥
    ϩάΛ'MVFOUEͰ
    தԝʹ

    View full-size slide

  33. ϩάΛதԝʹूΊΔ
    wͲ͜Ͱه࿥͞Εͯ΋͏·͘ूΊΒΕͨ
    ͍
    wूܭ΍ՄࢹԽͷͨΊʹσʔλΛม׵͠
    ͍ͨ

    View full-size slide

  34. ϩάΛதԝʹूΊΔ
    wͲ͜Ͱه࿥͞Εͯ΋͏·͘ूΊΒΕͨ
    ͍
    wूܭ΍ՄࢹԽͷͨΊʹσʔλΛม׵͠
    ͍ͨ

    View full-size slide

  35. 'MVFOUE
    wγεςϜͷ֤ॴ͔Βग़ྗ͞ΕΔϩάΛ
    ഑ૹɾू໿͢ΔͨΊͷϛυϧ΢ΣΞ
    wϓϥάΠϯʹΑͬͯूΊͨϩάΛଞͷ
    σʔλʹม׵΋ग़དྷΔ

    View full-size slide

  36. ϩάΛूΊΔߏ੒
    w֤αʔό͸ϩάϑΝΠϧʹɺϩάߦ
    ͷ+40/ͱͯ͠ग़ྗ
    w֤αʔόͷϑΝΠϧΛUBJMͯ͠ɺBHFOU
    ͕தԝʹૹΔ͚ͩ

    View full-size slide

  37. ͜͜·Ͱͷ·ͱΊ
    wಠࣗσʔλଐੑΛ෇͚Δ͚ͩͰΫϦο
    Ϋϩά͕ग़དྷΔΑ͏ʹ
    wϩά͸ࣗಈͰ'MVFOUEͷதԝʹू໿͞Ε
    ΔΑ͏ʹ
    ΫϦοΫ
    +4ͰΠϕϯτ
    ૹ৴
    αʔόͰ৘ใ෇͚ͯ
    +40/ʹه࿥
    ϩάΛ'MVFOUEͰ
    தԝʹ

    View full-size slide

  38. ूܭɾάϥϑԽ

    View full-size slide

  39. දࣔͷ੾ସ
    ू·ͬͨϩάͷ
    ूܭɾάϥϑԽ
    ΫϦοΫ৘ใͷه࿥
    ɾͲͪΒͷύλʔϯʁ
    ɾͲͷϘλϯʁ

    View full-size slide

  40. ू໿͞Εͨϩά
    w+40/͕'MVFOUEͰू·͍ͬͯΔ͚ͩ
    w͜ͷ··Ͱ͸Կ΋Θ͔Βͳ͍

    View full-size slide

  41. ूܭɾՄࢹԽՄೳʹ
    w"͸˓ΫϦοΫɺ#͸˚ΫϦοΫͱ͍͏
    ৘ใΛूܭ
    wද΍άϥϑʹΑΔՄࢹԽ

    View full-size slide

  42. ख๏
    wॊೈʹूܭNPOHP%#Λར༻
    w؆୯ʹάϥϑԽ,JCBOBΛར༻

    View full-size slide

  43. ϩάΛूܭՄೳʹ
    wNPOHP%#Λར༻
    w'MVFOUEͷ+40/NPOHP%#ʹ
    w+BWB4DSJQUΛ࢖ͬͯूܭ

    View full-size slide

  44. 'MVFOUENPOHP%#
    w'MVFOUEͰू໿͞Εͨϩά͕NPOHP%#
    ʹೖΔΑ͏ʹ

    View full-size slide

  45. 'MVFOUENPOHP%#
    wqVFOUQMVHJONPOHP਺ߦͷઃఆ
    w'MVFOUEͷUBH͝ͱͷDPMMFDUJPO͕ग़དྷ
    Δ

    View full-size slide

  46. 'MVFOUEઃఆྫ

    type mongo
    database fluent
    collection debug
    host mongohost
    port 10000
    capped
    capped_size 100m

    View full-size slide

  47. NPOHP%#Ͱूܭ
    w+BWB4DSJQUͰΫΤϦΛ࡞ΓɺूܭՄೳ

    View full-size slide

  48. {
    "tag" : "event",
    "event_name" : "guide-register-button",
    "units" : {
    "ABUnit::GuideForm" : 1,
    },
    }
    ϩάͷܗࣜ
    "ύλʔϯͷΫϦοΫ਺ूܭ
    db.event.find({
    "event_name":"guide-register-button",
    "units.ABUnit::GuideForm":1
    }).count();

    View full-size slide

  49. NPOHP%#Ͱूܭ
    wΫΤϦΛ࡞Ε͹ूܭՄೳʹ
    wΫΤϦΛ޻෉͢Ε͹ॊೈʹूܭͰ͖Δ

    View full-size slide

  50. άϥϑԽՄೳʹ
    wॊೈʹूܭग़དྷΔ͚ͩͰ໨త͸ୡ੒
    wͨͩ͠ຖճΫΤϦΛ࡞Δͷ͸େม
    wͬ͟ͱ"#ςετঢ়گΛάϥϑԽ͍ͨ͠

    View full-size slide

  51. άϥϑԽՄೳʹ
    w&MBTUJD4FBSDI,JCBOBΛར༻

    View full-size slide

  52. ,JCBOB
    w&MBTUJDTFBSDIͷΠϯσοΫεΛར༻͠
    ͯXFC্Ͱ༷ʑͳάϥϑΛ࡞Δπʔϧ
    w&MBTUJDTFBSDI͸4PMSͷΑ͏ͳશจݕࡧ
    γεςϜ

    View full-size slide

  53. 'MVFOUE,JCBOB
    w'MVFOUEͰू໿͞ΕͨϩάΛ
    &MBTUJDTFBSDIʹ
    wqVFOUQMVHJOFMBTUJDTFBSDI਺ߦͷ
    ઃఆ
    w,JCBOB͸&MBTUJDTFBSDIΛݟΔΑ͏ʹ

    View full-size slide

  54. 'MVFOUE,JCBOB

    View full-size slide

  55. 'MVFOUEઃఆྫ

    type elasticsearch
    host fluenthost
    port 9200
    logstash_format true
    logstash_prefix service
    type_name service

    View full-size slide

  56. ,JCBOBͰάϥϑԽ
    w'MVFOUEͷσʔλ͸&MBTUJDTFBSDIʹೖ
    ΔΑ͏ʹͳͬͨ
    wXFC্Ͱ,JCBOBΛར༻ͯ͠άϥϑԽͰ
    ͖Δ

    View full-size slide

  57. άϥϑԽσϞ
    wશσʔλΛදͰදࣔ͠
    wΫϦοΫσʔλͷΈʹݕࡧͰߜΓࠐΈ
    w"#ςετঢ়گ͕࣠ͷ๮άϥϑΛ௥Ճ

    View full-size slide

  58. άϥϑԽσϞ
    {
    "tag" : "event",
    "event_name" : "guide-register-button",
    "units" : {
    "ABUnit::GuideForm" : 1,
    },
    }
    ΫϦοΫσʔλΛݕࡧ
    "#ঢ়ଶΛ࣠ʹ๮άϥϑ

    View full-size slide

  59. ͜͜·Ͱͷ·ͱΊ
    w'MVFOUEͷϩά͸NPOHP%#
    &MBTUJDTFBSDIʹ
    wNPOHP%#ͰॊೈʹूܭՄೳʹ
    w,JCBOBͰ؆୯ʹάϥϑԽՄೳʹ

    View full-size slide

  60. ࠷ऴతͳ࢓૊Έ

    View full-size slide

  61. ؆୯ͳ"#ςετखॱ
    wϘλϯʹEBUBFWFOUOBNFͱ͍͏ଐੑ
    ௥Ճ
    wϦϦʔεޙ,JCBOBͰάϥϑԽ
    w࠷ऴతʹNPOHP%#Ͱूܭ͠ɺ݁ՌΛ
    ֬ೝ

    View full-size slide

  62. ·ͱΊ
    w͸ͯͳϒϩάͰ͸"#ςετ΍σʔλղ
    ੳͳͲͷ࢓ࣄ΋΍͍ͬͯ·͢
    w'MVFOUE NPOHP%# ,JCBOBΛར༻͠
    ͨ"#ςετΛ؆୯ʹߦ͏࢓૊ΈΛ঺հ
    wࣗࣾͰ࢓૊ΈΛ࡞ͬͨͷͰɺ΋ͬͱෳ
    ࡶͳ"#ςετ΋ग़དྷ·͢

    View full-size slide

  63. ͝ਗ਼ௌ͋Γ͕ͱ͏
    ͍͟͝·ͨ͠

    View full-size slide