ElasticBeatsを導入してみた話/Go Conference 2016 Spring

Transcript

1. &MBTUJD
   #FBUΛಋೖͯ͠Έͨ࿩ (P
   $POGFSFODF
   
   4QSJOH %BJDIJ
   )*3"5" $ZCFS"HFOU
   *OD
2. w גࣜձࣾαΠόʔΤʔδΣϯτ
 "EUFDI
   4UVEJP
 ॴଐ
   w %BJDIJ
   )*3"5"!EBJDIJME
   w (P
   3VCZ ࣗݾ঺հ
4. "HFOEB w &MBTUJD
   #FBUTʹ͍ͭͯ
   w #FBUTͷ࡞Γํ
   w ߏ੒ͱɺ,JCBOBʹදࣔ͢Δ·Ͱ
6. &MBTUJD
   #FBUT w %BUB
   4IJQQFST
   GPS
   &MBTUJDTFBSDI
   w &MBTUJDTFBSDI޲͚ͷ+40/Λग़ྗ
   w (P੡
   
   #JOBSZ
   w 3VOUJNF΁ͷґଘੑ͕ແ͍
   w ܰྔͰγϯϓϧ
7. &MBTUJD
   #FBUT
8. 0GGJDJBM
   #FBUT w 1BDLFUCFBU
   w 5PQCFBU
   w 'JMFCFBU
   w 8JOMPHCFBU
9. 1BDLFUCFBU
10. 1BDLFUCFBU • )551
    • .Z42-
    • 1PTUHSF42-
    • 3FEJT
    • 5ISJGU31$
    • .POHP%#
    • %/4
    • .FNDBDIF
11. 5PQCFBU
12. 'JMFCFBU
13. 8JOMPHCFBU
14. $PNNVOJUZ
    #FBUT w "QBDIFCFBU
    w /HJOYCFBU
    w %PDLFSCFBU
    w )551CFBU
    w /BHJPTDIFDLCFBU
    w FUDʜ
15. ࣮ࡍʹ͔ͭͬͯΈͨ
16. "QBDIF
    ,BGLB • -JOLFE*O͔Βެ։͞Εͨ෼ࢄϝοηʔδϯά γεςϜ
    • #SPLFS $MVTUFS
    1SPEVDFS
    $POTVNFSͷ ͭͷίϯϙʔωϯτͰߏ੒͞ΕΔ
17. "QBDIF
    ,BGLB • ϦΞϧλΠϜʹॲཧΛߦ͏ετϦʔϜܥͷॲཧͱɺ େ͖ͳεύΠΫ͕ൃੜ͢ΔॲཧͷҰ࣍ड͚ͱͯ͠ ࢖༻
18. 1BSUJUJPO
    0GGTFU $POTVNFS(SPVQ
19. 1BSUJUJPO
    0GGTFU -BH
    
20. ϞχλϦϯάπʔϧ • ZBIPPLBGLBNBOBHFS
    • RVBOUJpOE,BGLB0⒎TFU.POJUPS
    • MJOLFEJO#VSSPX
21. ϞχλϦϯάπʔϧ • WҎ߱ͷ৽͍͠0⒎TFU
    4UPSF΁ͷରԠ͕· ͪ·ͪ
    • 0⒎TFU
    ͷऔಘํ๏͕ͪΐͬͱඍົ
    • LJCBOBΛ࢖ͬͨΞΫηεϩάͷ؂ࢹ؀ڥ͕ط ʹ͋ΔͷͰɺಉ͡৔ॴͰݟΕΔΑ͏ʹ͍ͨ͠
22. LBGLBCFBU
23. LBGLBCFBU • 4IPQJGZTBSBNBΛ࢖ͬͯ#JOBSZ
    "1*Λ͋Ε ͜ΕݺΜͰ0⒎TFUΛऔಘ
    • +PMPLJB +.9
    XJUI
    +40/
    PWFS
    )551 
 ܦ༝Ͱ#SPLFS಺෦ͷ৘ใΛऔಘ
24. FMBTUJDCFBUHFOFSBUPS $ cookiecutter https://github.com/elastic/beat-generator.git
25. FMBTUJDCFBUHFOFSBUPS ɾbeater/kafkabeat.go ɾconfig/config.go ɾetc/beat.yml ͜ͷ̏ͭΛฤू͢Δ
26. FMBTUJDCFBUHFOFSBUPS ϥΠϒϥϦͷ؅ཧʹ͸glide
27. Configuration Logging Daemon / Service Publisher Beat custom logic events channel libbeat … MJCCFBU
    0WFSWJFX beater
28. #FBUFS
    JOUFSGBDF • #FBUFS͕΍Δ΂͖͜ͱ
    • 4USBUFHZ
    ύλʔϯ
29. $POGJH
    .FUIPE • ઃఆϑΝΠϧͷಡΈࠐΈ
    • $-*ϑϥάͷϋϯυϦϯά
30. $POGJH
    .FUIPE
31. $POGJH
    .FUIPE
32. 4FUVQ
    .FUIPE • ϝΠϯϧʔϓલͷॳظԽॲཧ
    • ΫϥΠΞϯτͷ઀ଓͱ͔͸͜͜Ͱߦ͏
33. 4FUVQ
    .FUIPE
34. 3VO
    .FUIPE • ϝΠϯϧʔϓ
    • σʔλͷऩू
    • 1VCMJTIFS΁ͷૹ৴
    • γάφϧϋϯυϦϯάͳͲ͸MJCCFBUʹ͓೚ͤ
35. 3VO
    .FUIPE
37. $MFBOVQ
    .FUIPE • 3VO
    Λൈ͚ͨ௚ޙʹݺ͹ΕΔ
    • ֬อͨ͠Ϧιʔεͷ։์ͳͲΛߦ͏
38. 4UPQ
    .FUIPE • 3VO
    ͰγάφϧΛड͚Δͱݺ͹ΕΔ
    • ϝΠϯϧʔϓΛࢭΊΔҝͷॲཧΛߦ͏
    • ىಈͨ͠HP
    SPVUJOFͷఀࢭ
39. EFWUPPMT • JOEFYUFNQMBUF
    HFOFSBUPS
    • EPDVNFOU
    HFOFSBUPS
    • CFBU
    QBDLFS
    • %PDLFSΛ࢖ͬͯDSPTTDPNQJMJOHͱQBDLBHJOH
    EFCɾSQNɾXJOɾPTY
    • ,JCBOB
    %BTICPBSE
    FYQPSUJNQPSU
    • FUD
40. Kafka Consumer Log Store Kafka Broker Kafka Broker in_tail GCE VPN elasticsearch elasticsearch kibana fluentd ߏ੒
41. ,JCBOB
    %BTICPBSE
42. ·ͱΊ • (Pͷܰྔɾߴ଎ɾϙʔλϏϦςΟ͸#FBUͷΑ ͏ͳ"HFOUʹඇৗʹϚον͢Δ
    • &MBTUJDTFBSDI
    
    ,JCBOBͷ؀ڥ͕͋Ε͹͙͢ ʹͰ΋࢝ΊΒΕΔ͓खܰ͞
    • ڞ௨෦෼Λ͏·͘੾Γग़͠γϯϓϧʹอͬͯ ͍ΔͨΊɺಠࣗͷ#FBUΛ࡞Δͷ͕؆୯
43. ·ͱΊ • LBGLBCFBU͸044Խ༧ఆͰ͢
    • ؒʹ߹͍·ͤΜͰͨ͠
    N @@ N
44. (P΍($1Λ࢖ͬͨ։ൃ͕͍ͨ͠ਓʂ