ElasticBeatsを導入してみた話/Go Conference 2016 Spring

Transcript

1. &MBTUJD
   #FBUΛಋೖͯ͠Έͨ࿩ (P
   $POGFSFODF
   
   4QSJOH %BJDIJ
   )*3"5" $ZCFS"HFOU
   *OD

2. w גࣜձࣾαΠόʔΤʔδΣϯτ
 "EUFDI
   4UVEJP
 ॴଐ
   w %BJDIJ
   )*3"5"!EBJDIJME
   w (P
   3VCZ ࣗݾ঺հ

3. None

4. "HFOEB w &MBTUJD
   #FBUTʹ͍ͭͯ
   w #FBUTͷ࡞Γํ
   w ߏ੒ͱɺ,JCBOBʹදࣔ͢Δ·Ͱ

5. None

6. &MBTUJD
   #FBUT w %BUB
   4IJQQFST
   GPS
   &MBTUJDTFBSDI
   w &MBTUJDTFBSDI޲͚ͷ+40/Λग़ྗ
   w (P੡
   
   #JOBSZ
   w

   3VOUJNF΁ͷґଘੑ͕ແ͍
   w ܰྔͰγϯϓϧ

7. &MBTUJD
   #FBUT

8. 0GGJDJBM
   #FBUT w 1BDLFUCFBU
   w 5PQCFBU
   w 'JMFCFBU
   w 8JOMPHCFBU

9. 1BDLFUCFBU

10. 1BDLFUCFBU • )551
    • .Z42-
    • 1PTUHSF42-
    • 3FEJT
    •

    5ISJGU31$
    • .POHP%#
    • %/4
    • .FNDBDIF

11. 5PQCFBU

12. 'JMFCFBU

13. 8JOMPHCFBU

14. $PNNVOJUZ
    #FBUT w "QBDIFCFBU
    w /HJOYCFBU
    w %PDLFSCFBU
    w )551CFBU
    w

    /BHJPTDIFDLCFBU
    w FUDʜ

15. ࣮ࡍʹ͔ͭͬͯΈͨ

16. "QBDIF
    ,BGLB • -JOLFE*O͔Βެ։͞Εͨ෼ࢄϝοηʔδϯά γεςϜ
    • #SPLFS $MVTUFS
    1SPEVDFS
    $POTVNFSͷ ͭͷίϯϙʔωϯτͰߏ੒͞ΕΔ

17. "QBDIF
    ,BGLB • ϦΞϧλΠϜʹॲཧΛߦ͏ετϦʔϜܥͷॲཧͱɺ େ͖ͳεύΠΫ͕ൃੜ͢ΔॲཧͷҰ࣍ड͚ͱͯ͠ ࢖༻

18. 1BSUJUJPO
    0GGTFU $POTVNFS(SPVQ

19. 1BSUJUJPO
    0GGTFU -BH
    

20. ϞχλϦϯάπʔϧ • ZBIPPLBGLBNBOBHFS
    • RVBOUJpOE,BGLB0⒎TFU.POJUPS
    • MJOLFEJO#VSSPX

21. ϞχλϦϯάπʔϧ • WҎ߱ͷ৽͍͠0⒎TFU
    4UPSF΁ͷରԠ͕· ͪ·ͪ
    • 0⒎TFU
    ͷऔಘํ๏͕ͪΐͬͱඍົ
    • LJCBOBΛ࢖ͬͨΞΫηεϩάͷ؂ࢹ؀ڥ͕ط ʹ͋ΔͷͰɺಉ͡৔ॴͰݟΕΔΑ͏ʹ͍ͨ͠

22. LBGLBCFBU

23. LBGLBCFBU • 4IPQJGZTBSBNBΛ࢖ͬͯ#JOBSZ
    "1*Λ͋Ε ͜ΕݺΜͰ0⒎TFUΛऔಘ
    • +PMPLJB +.9
    XJUI
    +40/
    PWFS
    )551 
 ܦ༝Ͱ#SPLFS಺෦ͷ৘ใΛऔಘ

24. FMBTUJDCFBUHFOFSBUPS $ cookiecutter https://github.com/elastic/beat-generator.git

25. FMBTUJDCFBUHFOFSBUPS ɾbeater/kafkabeat.go ɾconfig/config.go ɾetc/beat.yml ͜ͷ̏ͭΛฤू͢Δ

26. FMBTUJDCFBUHFOFSBUPS ϥΠϒϥϦͷ؅ཧʹ͸glide

27. Configuration Logging Daemon / Service Publisher Beat custom logic events

    channel libbeat … MJCCFBU
    0WFSWJFX beater

28. #FBUFS
    JOUFSGBDF • #FBUFS͕΍Δ΂͖͜ͱ
    • 4USBUFHZ
    ύλʔϯ

29. $POGJH
    .FUIPE • ઃఆϑΝΠϧͷಡΈࠐΈ
    • $-*ϑϥάͷϋϯυϦϯά

30. $POGJH
    .FUIPE

31. $POGJH
    .FUIPE

32. 4FUVQ
    .FUIPE • ϝΠϯϧʔϓલͷॳظԽॲཧ
    • ΫϥΠΞϯτͷ઀ଓͱ͔͸͜͜Ͱߦ͏

33. 4FUVQ
    .FUIPE

34. 3VO
    .FUIPE • ϝΠϯϧʔϓ
    • σʔλͷऩू
    • 1VCMJTIFS΁ͷૹ৴
    • γάφϧϋϯυϦϯάͳͲ͸MJCCFBUʹ͓೚ͤ

35. 3VO
    .FUIPE

36. None

37. $MFBOVQ
    .FUIPE • 3VO
    Λൈ͚ͨ௚ޙʹݺ͹ΕΔ
    • ֬อͨ͠Ϧιʔεͷ։์ͳͲΛߦ͏

38. 4UPQ
    .FUIPE • 3VO
    ͰγάφϧΛड͚Δͱݺ͹ΕΔ
    • ϝΠϯϧʔϓΛࢭΊΔҝͷॲཧΛߦ͏
    • ىಈͨ͠HP
    SPVUJOFͷఀࢭ

39. EFWUPPMT • JOEFYUFNQMBUF
    HFOFSBUPS
    • EPDVNFOU
    HFOFSBUPS
    • CFBU
    QBDLFS
    • %PDLFSΛ࢖ͬͯDSPTTDPNQJMJOHͱQBDLBHJOH
    EFCɾSQNɾXJOɾPTY

    
    • ,JCBOB
    %BTICPBSE
    FYQPSUJNQPSU
    • FUD

40. Kafka Consumer Log Store Kafka Broker Kafka Broker in_tail GCE

    VPN elasticsearch elasticsearch kibana fluentd ߏ੒

41. ,JCBOB
    %BTICPBSE

42. ·ͱΊ • (Pͷܰྔɾߴ଎ɾϙʔλϏϦςΟ͸#FBUͷΑ ͏ͳ"HFOUʹඇৗʹϚον͢Δ
    • &MBTUJDTFBSDI
    
    ,JCBOBͷ؀ڥ͕͋Ε͹͙͢ ʹͰ΋࢝ΊΒΕΔ͓खܰ͞
    • ڞ௨෦෼Λ͏·͘੾Γग़͠γϯϓϧʹอͬͯ

    ͍ΔͨΊɺಠࣗͷ#FBUΛ࡞Δͷ͕؆୯

43. ·ͱΊ • LBGLBCFBU͸044Խ༧ఆͰ͢
    • ؒʹ߹͍·ͤΜͰͨ͠
    N @@ N

44. (P΍($1Λ࢖ͬͨ։ൃ͕͍ͨ͠ਓʂ