Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
ElasticBeatsを導入してみた話/Go Conference 2016 Spring
Daichi Hirata
April 23, 2016
Technology
3
4.9k
ElasticBeatsを導入してみた話/Go Conference 2016 Spring
Daichi Hirata
April 23, 2016
Tweet
Share
More Decks by Daichi Hirata
See All by Daichi Hirata
daic_h
7
3.2k
daic_h
0
49
daic_h
2
9.5k
Other Decks in Technology
See All in Technology
tenjuu99
1
310
khrd
1
670
dena_tech
3
650
jrfk
0
210
no24oka
0
100
ishiayaya
PRO
0
210
mahito
0
240
uzabase_saas_product
0
110
clustervr
0
250
go5paopao
4
540
noriyukitakei
0
100
hiroyaiizuka
0
190
Featured
See All Featured
rasmusluckow
318
18k
ammeep
656
54k
frogandcode
128
20k
rocio
155
11k
zakiwarfel
88
3.4k
philhawksworth
192
8.8k
smashingmag
283
47k
jonrohan
1021
380k
davidbonilla
70
3.6k
jnunemaker
PRO
40
4.6k
mthomps
38
2.3k
mojombo
359
62k
Transcript
&MBTUJD#FBUΛಋೖͯ͠Έͨ (P$POGFSFODF4QSJOH %BJDIJ)*3"5" $ZCFS"HFOU *OD
w גࣜձࣾαΠόʔΤʔδΣϯτ "EUFDI4UVEJP ॴଐ w %BJDIJ)*3"5"!EBJDIJME w (P 3VCZ ࣗݾհ
None
"HFOEB w &MBTUJD#FBUTʹ͍ͭͯ w #FBUTͷ࡞Γํ w ߏͱɺ,JCBOBʹදࣔ͢Δ·Ͱ
None
&MBTUJD#FBUT w %BUB4IJQQFSTGPS&MBTUJDTFBSDI w &MBTUJDTFBSDI͚ͷ+40/Λग़ྗ w (P #JOBSZ w
3VOUJNFͷґଘੑ͕ແ͍ w ܰྔͰγϯϓϧ
&MBTUJD#FBUT
0GGJDJBM#FBUT w 1BDLFUCFBU w 5PQCFBU w 'JMFCFBU w 8JOMPHCFBU
1BDLFUCFBU
1BDLFUCFBU • )551 • .Z42- • 1PTUHSF42- • 3FEJT •
5ISJGU31$ • .POHP%# • %/4 • .FNDBDIF
5PQCFBU
'JMFCFBU
8JOMPHCFBU
$PNNVOJUZ#FBUT w "QBDIFCFBU w /HJOYCFBU w %PDLFSCFBU w )551CFBU w
/BHJPTDIFDLCFBU w FUDʜ
࣮ࡍʹ͔ͭͬͯΈͨ
"QBDIF,BGLB • -JOLFE*O͔Βެ։͞Εͨࢄϝοηʔδϯά γεςϜ • #SPLFS $MVTUFS 1SPEVDFS $POTVNFSͷ ͭͷίϯϙʔωϯτͰߏ͞ΕΔ
"QBDIF,BGLB • ϦΞϧλΠϜʹॲཧΛߦ͏ετϦʔϜܥͷॲཧͱɺ େ͖ͳεύΠΫ͕ൃੜ͢ΔॲཧͷҰ࣍ड͚ͱͯ͠ ༻
1BSUJUJPO0GGTFU $POTVNFS(SPVQ
1BSUJUJPO0GGTFU -BH
ϞχλϦϯάπʔϧ • ZBIPPLBGLBNBOBHFS • RVBOUJpOE,BGLB0⒎TFU.POJUPS • MJOLFEJO#VSSPX
ϞχλϦϯάπʔϧ • WҎ߱ͷ৽͍͠0⒎TFU4UPSFͷରԠ͕· ͪ·ͪ • 0⒎TFUͷऔಘํ๏͕ͪΐͬͱඍົ • LJCBOBΛͬͨΞΫηεϩάͷࢹڥ͕ط ʹ͋ΔͷͰɺಉ͡ॴͰݟΕΔΑ͏ʹ͍ͨ͠
LBGLBCFBU
LBGLBCFBU • 4IPQJGZTBSBNBΛͬͯ#JOBSZ"1*Λ͋Ε ͜ΕݺΜͰ0⒎TFUΛऔಘ • +PMPLJB +.9XJUI+40/PWFS)551 ܦ༝Ͱ#SPLFS෦ͷใΛऔಘ
FMBTUJDCFBUHFOFSBUPS $ cookiecutter https://github.com/elastic/beat-generator.git
FMBTUJDCFBUHFOFSBUPS ɾbeater/kafkabeat.go ɾconfig/config.go ɾetc/beat.yml ͜ͷ̏ͭΛฤू͢Δ
FMBTUJDCFBUHFOFSBUPS ϥΠϒϥϦͷཧʹglide
Configuration Logging Daemon / Service Publisher Beat custom logic events
channel libbeat … MJCCFBU0WFSWJFX beater
#FBUFSJOUFSGBDF • #FBUFS͕Δ͖͜ͱ • 4USBUFHZύλʔϯ
$POGJH.FUIPE • ઃఆϑΝΠϧͷಡΈࠐΈ • $-*ϑϥάͷϋϯυϦϯά
$POGJH.FUIPE
$POGJH.FUIPE
4FUVQ.FUIPE • ϝΠϯϧʔϓલͷॳظԽॲཧ • ΫϥΠΞϯτͷଓͱ͔͜͜Ͱߦ͏
4FUVQ.FUIPE
3VO.FUIPE • ϝΠϯϧʔϓ • σʔλͷऩू • 1VCMJTIFSͷૹ৴ • γάφϧϋϯυϦϯάͳͲMJCCFBUʹ͓ͤ
3VO.FUIPE
None
$MFBOVQ.FUIPE • 3VOΛൈ͚ͨޙʹݺΕΔ • ֬อͨ͠Ϧιʔεͷ։์ͳͲΛߦ͏
4UPQ.FUIPE • 3VOͰγάφϧΛड͚ΔͱݺΕΔ • ϝΠϯϧʔϓΛࢭΊΔҝͷॲཧΛߦ͏ • ىಈͨ͠HPSPVUJOFͷఀࢭ
EFWUPPMT • JOEFYUFNQMBUFHFOFSBUPS • EPDVNFOUHFOFSBUPS • CFBUQBDLFS • %PDLFSΛͬͯDSPTTDPNQJMJOHͱQBDLBHJOH EFCɾSQNɾXJOɾPTY
• ,JCBOB%BTICPBSEFYQPSUJNQPSU • FUD
Kafka Consumer Log Store Kafka Broker Kafka Broker in_tail GCE
VPN elasticsearch elasticsearch kibana fluentd ߏ
,JCBOB%BTICPBSE
·ͱΊ • (PͷܰྔɾߴɾϙʔλϏϦςΟ#FBUͷΑ ͏ͳ"HFOUʹඇৗʹϚον͢Δ • &MBTUJDTFBSDI ,JCBOBͷڥ͕͋Ε͙͢ ʹͰ࢝ΊΒΕΔ͓खܰ͞ • ڞ௨෦Λ͏·͘Γग़͠γϯϓϧʹอͬͯ
͍ΔͨΊɺಠࣗͷ#FBUΛ࡞Δͷ͕؆୯
·ͱΊ • LBGLBCFBU044Խ༧ఆͰ͢ • ؒʹ߹͍·ͤΜͰͨ͠N @@ N
(P($1Λͬͨ։ൃ͕͍ͨ͠ਓʂ