Tweet
Tweet

Slide 1

Slide 1 text

1 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. 5SJHMBW %BUB%SJWFO8PSLqPX https://github.com/triglav-dataflow

Slide 2

Slide 2 text

2 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. 044 https://github.com/triglav-dataflow ˞υΩϡϝϯτ͸গʑ͓଴ͪԼ͍͞

Slide 3

Slide 3 text

Importdb MySQL HDFS Vertica Medjed 01:00 06:00 Argus 10:00 3 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. τϦάϥϑͱ͸ ❌Α͋͘ΔࠔΓ͝ͱ • σʔλۦಈͰͭͳ͕Γͷͳ͍πʔϧʹͭͳ͕ΓΛ࣋ ͨͤͯϫʔΫϑϩʔܗ੒Λࢧԉ͢Δπʔϧ • ࣌ؒࢦఆͰσʔλϑϩʔʹͭͳ͕ΓΛ͍࣋ͨͤͯΔ • ࠶࣮ߦͨ࣌͠ʹ࿈ܞ͞Εͳ͍ ɾТɾʆ

Slide 4

Slide 4 text

Importdb MySQL HDFS Vertica Medjed 01:00 Argus Triglav Monitor Event Monitor Event 4 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. τϦάϥϑͱ͸ ⭕ରॲ๏ • σʔλۦಈͰͭͳ͕Γͷͳ͍πʔϧʹͭͳ͕ΓΛ࣋ ͨͤͯϫʔΫϑϩʔܗ੒Λࢧԉ͢Δπʔϧ

Slide 5

Slide 5 text

5 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Α͋͘ΔϫʔΫϑϩʔπʔϧͰ͸μϝʁ • શͯͷπʔϧΛ࣮ߦϗετͰ࣮ߦͰ͖Δલఏ • ྫ શೖΓ+FOLJOTεϨʔϒ • ݹ ͍ʹ͑͠ ͷπʔϧͷଘࡏ • ࣮ߦϗετΛҠͤͳ͍ Ҡͨ͘͠ͳ͍ • 31$ "1* Λ࣋ͨͳ͍ • +FOLJOTಉ࢜ͷ࿈ܞ ❌ࠔΓ͝ͱ %JHEBH -VJHJ +FOLJOTͳͲ৭ʑ͋Δ

Slide 6

Slide 6 text

5SJHMBWΞʔΩςΫνϟ ొ৔ਓ෺ 6 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. DB agent consumer Triglav Core (Broker) 2. send message 1. polling message 1. watch every minute • .FTTBHF2VFVF • "HFOU 1SPEVDFS • ετϨʔδΛ؂ࢹ • $POTVNFS • ΠϕϯτΛऔಘͯ͠δϣϒ࣮ߦ • QVMMܕ • $PSF #SPLFS • ΠϕϯτΛ؅ཧ • QVMM͞Εͯ΋͙͢ʹΠϕϯτ͸ ফ͑ͳ͍ σϑΥϧτ೔อ࣋

Slide 7

Slide 7 text

HDFS Vertica BigQuery Triglav agent agent agent Medjed Jenkins Argus Consumer Agent (Producer) Copyright (C) DeNA Co.,Ltd. All Rights Reserved. 5SJHMBWΞʔΩςΫνϟ 7 • 5SJHMBWΛ)VCʹͨ͠ΠϕϯτۦಈΞʔΩςΫνϟ • $POTVNFS͸5SJHMBWʹ͚ͩରԠ͢Ε͹ྑ͍

Slide 8

Slide 8 text

4XBHHFS 8 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • 5SJHMBWͷ"1*࢓༷Λ4XBHHFSͰهड़ • USJHMBWDMJFOUSVCZɺUSJHMBWDMJFOUKBWB Λࣗಈੜ੒͍ͯ͠Δ RESTful APIͷ࢓༷Λهड़͢Δ࢓༷ͱͦͷπʔϧ܈

Slide 9

Slide 9 text

USJHMBWBHFOUIEGT KSVCZ 9 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • ؂ࢹର৅ͷ)%'4ύεΛ5SJHMBWίΞ͔Βऔಘ • ຖ෼؂ࢹ͢Δ • MTM3QBUIUP:NE • σϑΥϧτͰ͸೔෼ • )BEPPQͳΒ*/PUJGZ"1*Λར༻͠ ͯΠϕϯτۦಈͰߋ৽৘ใΛऔΕΔ • લճͷ࠷৽࣌ؒͱߋ৽࣌ؒΛൺֱͯ͠ɺ৽ͨʹσʔ λ͕ߋ৽͞Ε͍ͯΕ͹TFOENFTTBHF

Slide 10

Slide 10 text

USJHMBWBHFOUWFSUJDB DSVCZ 10 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • جຊతʹ͸USJHMBWBHFOUIEGTͱಉ͡ • ςʔϒϧͷߦ͕͍࣋ͬͯΔFQPDI৘ใΛར༻ • લճͷ࠷৽࣌ؒͱߋ৽࣌ؒΛൺֱͯ͠ɺ৽ͨʹσʔ λ͕ߋ৽͞Ε͍ͯΕ͹ύʔςΟγϣϯ୯ҐͰ·ͱΊ ͯTFOENFTTBHF

Slide 11

Slide 11 text

KFOLJOTUSJHHFSQPMMJOH@USJHMBW 11 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • +FOLJOTϓϥάΠϯ • 5SJHMBWʹQPMMJOHͯ͠δϣϒ࣮ߦ

Slide 12

Slide 12 text

12 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • .Z42-Ͱ.FTTBHF2VFVF࣮૷ ΧϥϜ໊ આ໌ ิ଍ id ϝοηʔδID resource_uri σʔλϦιʔεͷURI ྫ) hdfs://path/to/%Y-%m-%d MQͷτϐ οΫʹ૬౰ resource_unit daily or hourly resource_timezone λΠϜκʔϯ ex) +09:00 resource_time Ϧιʔεߋ৽࣌ؒ 4&-&$5GSPNNFTTBHFTXIFSFJE \PGGTFU^"/%SFTPVSDF@VSJ\VSJ^ 5SJHMBW࣮૷ৄࡉ

Slide 13

Slide 13 text

5SJHMBWύϑΥʔϚϯε 13 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • USJHMBWBHFOU IEGTcWFSUJDB • ຖ෼؂ࢹ͢Δ͕ສఔ౓͸໰୊ͳ͠ • 4FOENFTTBHF͸සൟʹൃੜ͠ͳ͍ͨΊ ΄ͱΜͲؾʹͳΒͳ͍ • IPVSMZϦιʔεͰ΋೔ʹ௨ৗճ • ςʔϒϧ͋ͬͯ΋ ೔ఔ౓ • 'FUDINFTTBHF͸ຖ෼ߦΘΕΔ͕ɺ ΞϓϦαʔό΍NZTRMͷεϨʔϒΛ૿΍͢ ͳͲͰεέʔϧΞ΢τ༰қ

Slide 14

Slide 14 text

'VUVSF8PSL 14 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • ґଘͷՄࢹԽ • 'SPOU6*ͷ࡞੒ • ϩΰ • USJHMBWDPOTVNFSDMJ • Πϕϯτ͕དྷͨΒҾ਺ͷίϚϯυΛ࣮ߦ