Tweet
Tweet

Slide 1

Slide 1 text

A survey of anomaly detec2on methodologies for web system ΫοΫύουגࣜձࣾ ΠϯϑϥετϥΫνϟʔ෦ SRE άϧʔϓ ٢઒ ཽଠ ( @rrreeeyyy ) Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 1

Slide 2

Slide 2 text

!SSSFFFZZZ !SSSFFFZZZ IUUQTSSSFFFZZZDPN :PTIJLBXB3ZPUB Me • Yoshikawa Ryota ( @rrreeeyyy [reɪ] ) • ΫοΫύουגࣜձࣾ (2017/01 ʙ) • ΠϯϑϥετϥΫνϟʔ෦ SRE άϧʔϓ • ڵຯྖҬ • ϞχλϦϯάɾ࣌ܥྻσʔλϕʔε • ෼ࢄγεςϜɾϩʔυόϥϯα • झຯ • League of Legends, ΀Α΀Α, FF14 Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 2

Slide 3

Slide 3 text

໨࣍ • എܠͱ໨త • ௐࠪख๏ • ௐࠪ಺༰ͱ݁Ռ • ߟ࡯ • ·ͱΊ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 3

Slide 4

Slide 4 text

എܠͱ໨త Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 4

Slide 5

Slide 5 text

എܠ • SRE ۀ຿ͷओ໨తͷͰ͋Δ Web γεςϜͷ৴པੑͷ޲্ • ো֐ͷະવͷ༧๷ͱฏۉ෮چ࣌ؒͷ୹ॖ͕ख๏ͷҰͭ • γεςϜͷϝτϦΫεͷҟৗΛਖ਼֬ʹૣ͘ݟ͚ͭΔඞཁ͕͋Δ • ༷ʑͳϝτϦΫεΛߴղ૾ͰऔಘͰ͖Δج൫͕੔͖ͬͯͨ • ҰํͰݱ৔ϨϕϧͰͷϝτϦΫεͷղੳʹର͢ΔΞϓϩʔν͸ະͩʹශऑ • ᮢ஋ϕʔεͷҟৗݕ஌ɾ୯७ͳճؼʹΑΔҟৗݕ஌ͳͲ... • Ұํɺ࣌ܥྻσʔλղੳࣗମͷ෼໺͸͋Δఔ౓੒ख़͍ͯ͠Δ/੒௕͠ଓ͚͍ͯΔ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 5

Slide 6

Slide 6 text

໨త • Web γεςϜͷϝτϦΫεͷಛੑΛ෼ੳ͢Δ • ಛੑ͔ΒͲ͏͍ͬͨղੳΛ͢Δͷ͕޲͍͍ͯΔͷ͔ௐࠪ͢Δ • ଞ෼໺ͷ࣌ܥྻσʔλղੳͷख๏ͷྑ͍ͱ͜ΖΛऔΓೖΕ͍ͨ • (ݸਓతͳཧ༝) αʔόϝτϦΫεͱ͍͏࣌ܥྻσʔλΛ৮͖ͬͯͨ • ҰํͰ࣌ܥྻσʔλղੳͷΑ͏ͳֶज़తͳΞϓϩʔνʹແ಴ணͩͬͨ • ࠓճͷΑ͏ͳௐࠪΛ௨ͯ࣌͡ܥྻσʔλղੳʹৄ͘͠ͳ͍͖͍ͬͯͨ • SRE ͷΑ͏ͳ৬छͷਓશମͰ࣌ܥྻσʔλղੳͷ͜ͱΛߟ͍͑ͨ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 6

Slide 7

Slide 7 text

ௐࠪख๏ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 7

Slide 8

Slide 8 text

ௐࠪख๏ • Web αʔϏεʹݱΕΔϝτϦΫεΛز͔ͭͷύλʔϯʹ෼ྨ • චऀͷܦݧ΍Ұൠతͳ Web αʔϏεͷ܏޲͔Β • Web αʔϏεʹݱΕΔҟৗʹ͍ͭͯز͔ͭͷύλʔϯʹ෼ྨ • ͪ͜Β΋චऀͷܦݧ΍Ұൠతͳ Web αʔϏεͷ܏޲͔Β • ҟৗݕ஌ͷख๏ࣗମͷௐࠪ࿦จͷख๏ͱরΒ͠߹ΘͤͯΈΔ • ෼ྨͨ͠ϝτϦΫεʹద͍ͯ͠Δ͔ • ൃੜ͠͏Δҟৗ͕ݕ஌Ͱ͖ͦ͏͔ • Ͳ͏͍ͬͨछྨͷϝτϦΫεʹ͸Ͳ͏͍ͬͨख๏ͷҟৗݕ஌͕޲͍͍ͯΔ͔·ͱΊΔ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 8

Slide 9

Slide 9 text

ௐࠪ಺༰ͱ݁Ռ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 9

Slide 10

Slide 10 text

Web γεςϜͷϝτϦΫεύλʔϯ • Web γεςϜʹݱΕΔϝτϦΫεΛҎԼͷΑ͏ʹ෼ྨ͢Δ • Web αʔϏεʹΑ͘ݱΕΔมԽͷ֎తཁҼผ • ϢʔβͷΞΫηεʹ൐ͬͯେ͖͘มԽ͢ΔϝτϦΫε • ಛఆͷॲཧ͕࣮ߦ͞Ε͍ͯΔࡍʹେ͖͘มԽ͢ΔϝτϦΫε • ΞΫηε΍ಛఆͷॲཧʹؔΘΒͣมԽ͢ΔϝτϦΫε • ΋ͪΖΜ֎తཁҼ͕ 1 ϝτϦΫεͰෳ߹తʹൃੜ͢Δύλʔϯ΋͋Δ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 10

Slide 11

Slide 11 text

ϢʔβͷΞΫηεʹ൐ͬͯେ͖͘มԽ͢ΔϝτϦΫε • ྫ͑͹ҎԼͷΑ͏ͳϝτϦΫεͳͲ͕ߟ͑ΒΕΔ • LB/Web/DB αʔόͷ CPU ࢖༻཰, Traffic, αʔό୆਺... ͳͲ • αʔϏεͷಛੑʹ΋ΑΔ͕ҎԼͷมಈཁҼΛ࣋ͭ͜ͱ͕ଟ͍ • ܏޲มಈɾ॥؀มಈɾقઅมಈɾෆنଇมಈ • ͜Ε͸ͦ΋ͦ΋ϢʔβͷΞΫηεʹมಈཁҼ͕͋ΔͨΊ • Web αʔϏεʹݱΕΔϝτϦΫεͷதͰ࠷΋ෳࡶʹมԽ͢Δͱߟ͑ΒΕΔ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 11

Slide 12

Slide 12 text

όονॲཧ͕࣮ߦ͞Ε͍ͯΔࡍʹେ͖͘มԽ͢ΔϝτϦΫε • ྫ͑͹ҎԼͷΑ͏ͳϝτϦΫε͕ߟ͑ΒΕΔ • όοναʔόͷ CPU ࢖༻཰, Traffic, ... ͳͲ • αʔϏεͷಛੑʹ΋ΑΔ͕ҎԼͷมಈཁҼΛ࣋ͭ͜ͱ͕ଟ͍ • ܏޲มಈɾ॥؀มಈɾෆنଇมಈ • ಛఆͷपظͰॲཧ͕ߦΘΕΔͨΊ • όον࣌ؒதͷΈ஫ࢹ͢Ε͹Α͍͕όονຖʹಛੑ͕ҟͳΔՄೳੑ΋͋Δ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 12

Slide 13

Slide 13 text

ΞΫηε΍όονॲཧʹؔΘΒͣมԽ͢ΔϝτϦΫε • ྫ͑͹ҎԼͷΑ͏ͳϝτϦΫε͕ߟ͑ΒΕΔ • σΟεΫ࢖༻ྔ, Swap ࢖༻ྔ, ... • σΟεΫ࢖༻ྔ͸αʔόͷಛੑʹΑΔ • αʔϏεͷಛੑʹ΋ΑΔ͕ҎԼͷมಈཁҼΛ࣋ͭ͜ͱ͕ଟ͍ • ܏޲มಈɾෆنଇมಈ • ௕ظ܏޲͕Θ͔Ε͹Α͍ɾେ͖͘มԽ͢Δͱͦ΋ͦ΋ҟৗͳ͜ͱ͕ଟ͍ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 13

Slide 14

Slide 14 text

Web γεςϜʹى͜ΓಘΔো֐ύλʔϯ • ো֐࣌ʹϝτϦΫε͕औΓಘΔಈ͖ͱͯ͠ҎԼͷύλʔϯʹ͍ͭͯߟ͑Δ • Spike/Falling ύλʔϯ • Flapping/Stopping ύλʔϯ • Satula1on ύλʔϯ • ϢʔβମݧʹӨڹ͕͋Δ࣌͸ԿΕ͔ͷϝτϦΫε্͕هͷಈ͖Λ͢ΔͱԾఆ • ো֐தʹϝτϦΫεʹมԽ͕ى͖ͳ͍৔߹͸औಘର৅Λݟ௚͢ඞཁ͕͋Δ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 14

Slide 15

Slide 15 text

Web γεςϜʹى͜ΓಘΔো֐ύλʔϯ • Spike/Falling ύλʔϯ • ϝτϦΫε͕ٸܹʹ্ঢ΋͘͠͸Լ߱͢ΔΑ͏ͳύλʔϯ • ओʹ֎Ε஋΍มԽ఺͕෼͔Δ͜ͱͰݕग़Մೳ • Flapping/Stopping ύλʔϯ • ϝτϦΫε͕ٸʹ୹͍ظؒͰৼಈ࢝͠ΊΔΑ͏ͳύλʔϯ • पظӡಈ͕୹͘ͳ͚ͬͨͩͳͲͩͱ֎Ε஋͚ͩͰ͸ݕग़͕೉͍͠ • पظӡಈ͍ͯͨ͠ϝτϦΫε͕ٸʹҰఆ࣌ؒपظӡಈΛ΍ΊΔΑ͏ͳύλʔϯ • पظӡಈͷִ͕ؒ௕͘ͳ͚ͬͨͩͳͲͩͱ֎Ε஋͚ͩͰ͸ݕग़͕೉͍͠ • Satura2on ύλʔϯ • ϝτϦΫε͕ʢԿΒ͔ͷ੍ݶͰʣҰఆ্ݶͰఀ଺ͯ͠͠·͏ύλʔϯ • ୯ҰͷϝτϦΫε͚ͩͰ͸ͲͷཁҼͰఀ଺͍ͯ͠Δͷ͔ಛఆ͕ࠔ೉ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 15

Slide 16

Slide 16 text

Spike ύλʔϯͷྫ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 16

Slide 17

Slide 17 text

Flapping ύλʔϯͷྫ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 17

Slide 18

Slide 18 text

Stopping ύλʔϯͷྫ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 18

Slide 19

Slide 19 text

Satura&on ύλʔϯͷྫ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 19

Slide 20

Slide 20 text

Web γεςϜͷϝτϦΫεɾো֐ಛੑ • ଟ͘ͷϝτϦΫεͰਓؒͷ࡞ۀͳͲʹΑΔසൟʹൃੜ͢Δෆنଇมಈ͕͋Δ • ྫ͑͹σϓϩΠɾΦϖϨʔγϣϯͳͲ ... • ςϨϏ์ө΍ SNS Ͱͷ֦ࢄͳͲʹΑΔΞΫηεεύΠΫ͕͋Δ৔߹͕͋Δ • ֶशʹ͔͔Δ࣌ؒ͸͋Δఔ౓௕ͯ͘΋໰୊ͳ͍͕ҟৗͷݕ஌͸ߴ଎Ͱ͋Δඞཁ͕͋Δ • ෳ਺ͷϝτϦΫεΛಉ࣌ʹ֬ೝͯ͠ҟৗɾਖ਼ৗΛ൑அ͢Δ͜ͱ΋ଟ͍ • جຊతʹ͸ط஌ͷҟৗσʔλ͸े෼Ͱͳ͍͜ͱ͕ଟ͍ • ༩͑ΒΕͨਖ਼ৗσʔλΛϞσϦϯά͢ΔܗͰҟৗݕ஌Λߦ͏ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 20

Slide 21

Slide 21 text

ҟৗݕ஌ͷख๏ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 21

Slide 22

Slide 22 text

ҟৗݕ஌ͷख๏ • ΞΫηε΍όονॲཧʹؔΘΒͣมԽ͢ΔϝτϦΫε • قઅੑ΍पظੑʹΑΔมಈ͕খ͍͞ͷͰ୯७ͳճؼϞσϧͰຬ଍Ͱ͖Δέʔε͕ଟͦ͏ • όονॲཧ͕࣮ߦ͞Ε͍ͯΔࡍʹେ͖͘มԽ͢ΔϝτϦΫε • ಛʹपظੑʹΑΔมಈ͕େ͖͍ͷͰपظੑͷഉআΛߦ͏ • पظੑͷഉআΛߦͬͯ͠·͑͹୯७ͳճؼϞσϧͰຬ଍Ͱ͖Δέʔε͕ଟͦ͏ • ϢʔβͷΞΫηεʹ൐ͬͯେ͖͘มԽ͢ΔϝτϦΫε • ༷ʑͳมಈཁҼ͕͋ΔͨΊ୯७ͳճؼϞσϧʹམͱ͢ͷ͸ࠔ೉ͦ͏ • ϐʔΫ࣌ɾฏৗ࣌ͳͲͰLOF, GMMͳͲΛ༻͍Δέʔε΋͋Δ • Spike ύλʔϯͳͲ͸े෼ʹݕ஌Ͱ͖Δέʔε͕ଟͦ͏ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 22

Slide 23

Slide 23 text

ҟৗݕ஌ͷख๏ • Ұํ Flapping/Stopping, Satura1on ύλʔϯʹ͍ͭͯͷݕ஌͸೉͍͠ • ͍ΘΏΔ contextual anomaly ͱݺ͹ΕΔͰ͋Ζ͏΋ͷ • [1] ͷ (2008 ೥) ஈ֊Ͱ͸ contextual anomaly ͷจݙ͸গͳ͔ͬͨͱ͋Δ • ͜͏͍ͬͨ contextual anomaly ʹؔ͢Δݚڀ͸ۙ೥ਐΜͰ͖͍ͯΔ • CNN ͳͲΛ༻͍ͨੜ੒Ϟσϧͷֶश [6] • ࣌ܥྻσʔλΛ੒෼ʹ෼ղͯ͠ۂઢϑΟοςΟϯάͷ໰୊ͱͯ͠ղ͘෺ [7] Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 23

Slide 24

Slide 24 text

ߟ࡯ • ͜Ε·Ͱ୯ҰతʹʮαʔόϝτϦΫεʯͱ͍ͯͨ͠΋ͷΛߋʹࡉ͔͘෼ྨͨ͠ • ࡉ͔͘෼ྨ͢Δ͜ͱͰ୯७ͳϞσϧͰఆࣜԽͰ͖ͦ͏ͳϝτϦΫεͷଘࡏΛࣔͨ͠ • ҰํͰΦϖϨʔγϣϯͳͲʹΑΔෆنଇมಈΛͲ͏ѻ͏͔͸ཁݕ౼ • ࣮ࡍͷো֐ମݧ͔Βো֐ύλʔϯΛز͔ͭʹ෼ྨͨ͠ • ࣮ࡍͷ࿦จͱԠ౴ੑͷ੍໿͔Βબ୒Ͱ͖Δҟৗݕ஌ͷख๏Λݕ౼ͨ͠ • ίϯςΩετΛؚΉҟৗݕ஌ͳͲʹؔͯ͠͸Ҿ͖ଓ͖ௐ͕ࠪඞཁ • ఆࣜԽ͕े෼ͱ͍͑ΔΘ͚Ͱ͸ͳ͍ͷͰࠓޙ΋ϝτϦΫε΍ো֐ύλʔϯΛݕ౼͢Δඞཁ • ߋʹ࣮ࡍʹ࢖͏ͨΊʹ͸ֶश଎౓΍ਪ࿦଎౓ɾෛՙʹ͍ͭͯ΋ݕ౼͕ඞཁ • શͯͷϝτϦΫεʹରͯ͠ҟৗݕ஌Λߦ͏ΑΓ steady-state ͷΑ͏ͳ΋ͷΛఆٛͨ͠΄͏͕ྑ͍͔ݕ౼ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 24

Slide 25

Slide 25 text

·ͱΊ • ͍͔ͭ͘ͷαʔόϝτϦΫεʹ͍ͭͯ·ͱΊ܏޲Λࣔͨ͠ • αʔόϝτϦΫεͷ܏޲͔Βҟৗݕ஌ͷख๏ͷબ୒ʹ͍ͭͯߟ࡯ͨ͠ • ো֐࣌ʹݱΕΔϝτϦΫεͷڍಈʹ͍ͭͯ·ͱΊ܏޲Λࣔͨ͠ • ݱΕͦ͏ͳҟৗΛݕ஌͢ΔͨΊʹͲͷΑ͏ͳख๏Λ༻͍Ε͹Α͍͔ߟ࡯ͨ͠ • ௐࠪΛ௨ͯ࣌͡ܥྻσʔλղੳʹ͔͚ᷮͩৄ͘͠ͳͬͨ • ࠓճௐࠪͨ͠෼Ͱ΋଍Γͳ͍ͷͰҾ͖ଓ͖ۙ୅ͷख๏ʹ͍ͭͯ΋ௐ͕ࠪඞཁ • ಛʹ contextual/collec-ve anomaly detec-on ʹ͍ͭͯ Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 25

Slide 26

Slide 26 text

ࢀߟจݙ 1. CHANDOLA, Varun; BANERJEE, Arindam; KUMAR, Vipin. Anomaly detecCon: A survey. ACM compuCng surveys (CSUR), 2009, 41.3: 15. 2. KITAGAWA, G. Introducing to Time Series Modeling, Chapman & Hall. 2010. 3. HOCHENBAUM, Jordan; VALLIS, Owen S.; KEJARIWAL, Arun. AutomaCc anomaly detecCon in the cloud via staCsCcal learning. arXiv preprint arXiv:1704.07706, 2017. 4. ISLAM, Md Rafiqul, et al. A Comprehensive Survey of Time Series Anomaly DetecCon in Online Social Network Data. InternaConal Journal of Computer ApplicaCons, 2017, 180.3: 13-22. 5. HARVEY, Andrew C.; PETERS, Simon. EsCmaCon procedures for structural Cme series models. Journal of ForecasCng, 1990, 9.2: 89-108. 6. LAPTEV, Nikolay, et al. Time-series extreme event forecasCng with neural networks at uber. In: InternaConal Conference on Machine Learning. 2017. p. 1-5. 7. TAYLOR, Sean J.; LETHAM, Benjamin. ForecasCng at scale. The American StaCsCcian, 2018, 72.1: 37-45. Web System Architecture ݚڀձ (2018/11/17) | Yoshikawa Ryota ( @rrreeeyyy ) 26