Upgrade to Pro — share decks privately, control downloads, hide ads and more …

これが Cloud Native な セキュリティログ分析だ (仮)

これが Cloud Native な セキュリティログ分析だ (仮)

Cookpad techconf 2018のLTで講演した資料です

Masayoshi Mizutani

February 11, 2018
Tweet

More Decks by Masayoshi Mizutani

Other Decks in Technology

Transcript

  1. CookpadͰ΋ؤு͍ͬͯ·͕͢ 3 ηΩϡϦςΟ୲౰͸ؾʹͳΔ͜ͱ͕ͨ͘͞Μ ϥϯαϜ΢ΣΞ 42-*OKFUJPO EBZ"UUBDL 944 όϥϚΩܕϝʔϧ߈ܸ $43' %SJWFCZ%PXOMPBE߈ܸ

    ඪతܕ߈ܸ ηΩϡϦςΟઃఆϛε ύεϫʔυϦετܕ߈ܸ Ϋϥ΢υ্ͷΠϯελϯε 1BB4 ࣾһ͕ར༻͢Δ1$ ΦϑΟεωοτϫʔΫ ֎෦ͱͷϝʔϧ ۀ຿γεςϜ "DUJWF%JSFDUPSZ ΦϯϥΠϯετϨʔδ कΒͳ͚Ε͹ͳΒͳ͍෺ͨͪ ߈ܸͯ͘͠Δऀͨͪ Ϣʔβͷ৘ใ
  2. 5

  3. 7

  4. SIEMΛ࠶ߟ͢Δ 1. ϩάͷऩूͱਖ਼نԽ 2. ϩάͷ෼ੳΤϯδϯ 3. ϩάͷอ؅ɾݕࡧ 4. Ξϥʔτͷൃใ͓Αͼ؅ཧ 16

    AWSͷαʔϏε + α Ͱ୅ସͯ͠ΈΔ → CloudWatch + Fluentd + S3 + Lambdaແ → Lambdaແ → S3 + Graylog + AWS Athenaແ → PagerDuty + GHEແ
  5. ͜͏ͳͬͨ 17 ͜Ε͕ Cloud Native ͳηΩϡϦςΟϩά෼ੳͩ (Ծ) ϩάϑΝΠϧઃஔͷ ΠϕϯτΛݕग़ ʜ

    ʜ Lambda Lambda Lambda Kinesis Stream Kinesis Stream S3 S3 S3 S3 Athena ϧʔϧΛ༻͍ͨ Ξϥʔτͷݕ஌ Ξϥʔτͷൃใ ϩάͷม׵ EC2 Elasticsearch Service ߴ଎ͰΠϯλϥΫςΟϒͳ ୹ظతϩάͷݕࡧ ௕ظؒʹΘͨΔ ϩάͷݕࡧ GHE PagerDuty Slack Ξϥʔτͷൃใɾ؅ཧ Kinesis Stream Ξϥʔτͷௐࠪ CloudWatch Logs/Event EC2 instances ͦͷଞϓϩμΫτ
  6. AWSαʔϏεʴαͰ࡞ΔηΩϡϦςΟ؂ࢹΞʔΩςΫνϟ (1/4) ϩάϑΝΠϧઃஔͷ ΠϕϯτΛݕग़ ʜ ʜ Lambda Lambda Lambda Kinesis

    Stream Kinesis Stream S3 S3 S3 S3 Athena ϧʔϧΛ༻͍ͨ Ξϥʔτͷݕ஌ Ξϥʔτͷൃใ ϩάͷม׵ EC2 Elasticsearch Service ߴ଎ͰΠϯλϥΫςΟϒͳ ୹ظతϩάͷݕࡧ ௕ظؒʹΘͨΔ ϩάͷݕࡧ GHE PagerDuty Slack Ξϥʔτͷൃใɾ؅ཧ Kinesis Stream Ξϥʔτͷௐࠪ CloudWatch Logs/Event EC2 instances ͦͷଞϓϩμΫτ ϩάͷऩूͱਖ਼نԽ 18 w ͱʹ͔͘ϩά͸·ͣ4ʹ౤͛ࠐΉ w ॲཧͷલஈͰ·ͣอଘ͢Δ͜ͱͰϩά ଛࣦͷࣄނΛ๷͙ w ϑϧϚωʔδυ͔ͭεέʔϥϒϧͳ4 Λ࢖͏͜ͱͰӡ༻ෛՙΛԼ͛Δ w ϩάͷਖ਼نԽ͸-BNCEBΛ࢖ͬͯίʔ υ؅ཧˍςετΛॻ͍ͯ$*
  7. AWSαʔϏεʴαͰ࡞ΔηΩϡϦςΟ؂ࢹΞʔΩςΫνϟ (2/4) ϩάϑΝΠϧઃஔͷ ΠϕϯτΛݕग़ ʜ Lambda Lambda Lambda Kinesis Stream

    Kinesis Stream S3 S3 S3 S3 Athena ϧʔϧΛ༻͍ͨ Ξϥʔτͷݕ஌ Ξϥʔτͷൃใ ϩάͷม׵ EC2 Elasticsearch Service ߴ଎ͰΠϯλϥΫςΟϒͳ ୹ظతϩάͷݕࡧ ௕ظؒʹΘͨΔ ϩάͷݕࡧ GHE PagerDuty Slack Ξϥʔτͷൃใɾ؅ཧ Kinesis Stream Ξϥʔτͷௐࠪ CloudWatch Logs/Event EC2 instances ͦͷଞϓϩμΫτ ϩάͷ෼ੳΤϯδϯ 19 w 4ʹϑΝΠϧ͕อଘ͞ΕͨΠϕϯτΛ͏͚ ͱͬͨ-BNCEB'VODUJPO͕ϩάΛಡΈ ग़ͯ͠෼ੳ w ෼ੳΤϯδϯΛૄ݁߹ʹઃܭ͢Δ͜ͱͰɺ ༰қʹεέʔϧΞ΢τ w ݕ஌ΤϯδϯͷBUUBDIEFUBDI΋ࣗ༝ࣗࡏ w ΋ͪΖΜϧʔϧ΋ίʔυ؅ཧˍςετ
  8. AWSαʔϏεʴαͰ࡞ΔηΩϡϦςΟ؂ࢹΞʔΩςΫνϟ (3/4) ϩάϑΝΠϧઃஔͷ ΠϕϯτΛݕग़ ʜ Lambda Lambda Lambda Kinesis Stream

    Kinesis Stream S3 S3 S3 S3 Athena ϧʔϧΛ༻͍ͨ Ξϥʔτͷݕ஌ Ξϥʔτͷൃใ ϩάͷม׵ EC2 Elasticsearch Service ߴ଎ͰΠϯλϥΫςΟϒͳ ୹ظతϩάͷݕࡧ ௕ظؒʹΘͨΔ ϩάͷݕࡧ GHE PagerDuty Slack Ξϥʔτͷൃใɾ؅ཧ Kinesis Stream Ξϥʔτͷௐࠪ CloudWatch Logs/Event EC2 instances ͦͷଞϓϩμΫτ ϩάͷอ؅ɾݕࡧ 20 w ࠷ॳʹΞϥʔτ͕ൃใ͞Εͨࡍ ͷॳಈ૞ࠪ͸(SBZMPHͰࣗ༝ ʹݕࡧඞཁͳϩάͷநग़ w Ξϥʔτ͕௕ظؒʹ౉͍ͬͯΔ ͜ͱ͕Θ͔ͬͨ৔߹ɺ"UIFOB Λ࢖ͬͯେྔͷϩάΛݕࡧͯ͠ աڈʹḪͬͨਝ଎ͳௐ͕ࠪՄೳ
  9. AWSαʔϏεʴαͰ࡞ΔηΩϡϦςΟ؂ࢹΞʔΩςΫνϟ (4/4) ϩάϑΝΠϧઃஔͷ ΠϕϯτΛݕग़ ʜ Lambda Lambda Lambda Kinesis Stream

    Kinesis Stream S3 S3 S3 S3 Athena ϧʔϧΛ༻͍ͨ Ξϥʔτͷݕ஌ Ξϥʔτͷൃใ ϩάͷม׵ EC2 Elasticsearch Service ߴ଎ͰΠϯλϥΫςΟϒͳ ୹ظతϩάͷݕࡧ ௕ظؒʹΘͨΔ ϩάͷݕࡧ GHE PagerDuty Slack Ξϥʔτͷൃใɾ؅ཧ Kinesis Stream Ξϥʔτͷௐࠪ CloudWatch Logs/Event EC2 instances ͦͷଞϓϩμΫτ 21 Ξϥʔτͷൃใ͓Αͼ؅ཧ w ΦϖϨʔλ΁ͷ௨஌͸1BHFS%VUZ 4MBDLΛར༻ w ൃใͱಉ࣌ʹ(JUIVC&OUFSQSJTFʹJTTVFΛν έοτͱͯ͠࡞੒͠ɺΞϥʔτͷௐࠪʹؔ͢Δ৘ใΛ ूத؅ཧ w ͞Βʹࣗಈతʹ-BNCEB͕Ξϥʔτͷؔ࿈৘ใΛ֎ ෦αΠτ͔Βݕࡧ͠νέοτʹ͓·ͱΊ͢Δ
  10. ʢ࠶ܝʣ 22 ͜Ε͕ Cloud Native ͳηΩϡϦςΟϩά෼ੳͩ (Ծ) ϩάϑΝΠϧઃஔͷ ΠϕϯτΛݕग़ ʜ

    ʜ Lambda Lambda Lambda Kinesis Stream Kinesis Stream S3 S3 S3 S3 Athena ϧʔϧΛ༻͍ͨ Ξϥʔτͷݕ஌ Ξϥʔτͷൃใ ϩάͷม׵ EC2 Elasticsearch Service ߴ଎ͰΠϯλϥΫςΟϒͳ ୹ظతϩάͷݕࡧ ௕ظؒʹΘͨΔ ϩάͷݕࡧ GHE PagerDuty Slack Ξϥʔτͷൃใɾ؅ཧ Kinesis Stream Ξϥʔτͷௐࠪ CloudWatch Logs/Event EC2 instances ͦͷଞϓϩμΫτ