Upgrade to Pro — share decks privately, control downloads, hide ads and more …

横断SREの立ち上げと、AWSセキュリティへの取り組みの軌跡

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers.
Avatar for adachi.ryo adachi.ryo
January 25, 2025

 横断SREの立ち上げと、AWSセキュリティへの取り組みの軌跡

Avatar for adachi.ryo

adachi.ryo

January 25, 2025
Tweet

More Decks by adachi.ryo

Other Decks in Technology

Transcript

  1. 2

  2. 3

  3. 4

  4. 5 ࣗݾ঺հ ҆ୡ ྋ(@adachin0817) ɾϑΝΠϯσΟ(ג) / ϓϩμΫτ։ൃ෦/Senior SRE ɾBlog: blog.adachin.me/wiki.adachin.me

    ɾTechBull(ٕज़ίϛϡχςΟ) techbull.cloud ɹɾʮSRE/ۦ͚ग़͠ΤϯδχΞͷϝϯλϦϯά ྦྷܭ300໊↑ʯ ɾ͔ͭͯ͸OSS൛VulsͷίϯτϦϏϡʔλʔ΍Πϕϯτओ࠵ͳͲ ɾ89೥ੜ·Εɺ౦ژ౎଍ཱ۠ग़਎Ͱ࡛ۄݝय़೔෦ࢢ͕஍ݩ ɾ࠺ͱϑϨϯνϒϧυοάͷࣂ͍ओͰ΋͋Δ
  5. 6

  6. ԣஅSREνʔϜ্ཱͪ͛ʹ͍ͭͯ • ڈ೥·ͰόοΫΤϯυνʔϜ͕ΠϯϑϥΛ୲౰͍ͯͨ͠ ◦ αʔϏεͷ֦େʹ൐͍ɺόοΫΤϯυνʔϜͷϦιʔε͕ෆ଍ ◦ SREతͳվળ͕े෼ʹߦ͑ͳ͍ঢ়گ͕ଓ͍͍ͯͨͨΊɺԣஅSREνʔϜ͕ൃ଍ ◦ ଐਓԽ͠ͳ͍Α͏શαʔϏεΛର৅ʹ೺Ѳ͍ͯ͘͠ ◦

    ݩʑόοΫΤϯυͰΠϯϑϥ୲౰͍ͯͨ͠ϝϯόʔ͸Embedded SREͱͯ͠ࢀՃ • ԣஅSREνʔϜ͸ݱࡏ4໊Ͱ׆ಈ ◦ ϦʔμʔɺγχΞ໊̎ɺδϡχΞ1໊ ◦ Embedded SRE͸4໊ ◦ GitHubͷΧϯόϯํࣜͰλεΫ؅ཧ͓ͯ͠Γɺேձ͸30෼Ͱ׬݁ ◦ Findy Team+Λ׆༻͠ɺຖिKPTΛߦ͍ͬͯΔ 8
  7. ԣஅSREνʔϜͷҐஔ͚ͮͱϛογϣϯ • SREͷଘࡏҙٛ ◦ SRE͸ಓΛ࡞ΔͨΊʹଘࡏ͢Δ(։ൃͷεϐʔυͱ҆શੑΛཱ྆) ◦ ϦεΫΛड͚ೖΕɺ؅ཧ͢Δ(ো֐ͷϦεΫΛ࠷খݶʹ཈͑ͭͭɺޮ཰తͳӡ༻Λ໨ࢦ͢) ◦ SLOΛܭଌ͢Δ(৴པੑͷόϥϯεΛऔΔͨΊͷج४Λࡦఆ) ◦

    τΠϧͷ࡟ݮͱࣗಈԽ(Ձ஋ͷߴ͍ۀ຿ʹूதͰ͖Δ؀ڥΛఏڙ) ◦ ϓϩμΫτͷࢧԉ(։ൃεϐʔυͱ৴པੑͷόϥϯεΛอͭͨΊʹٕज़ࢧԉ) ◦ ηΩϡϦςΟͷՄࢹԽͱڧԽ (જࡏతͳϦεΫΛൃݟ͠ɺγεςϜΛΑΓ҆શͳঢ়ଶʹอͭ) • ୹ظϛογϣϯ ◦ ʮϑΝΠϯσΟͷࣄۀ੒௕Λࢧ͑ΔͨΊʹɺSRE૊৫ͷ͋Γํͷཱ֬ʯ • தظϛογϣϯ ◦ ʮࣾһશһ͕ࣄۀ੒௕ʹूதͰ͖ΔΑ͏ͳ࢓૊ΈΛߏங͠ɺ҆શʹఏڙʯ 9
  8. ԣஅSREνʔϜ͕ҙ͍ࣝͯ͠Δ͜ͱ • ։ൃνʔϜͱͷ৴པؔ܎ͷߏங ◦ Ͱ͖ͨͯͷνʔϜ͸Φʔϓϯͳίϛϡχέʔγϣϯ͕ॏཁ ◦ ໰୊΍ෆ҆Λڞ༗͠΍͍ؔ͢܎ੑΛங͖ɺ৴པੑ͸ձ࿩͔Β࢝ΊΔ͜ͱ • Embedded SREͱͷڠಇ

    ◦ Embedded SREͱີʹ࿈ܞ͠ɺݱ৔Ͱͷ՝୊ײΛٵ্͍͛Δ • จԽͷৢ੒ͱνʔϜͷҰମײ ◦ ຖि༵ۚ೔ʹৼΓฦΓձΛߦ͍ɺKibelaͰશΤϯδχΞʹڞ༗͠ೝ஌౓Λ޲্ ◦ νʔϜͷ੒Ռ΍՝୊ղܾʹ͸ɺ࣭ͷߴ͍ίϛϡχέʔγϣϯ͕ॏཁ ◦ εϐʔυײΛॏࢹͭͭ͠ɺཱࣗͨ͠νʔϜͷҭ੒Λ໨ࢦ͢ 10
  9. AWSηΩϡϦςΟʹऔΓ૊ΉࡍͷୈҰา • ηΩϡϦςΟ਍அͱݱঢ়೺Ѳ ◦ ઃఆϛε΍੬ऑͳϦιʔεΛՄࢹԽ͠ɺ༏ઌͯ͠ରॲ͢΂͖ϦεΫΛಛఆ • ηΩϡϦςΟ؂ࢹͱΞϥʔτͷઃఆ ◦ ҟৗͳϩάΠϯͷݕग़ɺڴҖݕग़ ◦

    TrivyʹΑΔطଘݕग़ͱ৽نߏஙͰͷ࣮૷ • ηΩϡϦςΟϩάͷՄࢹԽ ◦ AWS WAFɺCloudTrailɺGuardDutyͷঢ়ଶΛઃఆ͢Δ͚ͩͳ͘ՄࢹԽ͢Δ • ηΩϡϦςΟڭҭͱҙࣝ޲্ ◦ νʔϜશମͷηΩϡϦςΟҙࣝΛߴΊΔ͜ͱ͕େ੾ ◦ ࠷৽ͷڴҖ΍ରࡦํ๏Λڞ༗͢Δ৔Λઃ͚Δ 16
  10. AWSηΩϡϦςΟڧԽʹ͓͚ΔπʔϧબఆͷΞϓϩʔν • ݩʑ͸AWS Security HubͰ࣮૷Λ͢Δ༧ఆͩͬͨ ◦ AWS OrganizationsͰ؅ཧ͍ͯ͠ΔͨΊɺ਺ेݸҎ্ͷΫϩεΞΧ΢ϯτ͕ଘࡏ ◦ σʔλ෼ੳͰ͸GCP΋ར༻͍ͯ͠ΔͨΊɺҰݩ؅ཧ͕Ͱ͖ͣɺҰ؏ੑ͕อͯͳ͍

    ◦ ༷ʑͳαʔϏε͕ಈ࡞͢ΔͨΊɺෳࡶʹͳΓ΍͘͢ɺίετ͕ߴ͘ͳΔ܏޲͕͋Δ ◦ ૢ࡞ੑɺධՁ݁Ռͷࢹೝੑ͕ѱ͘ɺτϦΞʔδͷूܭʹ޻਺͕͔͔Δ ◦ ରԠํ๏ͷυΩϡϝϯτ͕ӳޠͩΒ͚ͰΤϯδχΞ͕ૉૣ͘ରԠͰ͖ͳ͍ • ༷ʑͳAWSηΩϡϦςΟπʔϧΛࢼݧಋೖ ◦ ػೳ΍ૢ࡞ੑɺίετύϑΥʔϚϯεͷ؍఺͔Βൺֱݕ౼ ◦ Shisho Cloud͕࠷΋ཁ݅ʹద߹͠ɺಋೖͷܾఆʹʂ 17
  11. Shisho Cloudͷ࢖͍΍͢͞ • Simple is the best ◦ ϚϧνΫϥ΢υͷҰݩ؅ཧ ◦

    ηΩϡϦςΟઐ໳஌͕ࣝͳͯ͘΋ରԠՄೳ ◦ ϦεΫͷଈ࣌ՄࢹԽ ◦ ೔ຊޠରԠͷஸೡͳϨϙʔτ ◦ ಋೖͷ༰қ͞ͱݕग़݁Ռͷ଎͞ ◦ े෼ʹ४උ͞ΕͯΔϚωʔδυϙϦγʔ ◦ ΧελϚΠζੑͷߴ͞ ◦ ͳΜͱ͍ͬͯ΋Ձ͕͍֨҆ 19
  12. Shisho Cloudͷӡ༻ϙΠϯτ • ηΩϡϦςΟΨΠυϥΠϯϙϦγʔͷ࡞੒ ◦ ࢛൒ظ͝ͱʹ༏ઌ౓ͷߴ͍IssueΛ͢΂ͯରԠ͢Δ͜ͱΛ໨ඪʹઃఆ • ηΩϡϦςΟ؂ࢹͱΞϥʔτͷઃఆ ◦ ֤ϓϩδΣΫτʹઐ༻ͷSlackνϟϯωϧΛ࡞੒͠ɺؔ܎ऀΛר͖ࠐΉ࢓૊ΈΛߏங

    ◦ Embedded SRE޲͚ʹ৘ใڞ༗ͷ৔Λઃ͚Δ ◦ τϦΞʔδ͞ΕͨΞϥʔτ͸͢΂ͯରԠ͢Δඞཁ͸ͳ͘ɺ༏ઌ౓ΛݟۃΊͯରԠ • ηΩϡϦςΟରԠͷܗ֚ԽΛ๷͗ɺνʔϜͷཱࣗΛଅਐ ◦ ηΩϡϦςΟରԠͷܗ֚ԽΛ๷͗ɺνʔϜͷཱࣗΛଅਐ 20
  13. Shisho Cloudͷӡ༻՝୊ • ৽نΠϯϑϥߏங࣌ʹຖճΞϥʔτ͕ޡݕ஌͞ΕΔ໰୊͕ൃੜ ◦ ฐࣾͰ͸શͯͷΠϯϑϥߏஙΛTerraformͰ؅ཧ͍ͯ͠Δ ◦ ෛՙςετ؀ڥ΍৽نΠϯϑϥ؀ڥ͕ςϯϓϨʔτԽ͞Ε͍ͯͳ͍ ◦ ؀ڥ͝ͱʹηΩϡϦςΟϙϦγʔ͕౷Ұ͞Ε͍ͯͳ͍

    ◦ ຖճSlack௨஌͕ଟൃ͠ɺϊΠζͰຒ·ͬͯ͠·͏ ◦ ࠓޙ͸ϙϦγʔʹ४ڌͨ͠ڞ௨ςϯϓϨʔτԽΛݕ౼͍ͯ͠Δ • ॏཁͳ௨஌͕ຒ΋Εͯ͠·͏ ◦ SlackͰϝϯγϣϯ෇͖ͰCriticalɺHighΛ௨஌Ͱ͖ΔΑ͏ʹ 21
  14. ηΩϡϦςΟϩάج൫ • Amazon Security Lake ◦ Shisho Cloud͚ͩͰ͸AWS಺ͰϦΞϧʹԿ͕ ى͖͍ͯΔ͔ݟ͑ʹ͍͘ ◦

    CloudTrailɺWAFɺVPC Flow Logɺ Route53(DNS Query)Λର৅ʹՄࢹԽ͍ͯ͠Δ ◦ Security LakeͰ؆୯ʹҰݩ؅ཧ͕Մೳ ◦ ݄਺ສԁఔ౓Ͱ࣮૷Մೳ ◦ Amazon Managed GrafanaͰμογϡϘʔυԽ ◦ ࠓޙ͸SQLͷ݁Ռ͔ΒBedrockͰ෼ੳ༧ఆ 24
  15. ·ͱΊ/ࠓޙͷల๬ • AWSηΩϡϦςΟपΓ͸ՄࢹԽͯ͠ܧଓతʹ෼ੳͱରࡦΛ͢Δ͜ͱ • ϫʔΫϑϩʔͷΧελϚΠζΛ׆͔͖͠Εͯͳ͍ ◦ ඞཁʹԠͯ͡૊৫ݻ༗ͷϙϦγʔΛઃఆ͠ɺӡ༻ʹద༻͢Δ ◦ AWSΞΧ΢ϯτͷ൑ఆج४Λ໌֬Խ͠ɺCritical,HighϨϕϧͷݕ஌࿙ΕΛ๷ࢭ ◦

    طଘΞϥʔτͷվमͱ୨Է͠ • ηΩϡϦςΟϩάج൫ͷ෼ੳ ◦ CloudTrailɺWAFɺVPC Flow LogsɺRoute53(DNS Query) ΫΤϦϩά ◦ Security LakeΛ༻͍ͨՄࢹԽͱ෼ੳ ◦ μογϡϘʔυͷΧελϚΠζ΍ఆظతͳৼΓฦΓΛ࣮ࢪ͠ɺӡ༻վળΛਤΔ ◦ ηΩϡϦςΟΦϒβʔόϏϦςΟ 27
  16. 28

  17. 29

  18. 30