Tweet
Tweet

Slide 1

Slide 1 text

Katz Ueno (@katzueno) JAWS-UG ໊ݹ԰, 2023.7.11 How I ended up with CloudWatch (+α) after using several services 1 ॳ৺ऀͷਓͰ΋ɺ
 ͦ͏ͯͳ͍ਓ΋ CloudWatch Λ࢖ͬͯΈΑ͏+α

Slide 2

Slide 2 text

ࠓ೔ͷݴ͍͍ͨ͜ͱ My bottom lines of the day • ଞࣾͷΦϒβόϏϦςΟαʔϏε͸ศར • ͖ͪΜͱઃܭ͠ͳ͍ͱ௧͍໨ʹૺ͏͓͔͔ۚ͠Δ • CloudWatch ϝτϦΫεɺϩάɺAthena Ͱ݁ߏ͍͚Δ

Slide 3

Slide 3 text

͍Ζ͍ΖαʔϏεࢼ͚ͨ͠ΕͲ ·ͣ CloudWatch ࢖͏΂͔

Slide 4

Slide 4 text

ΧϨʔ 🍛 Ϛϥιϯ🏃݄ؒ400km, ंதധ, ϥΠϒ഑৴, ಈը੍࡞ ΞϚνϡΞແઢ4ڃ (1993) / TOEIC 990 (2013) / AWS SAA (2017~) CloudFront, S3, IVS, CloudWatch 4 Katz Ueno (Χπ - ্໺উ೭) Concrete CMS ࢖͍ CMS ͱΠϯϑϥ΍ͬͯ·͢ SNS: Twitter @katzueno / Instagram @katzueno & @katzcurry / Blog: katzueno.com ίϛϡχςΟ Cncrete CMS Japan / CoderDojo ඌு / JAWS UG ໊ݹ԰ ޷͖ͳ৯΂෺: झຯ: ࢿ֨: ޷͖ͳαʔϏε:

Slide 5

Slide 5 text

AWS SAP ߹֨ ઌ݄, 3೥ӽ͠ • ࢁԼ͞ΜͷຊΛങͬͯ߹֨
 ͠·ͨ͠ʂ

Slide 6

Slide 6 text

ࠓ೔ͷΞδΣϯμ Today’s agenda • CloudWatch ·Ͱͷวྺ • ֤αʔϏεͰͷར఺ɾܽ఺
 (+ࣦഊஊ) • CloudWatch (+Athena) Ͱ͍͍ ͡ΌΜ • ͓·͚: SRE ࢧ෦ͷ঺հ Stable Di ff usion Ͱ “Agenda” Ͱੜ੒

Slide 7

Slide 7 text

Katz ͷΦϒβϏϦςΟͷεϖοΫ • 3rd Party • Mackerel (2014~): ͘͞Β VPS Ͱར༻։࢝ • New Relic (2016~): AWSར༻ऀ͸ແྉͷ࣌୅ • Datadog (2021~): • AWS (CloudWatch ྺ): 2016~ • EC2, RDS த৺ɻίϯςφ͸ະͩɻ ͭ·Γɺ͠ΐ΅͍Ͱ͢ 😅 😅 😅 ඞཁʹԠͯ͡ ࠷௿ݶ͔͠ ׆༻ͯ͠ͳ͍ 😅😅😅😅😅

Slide 8

Slide 8 text

ग़ձฤ ͸ͯͳ: mackerel

Slide 9

Slide 9 text

mackarel ͸ͯͳࣾ • 2014೥αʔϏεΠϯ • ࠃ࢈؂ࢹπʔϧ • ݟ΍͍͢؅ཧը໘

Slide 10

Slide 10 text

mackarel ͸ͯͳࣾ Katz ͸ • CMS ͷ೔ຊޠαΠτ
 (͘͞ΒVPS)΍ Nextmode ͞ΜҊ݅ • AWS σϏϡʔલ͔Βར༻։࢝

Slide 11

Slide 11 text

mackarel ͸ͯͳࣾ Katz ͸ • ղܾ͔ͨͬͨ͠՝୊
 ͦ΋ͦ΋؂ࢹ͕Ͱ͖ͳ͔ͬͨ

Slide 12

Slide 12 text

mackarel ͸ͯͳࣾ ࢖ͬͯݟ͑ͨ՝୊ • ΞϓϦͷτϨʔεݟ͑ͳ͍ • ΞϓϦ (PHP) ΍ MySQL ͷΫΤϦ ύϑΥʔϚϯεݟ͑ͳ͍ • ΤϥʔτϨʔεͰ͖ͳ͍

Slide 13

Slide 13 text

࣍ͷεςοϓฤ New Relic

Slide 14

Slide 14 text

New Relic • 2008೥૑ۀ • AWS αʔόʔ಺ͩͬͨΒແྉ ͩͬͨ • APM (Application Performance Management) αʔϏε͕͋Δʂ

Slide 15

Slide 15 text

New Relic Katz ͸ • ͓٬͞Μ CMS Ҋ݅ • ղܾ͔ͨͬͨ͠՝୊ 
 PHP Τϥʔɾॲཧ଎౓ͷ؂ࢹ • 2015~2021 ͰϝΠϯͰ࢖͏

Slide 16

Slide 16 text

New Relic ྑ͍ͱ͜Ζ • APM ศརʂ • Τϥʔ΋Θ͔Δʂ • Ͳ͜Ͱॲཧʹ͔͔࣌ؒͬͯΔ͔Θ͔ Δʂ ෆຬͳͱ͜Ζ • ΤʔδΣϯτ͕ॏ͍ (ޙड़) • ྉۚߴ͍

Slide 17

Slide 17 text

New Relic ৽ͨͳ՝୊ (౰࣌) • AWSϢʔβʔແྉͩͬͨྉ͕ۚ༗ྉʹͳͬͨ (APM+Πϯϑϥ) • (New Relic ͕ѱ͘ͳ͍͚ΕͲ) ͓ۚΛ෷͏෼ͷରՁ͋Δʁ • ෼ੳ͢Δ͕࣌ؒແ͍ • agent ͷύϑΥʔϚϯεϩε͕େ͖͔ͬͨ (ޙड़)

Slide 18

Slide 18 text

࣍ͦ͜͸ʂ Datadog

Slide 19

Slide 19 text

Datadog • 2010೥૑ۀ • New Relic ʹൺ΂ͯαϙʔτϓ ϥοτϑΥʔϜ͕ଟ͍ (Docker ΍ k8s )

Slide 20

Slide 20 text

Datadog Katz ͸ • ͓٬͞Μ CMS Ҋ݅ • ղܾ͔ͨͬͨ͠՝୊ 
 New Relic ͷ஗͍ΤʔδΣϯτ • 2021~ ࢖͍࢝ΊΔ

Slide 21

Slide 21 text

Datadog ྑ͍ͱ͜Ζ • ؅ཧը໘͕Έ΍͍͢ (૬ੑ?) • ΤʔδΣϯτ͕͍ܰ (?) • Τϥʔݕࡧ͕؆୯ ෆຬͳͱ͜Ζ • ͦΕͰ΋ΤʔδΣϯτ͕ॏ͍ (ޙड़) • Data Injection Ϧϛοτग़ݱ😢

Slide 22

Slide 22 text

Datadog ྑ͍ͱ͜Ζ • ؅ཧը໘͕Έ΍͍͢ (૬ੑ?) • ΤʔδΣϯτ͕͍ܰ (?) • Τϥʔݕࡧ͕؆୯ ෆຬͳͱ͜Ζ • ͦΕͰ΋ΤʔδΣϯτ͕ॏ͍ (ޙड़) • Data Injection Ϧϛοτग़ݱ😢

Slide 23

Slide 23 text

Datadog ྑ͍ͱ͜Ζ • ؅ཧը໘͕Έ΍͍͢ (૬ੑ?) • ΤʔδΣϯτ͕͍ܰ (?) • Τϥʔݕࡧ͕؆୯ ෆຬͳͱ͜Ζ • ͦΕͰ΋ΤʔδΣϯτ͕ॏ͍ (ޙड़) • Data Injection Ϧϛοτग़ݱ😢

Slide 24

Slide 24 text

Datadog ྑ͍ͱ͜Ζ • ؅ཧը໘͕Έ΍͍͢ (૬ੑ?) • ΤʔδΣϯτ͕͍ܰ (?) • Τϥʔݕࡧ͕؆୯ ෆຬͳͱ͜Ζ • ͦΕͰ΋ΤʔδΣϯτ͕ॏ͍ (ޙड़) • Data Injection Ϧϛοτग़ݱ😢 27TB!!! ͝ΊΜͳ͞ʔʔʔ͍

Slide 25

Slide 25 text

Datadog ৽ͨͳ՝୊ (౰࣌) • Agent ͷύϑΥʔϚϯε௿Լ͸ಉͩͬͨ͡ • σʔλΛૹΓ͗ͯͯ͢ɺ؂ࢹϩάग़ྗΧοτ͠ͳͪ͘Ό͍͚ͳ͔ͬͨ😅 • (MySQL ͷϩά͚ͩ O ff ʹ͢ΔઃఆͳͲ͸؆୯ʹग़དྷ͚ͨͲɾɾɾ

Slide 26

Slide 26 text

࠷ޙͷखஈ !? Zabbix !? লུɾɾɾ

Slide 27

Slide 27 text

ΤʔδΣϯτͷ ύϑΥʔϚϯεͷ ࣦഊʹ͍ͭͯ

Slide 28

Slide 28 text

New Relic & Datadog ΤʔδΣϯτͷࣦഊ ྆αʔϏεͱ΋ѱ͘ͳ͍ (?) • ΤʔδΣϯτͷϦιʔεফඅΛܭࢉͯ͠ͳ͔ͬͨ😅 • ෛՙςετ͸ΤʔδΣϯτΠϯετʔϧલ • ຊ൪ӡ༻։࢝ͨ͠ޙʹΠϯετʔϧ • CPU, ϝϞϦΛ࠷େݶʹ׆͔ͨ͢ΊͷνϡʔχϯάΛ͍ͯͨ͠ • ຊ൪Ͱӡ༻։࢝ͨ͠ࡍʹɺෛՙΦʔόʔͰΤϥʔଟൃ😭😭😭😭😭 • ͡Ό͋ࢼݧʁ → ։ൃɾݕূ؀ڥͷͨΊʹ՝ۚΛ૿΍ͨ͘͠ͳ͍

Slide 29

Slide 29 text

New Relic & Datadog ΤʔδΣϯτͷࣦഊ ྆αʔϏεͱ΋ѱ͘ͳ͍ (?) • ΤʔδΣϯτͷෛՙΛߟ͑ͯαʔόʔνϡʔχϯά͢΂͖ͩͬͨ • αʔϏε΁ͷϩάͷ༰ྔͱͦͷίετΛߟ͑ͳ͚Ε͹͍͚ͳ͔ͬͨ

Slide 30

Slide 30 text

αʔυόʔςΟʔ؂ࢹͷ·ͱΊ

Slide 31

Slide 31 text

αʔυύʔςΟʔ؂ࢹπʔϧͷ·ͱΊ ௒ݸਓతͳέʔε • ͓͕͔͔ۚΔ • ෷͍ͬͯΔ͓ۚ෼ɺӡ༻Ͱ͖͍ͯΔ͔ෆ҆
 ʢΤϥʔൃੜ࣌͸͘͢͝ศརʂʣ • ։ൃɾݕূ؀ڥͰ࢖͑ͳ͍ (අ༻) • ΤʔδΣϯτෛՙˍσʔλྔ໰୊

Slide 32

Slide 32 text

ͦ͜Ͱؾ͍ͮͨ ɾɾɾ

Slide 33

Slide 33 text

αʔυύʔςΟʔ؂ࢹπʔϧͷ·ͱΊ ௒ݸਓతͳέʔε • ͓͕͔͔ۚΔ • ෷͍ͬͯΔ͓ۚ෼ɺӡ༻Ͱ͖͍ͯΔ͔ෆ҆
 ʢΤϥʔൃੜ࣌͸͘͢͝ศརʂʣ • ։ൃɾݕূ؀ڥͰ࢖͑ͳ͍ (අ༻) • ΤʔδΣϯτෛՙˍσʔλྔ໰୊

Slide 34

Slide 34 text

αʔυύʔςΟʔ؂ࢹπʔϧͷ·ͱΊ ௒ݸਓతͳέʔε • ͓͕͔͔ۚΔ • ෷͍ͬͯΔ͓ۚ෼ɺӡ༻Ͱ͖͍ͯΔ͔ෆ҆
 ʢΤϥʔൃੜ࣌͸͘͢͝ศརʂʣ • ։ൃɾݕূ؀ڥͰ࢖͑ͳ͍ (අ༻) • ΤʔδΣϯτෛՙˍσʔλྔ໰୊ CloudWatch ͸ैྔ՝ۚ Amazon Linux ܥ ΤʔδΣϯτΠϯετʔϧ͠΍͍͢ ςετ͠΍͍͢ }

Slide 35

Slide 35 text

CloudWatch ͋Δ͡ΌΜ + Athena

Slide 36

Slide 36 text

ͱΓ͓֮͋͑ͣ͑ͯ͜͏ • CloudWatch Metrics & Dashboard • CloudWatch Logs • CloudWatch Alarm • Athena (ELB & CloudFront ͱ͔)

Slide 37

Slide 37 text

CloudWatch Ͱे෼͔΋ʁ ࠷ॳଆඋΘͬͯΔ͠ • গͳ͍අ༻Ͱ࢝ΊΒΕΔɿैྔ՝ۚ • ͦ΋ͦ΋ඪ४ͰϝτϦΫε͍ͭͯΔ • CloudWatch ϩά -> ΦʔτεέʔϦϯάΠϯελϯ εͳͲͩͱݕࡧָ͕ • (Amazon Linux) ͸ΤʔδΣϯτΠϯετʔϧؾܰ

Slide 38

Slide 38 text

CloudWatch ͷ͍͍ͱ͜Ζ

Slide 39

Slide 39 text

CloudWatch ͷར఺: Metrics • ඪ४ϝτϦΫε͕ͱΓ͍͋͑ͣͭͯ͘Δ • ϝϞϦͳͲ͸ΧελϜϝτϦΫεͰରԠ

Slide 40

Slide 40 text

CloudWatch ͷར఺: Metrics • ౦ژϦʔδϣϯ

Slide 41

Slide 41 text

CloudWatch ͷར఺: Dashboard • Dashboard ͰϝτϦΫεΛΧελϚΠζͰ͖Δ ࣾ಺νʔϜͰ ϝτϦΫεͷ ϓϦηοτΛ ڞ༗Ͱ͖Δ

Slide 42

Slide 42 text

CloudWatch ͷར఺: Dashboard • Dashboard ͰϝτϦΫεΛΧελϚΠζͰ͖Δ ࣾ಺νʔϜͰ ϝτϦΫεͷ ϓϦηοτΛ ڞ༗Ͱ͖Δ ࠓ CDK Ͱ؆୯ʹ࡞ΕΔΑ͏ʹܭըத

Slide 43

Slide 43 text

CloudWatch ͷར఺: Logs • ϩάςΩετ৘ใΛૹΔ • ݕࡧ͕Ͱ͖Δ • άϧʔϓԽͰ͖Δ • Φʔτεέʔϧάϧʔϓͷϩά Λ·ͱΊͯݕࡧͳͲͰ͖Δ

Slide 44

Slide 44 text

CloudWatch ͷར఺: Logs • ϩάͷอ࣋ظؒΛ͏·͍͜ͱௐ੔ͯ͠஋ஈઅ໿ (S3 ΑΓ΋ׂߴͳͷͰ)

Slide 45

Slide 45 text

CloudWatch ͷར఺: Alarms • ͦ΋ͦ΋ΦʔτεέʔϦϯάͯ͠Δ࣌ͱ͔ઃఆ͠·͢ • ͡Ό͋ Alarm ઃఆͯ͠Ξϥʔτग़͠·͠ΐ͏

Slide 46

Slide 46 text

CloudWatch ͷར఺: Alarms • ౦ژϦʔδϣϯͷ஋ஈ

Slide 47

Slide 47 text

Athena ࢖ͬͯ ELB & CloudFront ϩάղੳ

Slide 48

Slide 48 text

CloudFront & ELB ϩάͷ՝୊ • gz ϑΝΠϧͨ͘͞ΜͰɺͲ͔͜ʹಥͬࠐΜͰղੳ͕ඞཁ

Slide 49

Slide 49 text

ͦ͜Ͱʂʂ Athena ࢖ͬͯ ELB & CloudFront ϩάղੳ

Slide 50

Slide 50 text

Amazon Athena ͱ͸ • S3 ͷதͷσʔλΛSQLͷΑ͏ʹݕࡧͰ͖ΔαʔϏε

Slide 51

Slide 51 text

AWS Athena Ͱϩάղੳʂ • AWS υΩϡϝϯτͰςʔϒϧఆ͕ٛ͢Ͱʹެ։͞Εͯ·͢ʂ

Slide 52

Slide 52 text

AWS Athena Ͱϩάղੳʂ • AWS υΩϡϝϯτͰςʔϒϧఆ͕ٛ͢Ͱʹެ։͞Εͯ·͢ʂ

Slide 53

Slide 53 text

AWS Athena Ͱϩάղੳʂ • SQL ΫΤϦͷྫ:
 
 7/11 ʹ CloudFront Distribution ʹΞΫηεͨ͠ IP ΞυϨεͷଟ͍ॱΛݕࡧ͢Δ SQL SELECT client_ip, x_edge_location, count(*) AS num FROM cf_log WHERE request_date = '2023-07-11' GROUP BY 1,2 ORDER BY count(*) DESC LIMIT 100;

Slide 54

Slide 54 text

AWS Athena Ͱϩάղੳʂ • SQL ΫΤϦͷྫ:
 
 7/11 ʹ CloudFront Distribution ʹΞΫηεͨ͠ IP ΞυϨεͷଟ͍ॱΛݕࡧ͢Δ SQL SELECT client_ip, x_edge_location, count(*) AS num FROM cf_log WHERE request_date = '2023-07-11' GROUP BY 1,2 ORDER BY count(*) DESC LIMIT 100;

Slide 55

Slide 55 text

AWS Athena Ͱϩάղੳʂ • ౦ژϦʔδϣϯͷྉۚ σʔλ͕ଟ͍ͱ༹͓͕͚ۚΔʂʂ S3 ϥΠϑαΠΫϧઃఆͯ͠

Slide 56

Slide 56 text

ࠓ೔ͷݴ͍͍ͨ͜ͱ My bottom lines of the day • ଞࣾͷΦϒβόϏϦςΟαʔϏε͸ศར • ͖ͪΜͱઃܭ͠ͳ͍ͱ௧͍໨ʹૺ͏͓͔͔ۚ͠Δ • CloudWatch ϝτϦΫεɺϩάɺAthena Ͱ݁ߏ͍͚Δ • ͏·͘ӡ༻͢Ε͹અ໿Ͱ͖Δʂ • ؅ཧը໘ઃܭ΋ CDK ͱ͔Ͱ࡞ΕΔ (ܭըத)

Slide 57

Slide 57 text

ࠓ೔ͷݴ͍͍ͨ͜ͱ My bottom lines of the day • ͻͱ·ͣ΍ͬͯΈΔ͜ͱ • CloudWatch Dashboard Λ࡞ͬͯΈΔ • CloudWatch Logs ʹϩάΛૹͬͯݕࡧͰ͖ΔΑ͏ʹͯ͠ΈΔ

Slide 58

Slide 58 text

͓·͚ ͕࣌ؒ༨ΔΒ͍͠ͷͰ

Slide 59

Slide 59 text

YouTube νϟϯωϧݟͯ·͔͢ʁ JAWS UG

Slide 60

Slide 60 text

JAWS UG SRE ࢧ෦ #6 ͕໘ന͔ͬͨ https://www.youtube.com/watch?v=9w8U-jd7DqI&t=1456s

Slide 61

Slide 61 text

JAWS UG SRE ࢧ෦ #6 2023/5/29 0:00:00 Φʔϓχϯά SREࢧ෦ ΈͷΔΜ @minorun365 0:05:05 ΞΠεϒϨΠΫLT SREࢧ෦ ;Δ΍ @saramune 0:09:14 ʮΤϯδχΞϦϯάͰӡ༻Λվળ͢ΔʯͨΊͷAmazon CloudWatch׆༻ ΞϚκϯ΢ΣϒαʔϏεδϟ ύϯ߹ಉձࣾɹ௡ڷ ޫ໌ @misstuu 0:39:50 શAWSΤϯδχΞʹ๋͙ɺCloudWatch ઃܭɾӡ༻ ދͷר ΞϥΠͱ΢ϚΧπ @msy78 @HorseVictory 1:03:40 Amazon CloudWatch SyntheticsͰ࢝ΊΔ֎ܗ؂ࢹʢԾʣ SREϗʔϧσΟϯάε 佂ా ߁ฏ @boy_hap 1:12:50 CloudWatchͰόϨΔʮ܅ɺ࢓ࣄதʹyoʓtube؍ͯͨΑͶʁʯ Ϋϥ΢υηϯτϦοΫ ֯த ݯଠ࿠ @kado__gen 1:21:43 CloudWatchෳ߹ΞϥʔϜͰELBͷ5XXΛ͍͍ײ͡ʹݕ஌͠Α͏ͱͨ͠Β͏·͍͔͘ͳ͔ͬͨ࿩ k.goto @365_step_tech 1:32:36 CloudWatchΛݸਓ৘ใอޢͷ໡఺ʹ͠ͳ͍ͨΊʹ גࣜձࣾΩϟϯαʔεΩϟϯ อࡔ কฏ @shoheihosaka •

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

ΧϨʔ 🍛 Ϛϥιϯ🏃݄ؒ400km, ंதധ, ϥΠϒ഑৴, ಈը੍࡞ ΞϚνϡΞແઢ4ڃ (1993) / TOEIC 990 (2013) / AWS SAA (2017~) CloudFront, S3, IVS, CloudWatch 63 Katz Ueno (Χπ - ্໺উ೭) Concrete CMS ࢖͍ CMS ͱΠϯϑϥ΍ͬͯ·͢ SNS: Twitter @katzueno / Instagram @katzueno & @katzcurry / Blog: katzueno.com ίϛϡχςΟ Cncrete CMS Japan / CoderDojo ඌு / JAWS UG ໊ݹ԰ ޷͖ͳ৯΂෺: झຯ: ࢿ֨: ޷͖ͳαʔϏε: