OpenTelemetryを活用したObservability入門 / Introduction to Observability with OpenTelemetry

Slide 1

Slide 1 text

©Fusic Co., Ltd. CONFIDENTIAL 1 OpenTelemetryΛ׆༻ͨ͠ Observabilityೖ໳ 2025.03.23 ਗ਼Ո࢙࿠ @seike460 PHPerKaigi 2025

Slide 2

Slide 2 text

©Fusic Co., Ltd. 2 ࣗݾ঺հ ͸͡Ίʹ ਗ਼Ո ࢙࿠ (@seike460) S H I R O S E I K E גࣜձࣾFusicɹϓϦϯγύϧΤϯδχΞ/ΤόϯδΣϦετ AWS User Group Leaders AWS Community Builder Serverless - ίϛϡχςΟ - JAWS-UG Fukuoka - Fukuoka.php - Fukuoka.go - Serverless Meetup Fukuoka - Cloudflare Meetup Fukuoka - JP_Stripes Fukuoka - JBUG Fukuoka

Slide 3

Slide 3 text

©Fusic Co., Ltd. 3 CONTENTS ໨࣍ 1. Observability 2. OpenTelemetry 3. PHP/LaravelͰͷOpenTelemetryಋೖ

Slide 4

Slide 4 text

©Fusic Co., Ltd. 4 Observability 1

Slide 5

Slide 5 text

©Fusic Co., Ltd. 5 Observabilityͷجૅ (Metrics/Traces/Logs) ʮObservability (Մ؍ଌੑ)ʯͱ͸ͳΜͳͷ͔ʁ γεςϜ಺෦ͷঢ়ଶΛ֎෦͔Β؍ଌɾ೺Ѳ͢ΔೳྗΛࢦ͠ɺ ΦϒβʔόϏϦςΟͷࡾຊபͱͯ͠Α͘஌ΒΕΔͷ͕ϩάɾϝτϦΫεɾτϨʔεͱ͍ͬͨςϨϝτϦͰ͢ • ϩά (Logs) • ྫ֎΍Πϕϯτͷৄࡉͳه࿥ • ϝτϦΫε (Metrics) • CPU࢖༻཰΍ϦΫΤετ਺ɺϨΠςϯγʔͳͲͷఆྔతࢦඪ • τϨʔε (Traces) • ୯Ұɺ·ͨ͸ϚΠΫϩαʔϏεؒΛ·͙ͨϦΫΤετܦ࿏ΛՄࢹԽ

Slide 6

Slide 6 text

©Fusic Co., Ltd. 6 MonitoringͱObservability ैདྷͷ؂ࢹʢMonitoringʣ͸ࢮ׆؂ࢹ΍Ϧιʔε؂ࢹ͕த৺ ͔͠͠ϚΠΫϩαʔϏε΍αʔόʔϨεԽ͕ਐΉதͰ͸ݪҼಛఆ͕ࠔ೉ ʹ Observability͕ॏཁͳͷ͸ɺ୯ʹʮނো͍ͯ͠Δ͜ͱʯΛ஌ΔͷͰ͸ͳ͘ɺ ʮͲ͜ͰԿ͕ى͖͍ͯΔ͔ɺͲ͏௚ͤΔ͔ʯ·Ͱਝ଎ʹ೺Ѳ͢Δ͜ͱ ઌ΄Ͳͷ3ͭͷγάφϧΛ૬ޓʹؔ࿈෇͚ͯ׆༻͢Δ͜ͱͰ؍ଌྗΛൃش͢Δ ࢀߟURLɿhttps://opentelemetry.io/docs/

Slide 7

Slide 7 text

©Fusic Co., Ltd. 7 OpenTelemetry 2

Slide 8

Slide 8 text

©Fusic Co., Ltd. 8 OpenTelemetryͱ͸Կ͔ʁ OpenTelemetry (ҎԼɺOTel) ͸ CNCF (Cloud Native Computing Foundation) ͕ओಋ͢Δ ϕϯμʔχϡʔτϥϧͳςϨϝτϦऩूج൫ • ݴޠ΍ϑϨʔϜϫʔΫΛ໰Θͳ͍౷Ұ & SDK • PHPؚΉओཁݴޠ޲͚ʹAPIΛఏڙ • ଟ༷ͳΤΫεϙʔτઌ • CloudWatch, X-Ray, Datadog, Jaeger, Prometheus ౳΁ૹ৴Մೳ • ϕϯμʔχϡʔτϥϧ • ޙ͔Βผͷ؂ࢹπʔϧʹม͑ͯ΋ɺΞϓϦܭଌ෦෼͸ͦͷ·· ݩʑ͸GoogleͷOpenCensusͱOpenTracing͕౷߹ͯ͠஀ੜ͠ɺ ݱࡏͰ͸ Ϋϥ΢υωΠςΟϒͳՄ؍ଌੑͷඪ४ ʹͳΓͭͭ͋Δ AWS΋ADOT (AWS Distro for OpenTelemetry) Λ༻ҙ͢ΔͳͲڧྗʹαϙʔτ

Slide 9

Slide 9 text

©Fusic Co., Ltd. 9 OpenTelemetryͷશମΞʔΩςΫνϟ OpenTelemetry͸ҎԼͷཁૉͰߏ੒͞Ε·͢ 1. Instrumentation Libraries • ֤ϑϨʔϜϫʔΫ/ϥΠϒϥϦʹର͢Δܭଌίʔυ 2. OpenTelemetry API/SDKʢReceiverɺProcessor΋͜͜ʹʣ • ܭଌݺͼग़͠΍εύϯੜ੒ɺϝτϦΫεૹ৴ͳͲͷ࣮૷ 3. Exporter • ऩूσʔλΛ֎෦αʔϏε΁ૹΔػೳ (OTLP, Jaeger, Prometheus ͳͲ) 4. Collector • OTelσʔλΛड͚औΓɺόονॲཧ΍ϑΟϧλɺΤΫεϙʔτΛߦ͏ 5. Backend • σʔλΛՄࢹԽɾ෼ੳ͢ΔͨΊͷϓϥοτϑΥʔϜ (Grafana΍DatadogɺNew RelicɺAWS X-Ray/CloudWatch) ΞϓϦ → (OTel SDK) → Exporter → (OTel Collector) → ϞχλϦϯάπʔϧ ͱ͍͏ྲྀΕ ϕϯμʔϩοΫΠϯΛආ͚ͳ͕Βɺ෼ࢄτϨʔε΍ϝτϦΫεɺϩάΛ౷߹؅ཧ͕Մೳʹ ࢀߟURLɿhttps://opentelemetry.io/docs/demo/architecture/

Slide 10

Slide 10 text

©Fusic Co., Ltd. 10 ܭ૷ʢΠϯετϧϝϯςʔγϣϯʣ ܭ૷ʢ͚͍ͦ͏ʣͱ͸γεςϜ΍ΞϓϦέʔγϣϯͷʮ಺෦Ͱى͖͍ͯΔ͜ͱʯΛ ଌఆɾऩू͢ΔͨΊͷ࢓૊ΈΛಋೖ͢Δ͜ͱ ࣗಈܭ૷ ΞϓϦέʔγϣϯʹ࠷খݶͷมߋΛՃ͑Δ͚ͩͰɺ πʔϧ΍ϥΠϒϥϦ͕ࣗಈతʹτϨʔε΍ϝτϦΫεΛऩूͯ͘͠ΕΔ࢓૊Έ खಈܭ૷ ։ൃऀ͕ΞϓϦέʔγϣϯͷίʔυʹରͯ͠໌ࣔతʹܭଌͷͨΊͷॲཧΛຒΊࠐΉํ๏ ࣗಈܭ૷͢Δ͜ͱͰɺOpenTelemetryͷΤίγεςϜΛ؆୯ʹར༻͢Δ͜ͱ͕Մೳ

Slide 11

Slide 11 text

©Fusic Co., Ltd. 11 ͳͥOpenTelemetryΛ࢖͏ͷ͔ʁ • Ұ౓ܭ૷Λߦ͏͜ͱͰΞʔΩςΫνϟͷมߋ͕Մೳ • Datadog, New Relic, Jaeger, CloudWatchͳͲɺޙ͔Βࣗ༝ʹ੾Γସ͑Մೳ [OTLP -> ϓϩτίϧϨϕϧ] • ෳ਺γάφϧͷҰݩతऩू • τϨʔεɾϝτϦΫεɾϩάΛڞ௨ϑΥʔϚοτͰѻ͑ΔͨΊɺ؍ଌσʔλΛ༰қʹؔ࿈෇͚ • ίϛϡχςΟͱΤίγεςϜ • CNCFࡿԼϓϩδΣΫτͱͯ͠ਐԽ͕ૣ͘ɺϕϯμʔ΋ੵۃతʹରԠɻAWS, GCP, Azure͢΂ͯαϙʔτ • Ϋϥ΢υωΠςΟϒͱͷ਌࿨ੑ • Kubernetes΍ECS/FargateͳͲ෼ࢄ؀ڥͰͷҰׅ؅ཧ (Collector) ͕༰қ • ίετͱӡ༻ෛՙͷίϯτϩʔϧ • αϯϓϦϯά΍CollectorઃఆͰແବͳσʔλΛૹΒͣɺίετ࠷దԽΛਤΕΔ ୯ͳΔ؂ࢹπʔϧͱ͸ҧ͍ɺʮܭଌͷඪ४తͳԼ஍ʯͱͯ͠ͷ໾ׂΛ୲͍ͬͯΔ఺͕େ͖͍

Slide 12

Slide 12 text

©Fusic Co., Ltd. 12 OpenTelemetryʹ Profilling͕…! • ओཁγάφϧ͕Stable • τϨʔεɺϝτϦΫεɺϩάͷ࢓༷͕҆ఆ൛ͱཱͯ֬͠ • ϓϩϑΝΠϦϯάରԠਐߦத • CPU/ϝϞϦϓϩϑΝΠϦϯάͳͲͷγάφϧ௥Ճ͕ݕ౼͞Ε͍ͯΔ • ύϑΥʔϚϯε࠷దԽ • Auto-Instrumentation࣌ͷΦʔόʔϔου࡟ݮ͕ਐΈɺେن໛ӡ༻Ͱ΋ಋೖࣄྫଟ਺ ͜Ε͕࣮ݱ͢Δͱؔ਺ϨϕϧͰͷύϑΥʔϚϯεϞχλϦϯά͕Մೳʹ…ʂʁ ࢀߟURLɿhttps://opentelemetry.io/blog/2024/state-profiling/

Slide 13

Slide 13 text

©Fusic Co., Ltd. 13 PHP/LaravelͰͷOpenTelemetryಋೖ 3

Slide 14

Slide 14 text

©Fusic Co., Ltd. 14 PHPͰOpenTelemetryΛ࢖͏ཁ݅ • PHP 8.0Ҏ্ • OTel PHP֦ுͱࣗಈܭଌ͕ඞཁ • OTel֦ு (pecl install opentelemetry) • ࣗಈܭଌΛ༗ޮʹ͢ΔC੡֦ு • Composerύοέʔδ • open-telemetry/sdk, open-telemetry/exporter-otlp, open-telemetry/opentelemetry-auto-laravel ͳͲ • पลϥΠϒϥϦ • gRPC௨৴ͳΒgrpc֦ு΍Guzzle౳͕ඞཁ

Slide 15

Slide 15 text

©Fusic Co., Ltd. 15 Laravel΁ͷOpenTelemetryࣗಈܭ૷ • ୯ҰΞϓϦέʔγϣϯͳΒ #2·ͰͰOK • ෳ਺ΞϓϦέʔγϣϯ࣌ͷ෼ࢄτϨʔεͷ৔߹#3 • ϩάͱTraceΛؔ࿈෇͚Δ৔߹ #4·Ͱ

Slide 16

Slide 16 text

©Fusic Co., Ltd. 16 खಈܭ૷΋Մೳ͕ͩɺίʔυΛϝϯςφϯε͢Δ֮ޛΛ • ໌ࣔతʹʮ͔͜͜Β͜͜·Ͱʯͱ͍͏Span(ॲཧ୯Ґ)σʔλΛऔΔ͜ͱ΋ग़དྷΔ͕ ٯʹ͜͜ͷίʔυͷ੹೚ΛऔΔඞཁ͕͋Δ

Slide 17

Slide 17 text

©Fusic Co., Ltd. 17 Jaeger - a Distributed Tracing System L L L L LL L LL L L LL L LL L L L L L L L L LL L LL L L L L L LL L L L LLL L LLL L L • Jaeger͸ɺ෼ࢄτϨʔγϯάͷͨΊͷOSS • ෳ਺ͷϚΠΫϩαʔϏεؒͷϦΫΤετΛ௥੻͠ɺॲཧͷྲྀΕ΍஗ԆՕॴΛՄࢹԽ • OpenTelemetryͱ࿈ܞͯ͠ɺτϨʔεσʔλͷऩूɾอଘɾUIදࣔ

Slide 18

Slide 18 text

©Fusic Co., Ltd. 18 Jaeger <-> Laravel L L L L LL L LL L L LL L LL L L L L L L L L LL L LL L L L L L LL L L L LLL L LLL L L main-otel-demo other-service-laravel ࠓճ͸෼ࢄτϨʔεͷ࢓૊ΈΛମݧ͢ΔͨΊʹ ̎ͭͷLaravelΛ࡞੒͍ͯ͠·͢ɻ - main-otel-demo - other-service-caravel

Slide 19

Slide 19 text

©Fusic Co., Ltd. 19 JaegerͷUI ҰཡͰ͸Traceຖͷ଎౓͕ݟΕΔ ෆࣗવʹ஗͍ϦΫΤετΛநग़Մೳ ͚ͩ͜͜Ͱ΋͔ͳΓ Մ؍ଌੑ͸্͕͍ͬͯΔͱݴ͑Δ

Slide 20

Slide 20 text

©Fusic Co., Ltd. 20 Traceͷৄࡉ ϧʔτεύϯ -> εύϯͱ͍ܗͰ ॲཧΠϕϯτ୯ҐͰঢ়گ͕ݟΕΔ ࣮ࡍͲͷΑ͏ʹॲཧ͕ߦΘΕ͍ͯΔ͔ͳͲ SQLͷ࣮ߦͳͲ΋ؚΊͨग़ྗΛߦͬͯ͘ΕΔ ஗͍ΫΤϦ͕ͳΜͳͷ͔ ͱ͍͏৘ใ΋Ұ໨ྎવ

Slide 21

Slide 21 text

©Fusic Co., Ltd. 21 ෼ࢄτϨʔεͷ఻ൖ ϚΠΫϩαʔϏεؒͷϦΫΤετʹτϨʔεIDΛҾ͖ܧ͙ʹ͸ɺ HTTPϔομ traceparent (W3C Trace Context) Λར༻ • Laravel͕֎෦αʔϏεʹGuzzle౳ͰϦΫΤετΛૹΔࡍɺtraceparentΛࣗಈ෇༩ • ड͚औΔαʔϏεଆͰ΋OTel͕༗ޮͳΒɺͦͷIDͰεύϯΛ։࢝ • ݁ՌɺΤϯυπʔΤϯυͰ1ͭͷτϨʔεID͕ྲྀΕɺো֐ՕॴΛૉૣ͘ಛఆͰ͖Δ PSR-15/18౳ͷAuto-Instrumentationύοέʔδ͕ɺड৴࣌ & ૹ৴࣌ͷHTTPϔομૢ࡞ΛࣗಈԽ

Slide 22

Slide 22 text

©Fusic Co., Ltd. 22 ෼ࢄτϨʔε main-otel-demo͔Βɺ other-service-laravelΛGuzzleʹͯ APIίʔϧΛߦ͏ɻ other-service-laravelଆͰ΋ ࣗಈܭ૷Λߦ͍ͬͯΔͨΊɺ SQLͷ࣮ߦ·ͰؚΊͯग़ྗͯ͘͠ΕΔ

Slide 23

Slide 23 text

©Fusic Co., Ltd. 23 Τϥʔͷࡍ͸Logsͷ৘ใ΋ Τϥʔͷ৔߹͸ϩάͷ৘ใ΋ग़ྗ ࠓճͷ৔߹ɺother-service-laravelଆͰ ໌ࣔతʹΤϥʔΛൃੜͤͨ͞ ෼ࢄτϨʔε͕ग़དྷ͍ͯΔͷͰɺ main-otel-demoଆ͔Β؍ଌग़དྷ͍ͯΔ

Slide 24

Slide 24 text

©Fusic Co., Ltd. 24 ܭ૷Λ͍ͯ͠ͳ͍αʔϏε͸ৄࡉ·Ͱ͸ݟΕͳ͍ Otelܭ૷͕͞Ε͍ͯͳ͍৔߹͸ ৄࡉͳ৘ใ͸ݟΕͳ͍ ࠓճͷ৔߹ɺGuzzleͰexample.com΁઀ଓ Ͳ͜ʹ઀ଓͨ͠ͷ͔ͱ͍͏৘ใ͸͋Δ͕ Ͳ͜ʹ͕͔͔͍࣌ؒͬͯΔ͔ͳͲͷ৘ใ͸ͳ͍

Slide 25

Slide 25 text

©Fusic Co., Ltd. 25 ϩάͷू໿ͱτϨʔεIDͷඥ෇͚ • Monolog༻ϋϯυϥ (opentelemetry-logger-monolog) Λར༻ → ϩάΛOTLPܦ༝Ͱૹ৴͠τϨʔεIDΛ෇༩ • ·ͨ͸ Laravelඪ४ϩάΛΧελϚΠζ͠ɺ"[%datetime%] %trace_id% %message%" ͷܗͰϑΝΠϧग़ྗ ͜ΕʹΑΓɺޙͰτϨʔεIDΛखֻ͔ΓʹৄࡉϩάΛ௥͑ΔΑ͏ʹͳΓ·͢ τϨʔε + ϩάͷ࿈ܞ͸τϥϒϧγϡʔςΟϯάΛܶతʹޮ཰Խ͠·͢

Slide 26

Slide 26 text

©Fusic Co., Ltd. 26 AWS΁ͷOpenTelemetryσʔλૹ৴ (Collector) AWSͰར༻͢Δ৔߹AWS Distro for OpenTelemetryΛར༻͢Δ ʢOpenTelemetry Collector Λϕʔεʹͨ͠ ίϯςφΠϝʔδ ΍ Lambda ϨΠϠʔʣ • ECS(Fargate) + ADOT CollectorαΠυΧʔ • Laravelίϯςφ → 4317΁OTLPૹ৴ → Collector → X-Ray & CloudWatch Logs/Metrics • Lambda༻OTel Extension • LambdaͰࣗಈτϨʔεΛߦ͏ϨΠϠʔ CollectorΛ࢖͏ͱτϨʔεΛX-RayʹɺϝτϦΫεΛCloudWatch MetricsʹͳͲॊೈʹৼΓ෼͚Մೳ Amazon Managed Service for PrometheusɺAmazon Managed GrafanaͳͲ΋͋Γ

Slide 27

Slide 27 text

©Fusic Co., Ltd. 27 αϯϓϦϯά & σʔλ؅ཧ • αϯϓϦϯάઓུ • Head-based sampling • ϦΫΤετ։࢝࣌ʹ֬཰ͰαϯϓϦϯά -> ॏཁͳτϨʔεΛݟಀ͢Մೳੑ • Tail-based sampling • CollectorͰશσʔλΛड͚औ͔ͬͯΒΤϥʔ౳ͷॏཁτϨʔεͷΈอଘ • ίετ΍ύϑΥʔϚϯεʹԠͯ͡ɺຊ౰ʹඞཁͳ৘ใ͕ൈ͚མͪΔՄೳੑ͸࣮֬ʹݕ౼ ίετΛແ੍ݶʹར༻ͯ͠Α͍༁Ͱ͸ແ͍͕ɺ αϯϓϦϯάΑΓ΋σΟεΫͷอଘظؒΛௐઅ͢ΔͳͲผͷΞϓϩʔνͰίετ࠷దԽͨ͠΄͏͕ྑͦ͞͏ ࣮ӡ༻Ͱ࠷దͳObservabilityΛ࣮ݱ͠·͠ΐ͏

Slide 28

Slide 28 text

©Fusic Co., Ltd. 28 ·ͱΊ OpenTelemetry͸؍ଌͷͨΊͷඪ४ج൫ Point 01 FrameworkΛհ͢Δ͜ͱͰࣗಈܭ૷͕ՄೳɺΤίγεςϜʹ৐͔ͬΕΔ Point 02 Մ؍ଌੑΛγεςϜʹऔΓೖΕΔ͜ͱͰɺ໰୊ରԠೳྗͷ޲্ΛਤΓ·͠ΐ͏ Point 03