Java Developers Summit (2023-02-28)
ΫϥυωΠςΟϒ࣌ͷ ίϯςφڥʹ͓͚ΔJavaΞϓϦέʔγϣϯͷ ϝτϦΫεɾϩάɾτϨʔεϞχλϦϯάEverforth / Acroquest Technology / JJUGShin Tanimoto (@cero_t)
View Slide
ࣗݾհ• ୩ຊ ৺ (Shin Tanimoto)• 2ࣾͰಇ͘ΤϯδχΞ• גࣜձࣾEverforthʢCTOʣ• Acroquest Technologyגࣜձࣾ• ຊJavaϢʔβʔάϧʔϓʢදʣ• Twitter : @cero_t
ࠓͷϝΠϯςʔϚ
ྩʹ͓͚ΔΦϒβόϦςΟͷΞʔΩςΫνϟʹ͍ͭͯ
ϞχλϦϯάͷ3େཁૉ• ϝτϦΫε• ϩά• τϨʔε
ϞχλϦϯάͷ3େཁૉ• ϝτϦΫε : ࣌ܥྻͷใ• ϩά : λΠϜελϯϓ͖ͭͷจࣈྻ• τϨʔε : ݺͼग़͠ͷه
ϝτϦΫε• ࣌ܥྻͷใ• ྫ• OS : CPU༻ɺϝϞϦ༻ྔʢʣɺσΟεΫ༻ྔʢʣ• JavaVM : ώʔϓ༻ྔʢʣɺεϨουɺίωΫγϣϯϓʔϧ• App : ϦΫΤετɺΤϥʔʢʣ
ϝτϦΫε• αϯϓϦϯάϕʔε• ͦͷॠؒͷ͔ɺूܭͨ͠Λऔಘ͢Δ• CPU༻ : ʮඵؒʯͷ༻ͷฏۉԿ%͔• ϝϞϦ༻ྔ : ʮ͍·ʯԿMB͍ͬͯΔ͔• GCճ : ʮ͍··ͰʯʹԿճGCΛߦ͔ͬͨ• αϯϓϦϯάִ͕ؒখ͍͞΄Ͳਫ਼্͕͕Δ͕ɺσʔλྔ͕૿͑Δ
ϝτϦΫε• ฏͷϞχλϦϯά• OSϨϕϧͷϝτϦΫε͚ͩऔΓ͕ͪ• JavaϨϕϧͷϝτϦΫεΛऔ͍ͬͯΕେͨ͠ͷ• ΞϓϦέʔγϣϯϨϕϧͷϝτϦΫεΛऔΕΔΑ͏ʹ͢Δͷ ૬ͳΓ͜Έ• ྩͰͯ͢औΔ͠
ϩά• ࣌ࠁΛ͏จࣈྻ• ྫ• ΞϓϦέʔγϣϯϩάɺOSͷϩάɺGCϩάɺ ϑϨʔϜϫʔΫϛυϧΣΞͷϩά• ॲཧΛߦͬͨɺΤϥʔ͕ى͖ͨɺԿΒ͔ͷෆ߹ͳ͜ͱ͕ى͖ͨ• ͦͷ࣌ͷίϯςΩετใʢมఆͳͲʣ
ϩά• Πϕϯτϕʔε• ࣄ͕ى͖ͨ࣌ʹه͢Δ• αϯϓϦϯάϕʔεͱҟͳΓɺͯ͢ه͞ΕΔ• จࣈྻͳͷͰϝτϦΫεʹൺͯαΠζ͕େ͖͍• 1ͭͷϦΫΤετʹΑͬͯۀଆͷσʔλετΞʹॻ͖ࠐ·ΕΔใྔΑΓ ϩάͷํ͕େ͖͘ͳΔ͜ͱ͕͋Δ• ΞϓϦέʔγϣϯͷσʔλετΞΑΓϩάͷσʔλετΞͷํ͕ ߴੑೳɾେ༰ྔͳͷΛٻΊΒΕΔ͜ͱ͕͋Δ
ϩά• ฏͷϞχλϦϯά• ͝ͱʹzipͯ͠ωοτϫʔΫετϨʔδʹஔ͖͕ͪ• ඞཁͳ࣌ʹμϯϩʔυͯ͠։͔͘ɺviͰݟʹߦ͖͕ͪ• ʮERRORʯͱ͍͏ϝοηʔδ͕͋Εύτϥϯϓ͕ճΓ͕ͪ• ྩͷϞχλϦϯάͰৗʹݕࡧͰ͖Δঢ়ଶʹ͢Δ͠• ͕ͦͯ͠ݟ͔ͭͬͨΒSlackͳͲʹ௨͢Δ͠
τϨʔε• ݺͼग़͠ͷه• ෳͷΞϓϦέʔγϣϯϓϩηεΛލ͍ͩελοΫτϨʔεͷΑ͏ͳͷ• ͍ͭɺ୭͕୭ΛݺΜͰ͔ɺͲΕ͘Βֻ͍͔͔ͬͨɺޭ͔ࣦͨ͠ഊ͔ͨ͠• ΠϕϯτϕʔεͰऔಘ• ͯ͢औಘ͢Δ͔ɺҰఆͷׂ߹Ͱऔಘ͢Δ͔ઃఆՄೳ
τϨʔε• ελοΫτϨʔε• Javaϓϩηε͚ͩͰ؆ܿ͢Δݺͼग़͠֊• ओʹྫ֎ൃੜ࣌ʹऔಘ͢Δ• ͦΕͳΒෳΞϓϦέʔγϣϯΛލ͍ͩτϨʔεऔΓ͍ͨΑͶ• τϨʔεɺൃੜ࣌ͷ֬ೝɺϘτϧωοΫݕग़ʹར༻Ͱ͖Δ
τϨʔε• ฏͷϞχλϦϯά• τϨʔεऔΒͳ͍ or औΕͳ͍• ྩͰऔΓ·͠ΐ͏
Spring Boot 3.0࣌ͷ ϞχλϦϯά
σϞ
σϞͰͬͨιʔείʔυ• https://github.com/cero-t/spring-store-2022• ࣮ߦखॱઃఆͷৄࡉͳ༰ϒϩάʹهࡌ• ຊεϥΠυͷ࠷ऴϖʔδʹ·ͱΊ·͢
ΞϓϦέʔγϣϯͷߏBFFPayment ServiceOrder ServiceStock ServiceDelivery ServiceItem ServiceCart ServiceQueueECαΠτͷόοΫΤϯυΛΠϝʔδ
ར༻ٕͨ͠ज़ελοΫ• ϚΠΫϩαʔϏεΞϓϦέʔγϣϯ• Java 17• Spring Boot 3.0 (without Spring Cloud)• Spring Boot Web• Micrometer• OpenTelemetry
ར༻ٕͨ͠ज़ελοΫ• GrafanaελοΫ• Grafana : ϞχλϦϯάͷUI• Prometheus : ϝτϦΫεͷऩूͱσʔλετΞ• Loki : ϩάͷσʔλετΞ• Tempo : τϨʔεͷσʔλετΞ
ϞχλϦϯάͷߏJava ProcessGrafanaTempoPrometheusLokiऔಘͯ͠ՄࢹԽϝτϦΫεϩάτϨʔεJava ProcessJava ProcessSpring Boot Application
Spring Boot + PrometheusͰ ϝτϦΫεऩू
Spring BootͰϝτϦΫεऩू• Spring BootΞϓϦέʔγϣϯͷϝτϦΫεใΛWeb APIͱͯ͠ ެ։ͯ͠ɺPrometheus͔Βఆظతʹऔಘͯ͠ऩूɾอଘ͢Δ• Grafana͕Prometheus͔ΒϝτϦΫεΛऔಘͯ͠ՄࢹԽ͢ΔSpring Boot ApplicationϝτϦΫεऔಘAPI PrometheusఆظతʹऔಘGrafanaऔಘͯ͠ՄࢹԽ
Spring BootͰϝτϦΫεऩू• ϝτϦΫεऔಘͷWeb APIΛެ։͢Δํ๏• Spring BootΞϓϦέʔγϣϯͷdependencyʹ ActuatorͱMicrometerΛՃ͢Δ• application.propertiesʹͯActuatorͷΤϯυϙΠϯτΛެ։͢Δ
ActuatorͱMicrometerΛՃ• pom.xml ͷdependencyʹ࣍ͷ2ͭΛՃ• Actuator : spring-boot-starter-actuator• Spring BootͷઃఆϝτϦΫεΛWeb APIͱͯ͠ެ։͢ΔͨΊͷϥΠϒϥϦ• Micrometer : micrometer-registry-prometheus• OSJavaVMɺSpring BootͷϝτϦΫεใΛऔಘ͢ΔͨΊͷϥΠϒϥϦ• “-prometheus” Prometheus͔ΒΞΫηε͢ΔͨΊͷΤϯυϙΠϯτΛ ActuatorΛ༻͍ͯެ։͢Δɻ• “-datadog” ͱ͍͏ϥΠϒϥϦఆظతʹDataDogʹϝτϦΫεΛૹ৴͢ΔͳͲ ϥΠϒϥϦʹΑͬͯڍಈ͕ҟͳΔ
ActuatorͱMicrometerΛՃ• pom.xml ͷdependencyʹ࣍ͷ2ͭΛՃorg.springframework.bootspring-boot-starter-actuatorruntimeio.micrometermicrometer-registry-prometheusruntime
ActuatorͷΤϯυϙΠϯτΛެ։• application.properties ʹͯɺMicrometerʹؔ͢Δ ActuatorͷΤϯυϙΠϯτΛެ։͢Δmanagement.endpoints.web.exposure.include=*management.metrics.distribution.percentiles-histogram.http.server.requests=truemanagement.metrics.tags.application=${spring.application.name}# Micrometerʹؔ͢ΔϝτϦΫε͚ͩެ։͢Δ߹ʹ࣍ͷઃఆͰྑ͍ # management.endpoints.web.exposure.include=prometheus
ActuatorͱMicrometerΛՃ• http://(ΞϓϦέʔγϣϯͷΞυϨε)/actuator/prometheus ʹ ΞΫηε͢ΔͱɺϝτϦΫεҰཡΛऔಘͰ͖Δprocess_files_max_files{application="bff",} 10240.0process_uptime_seconds{application="bff",} 15057.411jvm_threads_peak_threads{application="bff",} 43.0jvm_threads_states_threads{application="bff",state="new",} 0.0jvm_threads_states_threads{application="bff",state="runnable",} 11.0jvm_threads_states_threads{application="bff",state="terminated",} 0.0jvm_threads_states_threads{application="bff",state="waiting",} 12.0jvm_threads_states_threads{application="bff",state="timed-waiting",} 9.0jvm_threads_states_threads{application="bff",state="blocked",} 0.0
PrometheusͰϝτϦΫεऩू• ʢGrafanaPrometheusࣗମͷߏஙʹׂ͍ͭͯѪʣ• PrometheusͷઃఆϑΝΠϧʹSpring BootΞϓϦέʔγϣϯͷ ΞυϨεΛྻڍ͠ɺϝτϦΫεΛऩूͰ͖ΔΑ͏ʹ͢Δ
PrometheusͰϝτϦΫεऩू• PrometheusͷઃఆϑΝΠϧʹSpring BootΞϓϦέʔγϣϯͷ ΞυϨεΛྻڍ͠ɺϝτϦΫεΛऩूͰ͖ΔΑ͏ʹ͢Δscrape_configs:- job_name: "apps"metrics_path: "/actuator/prometheus"static_configs:- targets:["host.docker.internal:9000","host.docker.internal:9001","host.docker.internal:9002","host.docker.internal:9010"]
ΫϥυωΠςΟϒ࣌ʹ ΞϓϦέʔγϣϯͷΞυϨεΛ ྻڍ͢Δͱ͔ਖ਼ؾͰ͔͢ʁ
ΫϥυωΠςΟϒ࣌ʹ ΞϓϦέʔγϣϯͷΞυϨεΛ ྻڍ͢Δͱ͔ਖ਼ؾͰ͔͢ʁ͖ͪΜͱޙͰղઆ͠·͢
GrafanaʹμογϡϘʔυΛՃ• GrafanaίϛϡχςΟ͕࡞ͨ͠μογϡϘʔυΛެ։͍ͯ͠Δ• https://grafana.com/grafana/dashboards/• JVM (Micrometer) - 4701 ͕Φεεϝ
Prometheuspullܕ• ΞϓϦέʔγϣϯଆ͔ΒσʔλΛૹΔͷΛʮPushܕʯͱݺͿ• ΞϓϦέʔγϣϯʹऔΓʹߦ͘ͷΛʮPullܕʯͱݺͿ• PrometheusʮPullܕʯͰ͋ΓϝτϦΫεऩूͷϛυϧΣΞͱ ͯ͠ΘΓͱ͍͠෦ྨʢ*ݸਓͷײͰ͢ʣ• PullܕͰࢄΞϓϦέʔγϣϯͷϝτϦΫεΛͳ͘ ऩूͰ͖ΔʢޙͰղઆ͠·͢ʣ
Prometheuspullܕ• Pullܕͷར• ࢹରͱͳΔΞϓϦέʔγϣϯ͕໌֬Ͱ͋ΔͨΊ ʮϝτϦΫεΛऔಘͰ͖ͳ͔ͬͨʯͱ͍͏Λݕग़͍͢͠• PushܕͰʮૹΒΕͯ͜ͳ͔ͬͨʯ = ʮه͠ͳ͍ʯ͚ͩͱ ͳΓɺͦΕΛͱͯ͠ݕग़͢Δ͜ͱ͕͍͠• ࢹର͕ଟ͘ͳͬͨࡍʹɺPushܕͰϝτϦΫεऩूαʔό͕ύϯΫͯ͠ ͠·͏Մೳੑ͕͋Δ͕ɺPullܕͳΒऩूසΛ੍ޚ͢Δ͜ͱ͕Ͱ͖Δʮ͔ʯ
Spring Boot + LokiͰ ϩάऩू
Spring BootͰϩάऩू• Spring BootΞϓϦέʔγϣϯͷϩάΛPromtailͰఆظతʹऩूͯ͠ Lokiʹૹ৴͢ΔʢPullܕʣ• Grafana͕Loki͔ΒϩάΛऔಘͯ͠ՄࢹԽ͢ΔSpring Boot Applicationϩά ϑΝΠϧLokiఆظతʹऩूGrafanaऔಘͯ͠ՄࢹԽPromtailૹ৴
Ͱ͏ɺϩάϑΝΠϧͱ͔ ͍ͨ͘ͳ͍Ͱ͢ΑͶ
Spring BootͰϩάऩू• Spring BootΞϓϦέʔγϣϯͷϩάΛLoki4JͷLogback AppenderΛ ༻͍ͯఆظతʹLokiʹૹ৴͢ΔʢPushܕʣ• Grafana͕Loki͔ΒϩάΛऔಘͯ͠ՄࢹԽ͢ΔSpring Boot ApplicationLoki4J AppenderLokiఆظతʹૹ৴Grafanaऔಘͯ͠ՄࢹԽ
Loki4JͷLogback AppenderΛՃ• pom.xml ͷdependencyʹloki-logback-appenderΛՃ• ϑΝΠϧίϯιʔϧͰͳ͘LokiʹϩάΛૹΔΑ͏ͳ LogbackͷAppendercom.github.loki4jloki-logback-appender 1.3.2runtime
LogbackͷઃఆϑΝΠϧΛ࡞• LogbackͷઃఆϑΝΠϧΛ࡞͠ɺAppenderʹLoki4jAppenderΛ ࢦఆ͢Δʢlogback-loki.xmlʣhttp://localhost:3100/loki/api/v1/pushapp=${appName},host=${HOSTNAME},traceID=%X{traceId:-NONE},level=%level${FILE_LOG_PATTERN}true
LogbackͷઃఆϑΝΠϧΛ༗ޮʹ͢Δ• application.properties ʹͯɺ࡞ͨ͠LogbackͷઃఆϑΝΠϧΛ ༗ޮʹ͢Δ• ͍ͭͰʹɺޙͰτϨʔεऩूͷ࣌ʹ͏τϨʔεID ϩάʹग़ΔΑ͏ʹ͓ͯ͘͠logging.pattern.level=%5p [${spring.application.name:},%X{traceId:-},%X{spanId:-}]logging.config=classpath:logback-loki.xml
LokiͰϩάऩू• ʢLokiࣗମͷߏஙʹׂ͍ͭͯѪʣ• LokiΛىಈ͢ΕΞϓϦέʔγϣϯଆ͔ΒϩάΛPush͢ΔͷͰ LokiଆʹಛʹඞཁͳઃఆͳͲͳ͠
ϩάpush͔pull͔• ྺ࢙తʹϩάPullܕͰऩू• ϑΝΠϧʹग़ྗ͠ɺόονͰωοτϫʔΫετϨʔδʹҠಈͤ͞Δ• FluentdLogstashɺFilebeatͳͲͰऩूͯ͠ElasticsearchʹૹΔ• ϩάΛPushܕʹ͢Δͷෆ҆• ϩάʮγεςϜͷෆਖ਼ऴྃલͷ࠷ظͷʯ• ΞϓϦέʔγϣϯϓϩηε͕ࢮΜͰϩά͚͍ͩͨ͠• PushܕͰड͚औΔଆͷϛυϧΣΞ͕ࢭ·Δͱϩά͕ࣦΘΕΔ
ͦΜͳʹॏཁͳใΛ ͳͥϩάͳͲͱ͍͏ᐆດͳํ๏Ͱ ͦ͏ͱ͢Δͷ͔ʁ
ϩάͷ֓೦Λݟ͖͢• ఏݴ : ϩάͱ͍͏֓೦ͦͷͷΛݟ͖͢• ϩάॏཁͱ͞ΕΔΘΓʹݫີʹઃܭ͞Εͳ͍ࣄ͕ଟ͍• ํ͚ܾͩΊͯޙ։ൃऀͤʢͦͷΘΓʹɺ༗༻Ͱͳ͚ΕౖΒΕΔʣ• ॏཁͳใͱɺͦ͏Ͱͳ͍ใ͕ࡶଟʹࠞࡏ͢Δ• ຊʹॏཁͳͷɺϩάͰͳ͘ۀσʔλͱͯ͠ଊ͑Δ͖• ຊʹॏཁͳใԿ͔Λਫ਼ࠪ͠ɺݫີʹઃܭͨ͠͏͑Ͱ ৴པੑͷߴ͍σʔλετΞʹه͢Δ• ϩάͱ͍͏ΈʹཔΔͷʮ͋ͬͨΒخ͍͠ʯʹཹΊΔ͖
ϩάpush͔pull͔• ͦͷ্ͰɺϩάPushܕ͕ྑ͍ͷ͔ɺPullܕ͕ྑ͍ͷ͔• Pullܕͷํ͕ॊೈੑ͕ߴ͍• ऩूͯ͠Ճ͢Δͱ͍͏ϓϩηεΛܦΔ͜ͱ͕Ͱ͖Δ• ͦͷաఔʹKafkaRabbitMQΛڬΜͰෛՙΛࢄͰ͖Δ• Pushܕͷํ͕༰қ• ϩʔΧϧPCͰ࣮ߦ͢ΔࡍʹɺϩάϑΝΠϧΛग़ྗ͠ͳͯ͘ࡁΉ• Ճ͕ඞཁͳ͍ʢ͘͠JSONܗࣜͰग़ྗ͢Δʣ߹ ϩάऩूαʔόʹߴෛՙֻ͕͔ΔΑ͏ͳ݅Ͱͳ͍߹ͳͲͰ༗ޮ
Spring Boot + TempoͰ τϨʔεऩू
Spring BootͰτϨʔεऩू• τϨʔεऩूͷͨΊʹ࣍ͷ2͕ͭඞཁ• ᶃτϨʔεIDͷ• ʮͲͷϦΫΤετʹඥ͘ॲཧ͔ʯʮ୭͔ΒݺΕ͔ͨʯͱ͍͏ใΛΞϓϦέʔγϣϯؒͰͤ͞Δ• ᶄτϨʔεใͷੵ• τϨʔεใΛσʔλετΞʹૹͬͯੵ͢Δ
Spring BootͰτϨʔεIDͷ• Spring BootΞϓϦέʔγϣϯͷؒͷ௨৴ʢHTTPAMQPʣͷ ϔομʹτϨʔεIDΛೖΕͯૹ৴ɺड৴͢ΔSpring Boot Application1Spring Boot Application2Spring Boot Application4Spring Boot Application3traceparent = req001-01traceparent = req001-02traceparent = req001-01
Spring BootͰτϨʔεIDͷ• τϨʔεIDͷૹड৴ʹSpring Boot 2.x·ͰSpring Cloud SleuthΛ༻͍͍͕ͯͨɺSpring Boot 3.0͔ΒMicrometer Tracing͕ ར༻Ͱ͖ΔΑ͏ʹͳͬͨSpring Boot ApplicationMicrometer TracingRestTemplateSpring Boot ApplicationMicrometer TracingRestControllerHTTPϔομʹ τϨʔεIDͷ༩ ʢࣗಈʣHTTPϔομ͔Β τϨʔεIDͷऔಘ ʢࣗಈʣ
Micrometer TracingͷՃ• pom.xml ͷdependencyʹ micrometer-tracing-bridge-otel ͔ -brave ΛՃ͢ΔʢͲͪΒͰৼΔ͍ʹҧ͍ͳ͍ʣ• Spring BootͷRestTemplateSpring Cloud StreamΛ༻͍ͨ௨৴Ͱ τϨʔεIDͷ͕ߦΘΕΔʢࣗಈʣio.micrometermicrometer-tracing-bridge-otelruntime
Spring BootͰτϨʔεऩू• Spring BootΞϓϦέʔγϣϯͷτϨʔεใͷهΛ Open Telemetry͔BraveͷϥΠϒϥϦΛ༻͍ͯTempoʹૹ৴͢Δ• Grafana͕Tempo͔ΒτϨʔεใΛऔಘͯ͠ՄࢹԽ͢ΔSpring Boot ApplicationOpen TelemetryTempoఆظతʹૹ৴Grafanaऔಘͯ͠ՄࢹԽ
Open TelemetryͷՃ• Spring Boot͔ΒτϨʔεใΛૹ৴͢Δํ๏• pom.xml ͷdependencyʹ opentelemetry-exporter-zipkin ͔ zipkin-reporter-brave ΛՃ͢ΔʢৼΔ͍ʹҧ͍ͳ͍ʣ• ZipkinʹτϨʔεใΛૹΔϥΠϒϥϦ ʢTempo͕ZipkinޓAPIΛඋ͍͑ͯΔͨΊར༻Ͱ͖Δʣio.opentelemetryopentelemetry-exporter-zipkinruntime
LogbackͷઃఆϑΝΠϧΛ༗ޮʹ͢Δ• application.properties ʹͯɺτϨʔεใΛͲΕ͘Β͍ͷׂ߹ͰTempoʹૹΔ͔Λઃఆ͢ΔʢσϑΥϧτ10%ʣ• ։ൃதσϞͰ100%ʹͨ͠ํ͕ྑ͍• ӡ༻தͷΤϥʔൃੜ࣌ʹ༨ͣ͞τϨʔεΛऔಘ͢ΔͨΊʹ 100%ʹઃఆ͓ͯ͘͠ͱྑ͍management.tracing.sampling.probability=1.0
TempoͰτϨʔεऩू• ʢTempoࣗମͷߏஙʹׂ͍ͭͯѪʣ• TempoͷઃఆϑΝΠϧʹͯɺZipkinޓAPIΛ༗ޮʹ͢Δserver:http_listen_port: 3200distributor:receivers:zipkin:
τϨʔεpushͷΈ• ZipkinʢޓAPIؚΉʣτϨʔεใΛड͚औΔͨΊͷAPIΛ ެ։͓ͯ͠Γ֤ΞϓϦέʔγϣϯ͔Βͦ͜ʹτϨʔεใΛૹΔ ʮPushܕʯͷΞʔΩςΫνϟ͕࠾༻͞Ε͍ͯΔ• ͍ͣΕPullܕͷΤʔδΣϯτ͕ग़ͯ͘Δ͔Εͳ͍
Spring Boot 3.0 + k8sͷ ϞχλϦϯά
k8s্ͰPullܕΛӡ༻͢Δ՝• PrometheusͰϞχλϦϯάରͷΞυϨεΛྻڍͨ͠• k8sͷΑ͏ʹಈతʹϞχλϦϯάର͕૿ݮɾมԽ͢Δ߹ʹ ͲͷΑ͏ʹऩू͢Δͷ͔ʁ
αʔϏεσΟεΧόϦ• ΞϓϦέʔγϣϯͷϧʔςΟϯάϩʔυόϥϯγϯάͷͨΊʹ ༻͍ΒΕΔαʔϏεσΟεΧόϦͷΈΛϞχλϦϯάͰར༻͢Δ• k8sͰCoreDNSΛ༻͍ͨαʔϏεσΟεΧόϦΛߦ͏
αʔϏεσΟεΧόϦ• 20ඵͰ͔ΔαʔϏεσΟεΧόϦSpring Boot Application1Service RegistrySpring Boot Application1Spring Boot Application1Spring Boot Application1ᶃΞυϨεͷొ ᶄݺͼग़͍ͨ͠αʔϏεͷΞυϨεΛऔಘᶅݺͼग़͠ Spring Boot Application2
αʔϏεσΟεΧόϦ• αʔϏεσΟεΧόϦΛར༻ͨ͠ϞχλϦϯάSpring Boot Application1 PrometheusPromtailService RegistrySpring Boot Application1Spring Boot Application1Spring Boot ApplicationᶃΞυϨεͷొ ᶄϞχλϦϯά͍ͨ͠αʔϏεͷΞυϨεΛऔಘᶅݺͼग़͠
PrometheusͷαʔϏεσΟεΧόϦઃఆ• PrometheusͷઃఆϑΝΠϧʹͯɺkubernetes_sd_config ͰαʔϏεσΟεΧόϦΛઃఆ͢Δscrape_configs:- job_name: "apps"metrics_path: "/actuator/prometheus"kubernetes_sd_config:- role: pod relabel_configs:- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]action: keepregex: true- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]action: replacetarget_label: __metrics_path__regex: (.+)- source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]action: replacetarget_label: __address__regex: (.+?)(?::\d+)?;(\d+)replacement: $1:$2ଞʹ৭ʑͱઃఆ͕ඞཁ͔ͩΒ PrometheusPromtail helmͰΠϯετʔϧ͢Δͷ͕Φεεϝ
PrometheusͷαʔϏεσΟεΧόϦઃఆ• ΞϓϦέʔγϣϯΛσϓϩΠ͢ΔϚχϑΣετϑΝΠϧʹͯ annotationsͰϙʔτ൪߸ActuatorͷύεΛࢦఆ͢Δspec:template:metadata:labels:app: bffannotations:prometheus.io/scrape: "true"prometheus.io/port: "9000"prometheus.io/path: /actuator/prometheus
ίϯςφ࣌ͰPullͰେৎ• ϚΠΫϩαʔϏεͷΑ͏ʹಈతʹ૿ݮ͢ΔΞϓϦέʔγϣϯ܈Λ ϞχλϦϯά͢ΔࡍɺαʔϏεσΟεΧόϦͷΈͱซ༻͢Ε PullܕͰಛʹͳ͘ϞχλϦϯάڥΛߏஙͰ͖Δ
·ͱΊ
·ͱΊ• ϝτϦΫεɺϩάɺτϨʔεऔΕΔ͚ͩऔΓ·͠ΐ͏• GrafanaελοΫͰ͋ΔɺGrafanaɺPrometheusɺLokiɺTempoͰ ·ͱΊͯϞχλϦϯάͰ͖Δ• Spring Boot 3.0ͰϞχλϦϯάपΓͷϥΠϒϥϦͳͲཧ͞Ε GrafanaελοΫͳͲͰϞχλϦϯά͘͢͠ͳ͍ͬͯΔ• ϩά͔ΒτϨʔεͷભҠͳͲɺ͜ΕΒͷ3ͭΛͦΕͧΕ ભҠͰ͖ΔΑ͏ʹ͢ΔτϨϯυ͕͋Δ
·ͱΊ• PushܕɺPullܕͷ͍ͣΕͰͳ͘σʔλΛऩूͰ͖Δ• ૯ͯ͡ɺPushܕͷํ͕༰қɺPullܕͷํ͕ॊೈͱͳ͍ͬͯΔ• DataDogAWS CloudWatchͳͲجຊతʹPushܕ• ΞϓϦέʔγϣϯଆͷιʔείʔυઃఆϑΝΠϧ͕ Πϯϑϥʹґଘ͠ͳ͍Α͏ཹҙ͢Δͱ։ൃ͍͢͠Α
Resources• ιʔείʔυ• https://github.com/cero-t/spring-store-2022• υΩϡϝϯτ• Spring Boot 3.0ΞϓϦέʔγϣϯΛGrafanaελοΫͰՄࢹԽͯ͠Έͨɻ• https://cero-t.hatenadiary.jp/entry/2023/01/10/163547• GrafanaελοΫʹΑΔSpring BootΞϓϦέʔγϣϯࢹͷৄࡉʢͦͷ1 Grafana + Prometheusฤʣ• https://cero-t.hatenadiary.jp/entry/2023/01/11/181715• GrafanaελοΫʹΑΔSpring BootΞϓϦέʔγϣϯࢹͷৄࡉʢͦͷ2 Grafana + Lokiฤʣ• https://cero-t.hatenadiary.jp/entry/2023/01/12/130022• GrafanaελοΫʹΑΔSpring BootΞϓϦέʔγϣϯࢹͷৄࡉʢͦͷ3 Grafana + Tempoฤʣ• https://cero-t.hatenadiary.jp/entry/2023/01/15/203442
shintanimoto
masahiro_nishimi
os1ma
inouehi
zoetrope
kazutan
siketyan
yuya4
hirobe1999
alli83
taclose
youliangdao
akatsuki174
lara
colly
jacobian
caitiem20
hursman
phodgson
trallard
ddemaree
davidbonilla
mraible