Tweet
Tweet

Slide 1

Slide 1 text

Ϋϥ΢υωΠςΟϒ࣌୅ͷ
 ίϯςφ؀ڥʹ͓͚ΔJavaΞϓϦέʔγϣϯͷ
 ϝτϦΫεɾϩάɾτϨʔεϞχλϦϯά Everforth / Acroquest Technology / JJUG Shin Tanimoto (@cero_t)

Slide 2

Slide 2 text

ࣗݾ঺հ • ୩ຊ ৺ (Shin Tanimoto) • 2ࣾͰಇ͘ΤϯδχΞ • גࣜձࣾEverforthʢCTOʣ • Acroquest Technologyגࣜձࣾ • ೔ຊJavaϢʔβʔάϧʔϓʢ୅දʣ • Twitter : @cero_t

Slide 3

Slide 3 text

ࠓ೔ͷϝΠϯςʔϚ

Slide 4

Slide 4 text

ྩ࿨ʹ͓͚ΔΦϒβόϦςΟ ͷΞʔΩςΫνϟʹ͍ͭͯ

Slide 5

Slide 5 text

ϞχλϦϯάͷ3େཁૉ • ϝτϦΫε • ϩά • τϨʔε

Slide 6

Slide 6 text

ϞχλϦϯάͷ3େཁૉ • ϝτϦΫε : ࣌ܥྻͷ਺஋৘ใ • ϩά : λΠϜελϯϓ͖ͭͷจࣈྻ • τϨʔε : ݺͼग़͠ͷه࿥

Slide 7

Slide 7 text

ϝτϦΫε • ࣌ܥྻͷ਺஋৘ใ • ྫ • OS : CPU࢖༻཰ɺϝϞϦ࢖༻ྔʢ཰ʣɺσΟεΫ࢖༻ྔʢ཰ʣ • JavaVM : ώʔϓ࢖༻ྔʢ཰ʣɺεϨου਺ɺίωΫγϣϯϓʔϧ਺ • App : ϦΫΤετ਺ɺΤϥʔ਺ʢ཰ʣ

Slide 8

Slide 8 text

ϝτϦΫε • αϯϓϦϯάϕʔε • ͦͷॠؒͷ஋͔ɺूܭͨ͠஋Λऔಘ͢Δ • CPU࢖༻཰ : ʮඵؒʯͷ࢖༻཰ͷฏۉ͸Կ%͔ • ϝϞϦ࢖༻ྔ : ʮ͍·ʯ͸ԿMB࢖͍ͬͯΔ͔ • GCճ਺ : ʮ͍··ͰʯʹԿճGCΛߦ͔ͬͨ • αϯϓϦϯάִ͕ؒখ͍͞΄Ͳਫ਼౓্͕͕Δ͕ɺσʔλྔ͕૿͑Δ

Slide 9

Slide 9 text

ϝτϦΫε • ฏ੒ͷϞχλϦϯά • OSϨϕϧͷϝτϦΫε͚ͩऔΓ͕ͪ • JavaϨϕϧͷϝτϦΫεΛऔ͍ͬͯΕ͹େͨ͠΋ͷ • ΞϓϦέʔγϣϯϨϕϧͷϝτϦΫεΛऔΕΔΑ͏ʹ͢Δͷ͸ 
 ૬౰ͳ΍Γ͜Έ • ྩ࿨Ͱ͸͢΂ͯऔΔ΂͠

Slide 10

Slide 10 text

ϩά • ࣌ࠁΛ൐͏จࣈྻ • ྫ • ΞϓϦέʔγϣϯϩάɺOSͷϩάɺGCϩάɺ 
 ϑϨʔϜϫʔΫ΍ϛυϧ΢ΣΞͷϩά • ॲཧΛߦͬͨɺΤϥʔ͕ى͖ͨɺԿΒ͔ͷෆ౎߹ͳ͜ͱ͕ى͖ͨ • ͦͷ࣌ͷίϯςΩετ৘ใʢม਺஋΍ఆ਺஋ͳͲʣ

Slide 11

Slide 11 text

ϩά • Πϕϯτϕʔε • ࣄ͕ى͖ͨ࣌ʹه࿥͢Δ • αϯϓϦϯάϕʔεͱ͸ҟͳΓɺ͢΂ͯه࿥͞ΕΔ • จࣈྻͳͷͰϝτϦΫεʹൺ΂ͯαΠζ͕େ͖͍ • 1ͭͷϦΫΤετʹΑͬͯۀ຿ଆͷσʔλετΞʹॻ͖ࠐ·ΕΔ৘ใྔΑΓ 
 ϩάͷํ͕େ͖͘ͳΔ͜ͱ͕͋Δ • ΞϓϦέʔγϣϯͷσʔλετΞΑΓ΋ϩάͷσʔλετΞͷํ͕ 
 ߴੑೳɾେ༰ྔͳ΋ͷΛٻΊΒΕΔ͜ͱ͕͋Δ

Slide 12

Slide 12 text

ϩά • ฏ੒ͷϞχλϦϯά • ೔͝ͱʹzipͯ͠ωοτϫʔΫετϨʔδʹஔ͖͕ͪ • ඞཁͳ࣌ʹμ΢ϯϩʔυͯ͠։͔͘ɺviͰݟʹߦ͖͕ͪ • ʮERRORʯͱ͍͏ϝοηʔδ͕͋Ε͹ύτϥϯϓ͕ճΓ͕ͪ • ྩ࿨ͷϞχλϦϯάͰ͸ৗʹݕࡧͰ͖Δঢ়ଶʹ͢Δ΂͠ • ͦͯ͠໰୊͕ݟ͔ͭͬͨΒSlackͳͲʹ௨஌͢Δ΂͠

Slide 13

Slide 13 text

τϨʔε • ݺͼग़͠ͷه࿥ • ෳ਺ͷΞϓϦέʔγϣϯϓϩηεΛލ͍ͩελοΫτϨʔεͷΑ͏ͳ΋ͷ • ͍ͭɺ୭͕୭ΛݺΜͰ͔ɺͲΕ͘Βֻ͍͔͔ͬͨɺ੒ޭ͔ࣦͨ͠ഊ͔ͨ͠ • ΠϕϯτϕʔεͰऔಘ • ͢΂ͯऔಘ͢Δ͔ɺҰఆͷׂ߹Ͱऔಘ͢Δ͔͸ઃఆՄೳ

Slide 14

Slide 14 text

τϨʔε • ελοΫτϨʔε • Javaϓϩηε಺͚ͩͰ؆ܿ͢Δݺͼग़͠֊૚ • ओʹྫ֎ൃੜ࣌ʹऔಘ͢Δ • ͦΕͳΒෳ਺ΞϓϦέʔγϣϯΛލ͍ͩτϨʔε΋औΓ͍ͨΑͶ • τϨʔε͸ɺ໰୊ൃੜ࣌ͷ֬ೝ΍ɺϘτϧωοΫݕग़ʹར༻Ͱ͖Δ

Slide 15

Slide 15 text

τϨʔε • ฏ੒ͷϞχλϦϯά • τϨʔε͸औΒͳ͍ or औΕͳ͍ • ྩ࿨Ͱ͸औΓ·͠ΐ͏

Slide 16

Slide 16 text

Spring Boot 3.0࣌୅ͷ
 ϞχλϦϯά

Slide 17

Slide 17 text

σϞ

Slide 18

Slide 18 text

σϞͰ࢖ͬͨιʔείʔυ • https://github.com/cero-t/spring-store-2022 • ࣮ߦखॱ΍ઃఆͷৄࡉͳ಺༰͸ϒϩάʹهࡌ • ຊεϥΠυͷ࠷ऴϖʔδʹ·ͱΊ·͢

Slide 19

Slide 19 text

ΞϓϦέʔγϣϯͷߏ੒ BFF Payment 
 Service Order 
 Service Stock 
 Service Delivery 
 Service Item 
 Service Cart 
 Service Queue ECαΠτͷόοΫΤϯυΛΠϝʔδ

Slide 20

Slide 20 text

ར༻ٕͨ͠ज़ελοΫ • ϚΠΫϩαʔϏεΞϓϦέʔγϣϯ • Java 17 • Spring Boot 3.0 (without Spring Cloud) • Spring Boot Web • Micrometer • OpenTelemetry

Slide 21

Slide 21 text

ར༻ٕͨ͠ज़ελοΫ • GrafanaελοΫ • Grafana : ϞχλϦϯάͷUI • Prometheus : ϝτϦΫεͷऩूͱσʔλετΞ • Loki : ϩάͷσʔλετΞ • Tempo : τϨʔεͷσʔλετΞ

Slide 22

Slide 22 text

ϞχλϦϯάͷߏ੒ Java 
 Process Grafana Tempo Prometheus Loki औಘͯ͠ՄࢹԽ ϝτϦΫε ϩά τϨʔε Java 
 Process Java 
 Process Spring Boot 
 Application

Slide 23

Slide 23 text

Spring Boot + PrometheusͰ
 ϝτϦΫεऩू

Slide 24

Slide 24 text

Spring BootͰϝτϦΫεऩू • Spring BootΞϓϦέʔγϣϯͷϝτϦΫε৘ใΛWeb APIͱͯ͠ 
 ެ։ͯ͠ɺPrometheus͔Βఆظతʹऔಘͯ͠ऩूɾอଘ͢Δ • Grafana͕Prometheus͔ΒϝτϦΫεΛऔಘͯ͠ՄࢹԽ͢Δ Spring Boot 
 Application ϝτϦΫε औಘAPI Prometheus ఆظతʹऔಘ Grafana औಘͯ͠ՄࢹԽ

Slide 25

Slide 25 text

Spring BootͰϝτϦΫεऩू • ϝτϦΫεऔಘͷWeb APIΛެ։͢Δํ๏ • Spring BootΞϓϦέʔγϣϯͷdependencyʹ 
 ActuatorͱMicrometerΛ௥Ճ͢Δ • application.propertiesʹͯActuatorͷΤϯυϙΠϯτΛެ։͢Δ

Slide 26

Slide 26 text

ActuatorͱMicrometerΛ௥Ճ • pom.xml ͷdependencyʹ࣍ͷ2ͭΛ௥Ճ • Actuator : spring-boot-starter-actuator • Spring Bootͷઃఆ΍ϝτϦΫεΛWeb APIͱͯ͠ެ։͢ΔͨΊͷϥΠϒϥϦ • Micrometer : micrometer-registry-prometheus • OS΍JavaVMɺSpring BootͷϝτϦΫε৘ใΛऔಘ͢ΔͨΊͷϥΠϒϥϦ • “-prometheus” ͸Prometheus͔ΒΞΫηε͢ΔͨΊͷΤϯυϙΠϯτΛ 
 ActuatorΛ༻͍ͯެ։͢Δɻ • “-datadog” ͱ͍͏ϥΠϒϥϦ͸ఆظతʹDataDogʹϝτϦΫεΛૹ৴͢ΔͳͲ 
 ϥΠϒϥϦʹΑͬͯڍಈ͕ҟͳΔ

Slide 27

Slide 27 text

ActuatorͱMicrometerΛ௥Ճ • pom.xml ͷdependencyʹ࣍ͷ2ͭΛ௥Ճ org.springframework.boot spring-boot-starter-actuator runtime io.micrometer micrometer-registry-prometheus runtime

Slide 28

Slide 28 text

ActuatorͷΤϯυϙΠϯτΛެ։ • application.properties ʹͯɺMicrometerʹؔ͢Δ 
 ActuatorͷΤϯυϙΠϯτΛެ։͢Δ management.endpoints.web.exposure.include=* management.metrics.distribution.percentiles-histogram.http.server.requests=true management.metrics.tags.application=${spring.application.name} # Micrometerʹؔ͢ΔϝτϦΫε͚ͩެ։͢Δ৔߹ʹ͸࣍ͷઃఆͰ΋ྑ͍ 
 # management.endpoints.web.exposure.include=prometheus

Slide 29

Slide 29 text

ActuatorͱMicrometerΛ௥Ճ • http://(ΞϓϦέʔγϣϯͷΞυϨε)/actuator/prometheus ʹ 
 ΞΫηε͢ΔͱɺϝτϦΫεҰཡΛऔಘͰ͖Δ process_ fi les_max_ fi les{application="bff",} 10240.0 process_uptime_seconds{application="bff",} 15057.411 jvm_threads_peak_threads{application="bff",} 43.0 jvm_threads_states_threads{application="bff",state="new",} 0.0 jvm_threads_states_threads{application="bff",state="runnable",} 11.0 jvm_threads_states_threads{application="bff",state="terminated",} 0.0 jvm_threads_states_threads{application="bff",state="waiting",} 12.0 jvm_threads_states_threads{application="bff",state="timed-waiting",} 9.0 jvm_threads_states_threads{application="bff",state="blocked",} 0.0

Slide 30

Slide 30 text

σϞ

Slide 31

Slide 31 text

PrometheusͰϝτϦΫεऩू • ʢGrafana΍Prometheusࣗମͷߏஙʹ͍ͭͯ͸ׂѪʣ • PrometheusͷઃఆϑΝΠϧʹSpring BootΞϓϦέʔγϣϯͷ 
 ΞυϨεΛྻڍ͠ɺϝτϦΫεΛऩूͰ͖ΔΑ͏ʹ͢Δ

Slide 32

Slide 32 text

PrometheusͰϝτϦΫεऩू • PrometheusͷઃఆϑΝΠϧʹSpring BootΞϓϦέʔγϣϯͷ 
 ΞυϨεΛྻڍ͠ɺϝτϦΫεΛऩूͰ͖ΔΑ͏ʹ͢Δ scrape_con fi gs: - job_name: "apps" metrics_path: "/actuator/prometheus" static_con fi gs: - targets: [ "host.docker.internal:9000", "host.docker.internal:9001", "host.docker.internal:9002", "host.docker.internal:9010" ]

Slide 33

Slide 33 text

Ϋϥ΢υωΠςΟϒ࣌୅ʹ 
 ΞϓϦέʔγϣϯͷΞυϨεΛ 
 ྻڍ͢Δͱ͔ਖ਼ؾͰ͔͢ʁ

Slide 34

Slide 34 text

Ϋϥ΢υωΠςΟϒ࣌୅ʹ 
 ΞϓϦέʔγϣϯͷΞυϨεΛ 
 ྻڍ͢Δͱ͔ਖ਼ؾͰ͔͢ʁ ͖ͪΜͱޙͰղઆ͠·͢

Slide 35

Slide 35 text

σϞ

Slide 36

Slide 36 text

GrafanaʹμογϡϘʔυΛ௥Ճ • Grafana͸ίϛϡχςΟ͕࡞੒ͨ͠μογϡϘʔυΛެ։͍ͯ͠Δ • https://grafana.com/grafana/dashboards/ • JVM (Micrometer) - 4701 ͕Φεεϝ

Slide 37

Slide 37 text

σϞ

Slide 38

Slide 38 text

Prometheus͸pullܕ • ΞϓϦέʔγϣϯଆ͔ΒσʔλΛૹΔ΋ͷΛʮPushܕʯͱݺͿ • ΞϓϦέʔγϣϯʹऔΓʹߦ͘΋ͷΛʮPullܕʯͱݺͿ • Prometheus͸ʮPullܕʯͰ͋ΓϝτϦΫεऩूͷϛυϧ΢ΣΞͱ 
 ͯ͠͸ΘΓͱ௝͍͠෦ྨʢ*ݸਓͷײ૝Ͱ͢ʣ • PullܕͰ΋෼ࢄΞϓϦέʔγϣϯͷϝτϦΫεΛ໰୊ͳ͘ 
 ऩूͰ͖Δʢޙ൒Ͱղઆ͠·͢ʣ

Slide 39

Slide 39 text

Prometheus͸pullܕ • Pullܕͷར఺ • ؂ࢹର৅ͱͳΔΞϓϦέʔγϣϯ͕໌֬Ͱ͋ΔͨΊ 
 ʮϝτϦΫεΛऔಘͰ͖ͳ͔ͬͨʯͱ͍͏໰୊Λݕग़͠΍͍͢ • PushܕͰ͸ʮૹΒΕͯ͜ͳ͔ͬͨʯ = ʮه࿥͠ͳ͍ʯ͚ͩͱ 
 ͳΓɺͦΕΛ໰୊ͱͯ͠ݕग़͢Δ͜ͱ͕೉͍͠ • ؂ࢹର৅͕ଟ͘ͳͬͨࡍʹɺPushܕͰ͸ϝτϦΫεऩूαʔό͕ύϯΫͯ͠ 
 ͠·͏Մೳੑ͕͋Δ͕ɺPullܕͳΒऩूස౓Λ੍ޚ͢Δ͜ͱ͕Ͱ͖Δʮ͔΋ʯ

Slide 40

Slide 40 text

Spring Boot + LokiͰ
 ϩάऩू

Slide 41

Slide 41 text

Spring BootͰϩάऩू • Spring BootΞϓϦέʔγϣϯͷϩάΛPromtailͰఆظతʹऩूͯ͠ 
 Lokiʹૹ৴͢ΔʢPullܕʣ • Grafana͕Loki͔ΒϩάΛऔಘͯ͠ՄࢹԽ͢Δ Spring Boot 
 Application ϩά 
 ϑΝΠϧ Loki ఆظతʹऩू Grafana औಘͯ͠ՄࢹԽ Promtail ૹ৴

Slide 42

Slide 42 text

Ͱ΋΋͏ɺϩάϑΝΠϧͱ͔ 
 ࢖͍ͨ͘ͳ͍Ͱ͢ΑͶ

Slide 43

Slide 43 text

Spring BootͰϩάऩू • Spring BootΞϓϦέʔγϣϯͷϩάΛLoki4JͷLogback AppenderΛ 
 ༻͍ͯఆظతʹLokiʹૹ৴͢ΔʢPushܕʣ • Grafana͕Loki͔ΒϩάΛऔಘͯ͠ՄࢹԽ͢Δ Spring Boot 
 Application Loki4J 
 Appender Loki ఆظతʹૹ৴ Grafana औಘͯ͠ՄࢹԽ

Slide 44

Slide 44 text

Loki4JͷLogback AppenderΛ௥Ճ • pom.xml ͷdependencyʹloki-logback-appenderΛ௥Ճ • ϑΝΠϧ΍ίϯιʔϧͰ͸ͳ͘LokiʹϩάΛૹΔΑ͏ͳ 
 LogbackͷAppender com.github.loki4j loki-logback-appender 
 1.3.2 runtime

Slide 45

Slide 45 text

LogbackͷઃఆϑΝΠϧΛ࡞੒ • LogbackͷઃఆϑΝΠϧΛ࡞੒͠ɺAppenderʹLoki4jAppenderΛ 
 ࢦఆ͢Δʢlogback-loki.xmlʣ http://localhost:3100/loki/api/v1/push app=${appName},host=${HOSTNAME},traceID=%X{traceId:-NONE},level=%level ${FILE_LOG_PATTERN} true

Slide 46

Slide 46 text

LogbackͷઃఆϑΝΠϧΛ༗ޮʹ͢Δ • application.properties ʹͯɺ࡞੒ͨ͠LogbackͷઃఆϑΝΠϧΛ 
 ༗ޮʹ͢Δ • ͍ͭͰʹɺޙͰτϨʔεऩूͷ࣌ʹ࢖͏τϨʔεID΋ 
 ϩάʹग़ΔΑ͏ʹ͓ͯ͘͠ logging.pattern.level=%5p [${spring.application.name:},%X{traceId:-},%X{spanId:-}] logging.con fi g=classpath:logback-loki.xml

Slide 47

Slide 47 text

LokiͰϩάऩू • ʢLokiࣗମͷߏஙʹ͍ͭͯ͸ׂѪʣ • LokiΛىಈ͢Ε͹ΞϓϦέʔγϣϯଆ͔ΒϩάΛPush͢ΔͷͰ 
 Lokiଆʹ͸ಛʹඞཁͳઃఆͳͲͳ͠

Slide 48

Slide 48 text

σϞ

Slide 49

Slide 49 text

ϩά͸push͔pull͔ • ྺ࢙తʹϩά͸PullܕͰऩू • ϑΝΠϧʹग़ྗ͠ɺόονͰωοτϫʔΫετϨʔδʹҠಈͤ͞Δ • Fluentd΍LogstashɺFilebeatͳͲͰऩूͯ͠ElasticsearchʹૹΔ • ϩάΛPushܕʹ͢Δͷ͸ෆ҆ • ϩά͸ʮγεςϜͷෆਖ਼ऴྃ௚લͷ࠷ظͷ੠ʯ • ΞϓϦέʔγϣϯϓϩηε͕ࢮΜͰ΋ϩά͚ͩ͸࢒͍ͨ͠ • PushܕͰ͸ड͚औΔଆͷϛυϧ΢ΣΞ͕ࢭ·Δͱϩά͕ࣦΘΕΔ

Slide 50

Slide 50 text

ͦΜͳʹॏཁͳ৘ใΛ
 ͳͥϩάͳͲͱ͍͏ᐆດͳํ๏Ͱ
 ࢒ͦ͏ͱ͢Δͷ͔ʁ

Slide 51

Slide 51 text

ϩάͷ֓೦Λݟ௚͢΂͖ • ఏݴ : ϩάͱ͍͏֓೦ͦͷ΋ͷΛݟ௚͢΂͖ • ϩά͸ॏཁͱ͞ΕΔΘΓʹݫີʹઃܭ͞Εͳ͍ࣄ͕ଟ͍ • ํ਑͚ܾͩΊͯޙ͸։ൃऀ೚ͤʢͦͷΘΓʹɺ༗༻Ͱͳ͚Ε͹ౖΒΕΔʣ • ॏཁͳ৘ใͱɺͦ͏Ͱͳ͍৘ใ͕ࡶଟʹࠞࡏ͢Δ • ຊ౰ʹॏཁͳ΋ͷ͸ɺϩάͰ͸ͳ͘ۀ຿σʔλͱͯ͠ଊ͑Δ΂͖ • ຊ౰ʹॏཁͳ৘ใ͸Կ͔Λਫ਼ࠪ͠ɺݫີʹઃܭͨ͠͏͑Ͱ 
 ৴པੑͷߴ͍σʔλετΞʹه࿥͢Δ • ϩάͱ͍͏࢓૊ΈʹཔΔ΋ͷ͸ʮ͋ͬͨΒخ͍͠ʯʹཹΊΔ΂͖

Slide 52

Slide 52 text

ϩά͸push͔pull͔ • ͦͷ্Ͱɺϩά͸Pushܕ͕ྑ͍ͷ͔ɺPullܕ͕ྑ͍ͷ͔ • Pullܕͷํ͕ॊೈੑ͕ߴ͍ • ऩूͯ͠Ճ޻͢Δͱ͍͏ϓϩηεΛܦΔ͜ͱ͕Ͱ͖Δ • ͦͷաఔʹKafka΍RabbitMQΛڬΜͰෛՙΛ෼ࢄͰ͖Δ • Pushܕͷํ͕༰қ • ϩʔΧϧPCͰ࣮ߦ͢Δࡍʹ΋ɺϩάϑΝΠϧΛग़ྗ͠ͳͯ͘ࡁΉ • Ճ޻͕ඞཁͳ͍ʢ΋͘͠͸JSONܗࣜͰग़ྗ͢Δʣ৔߹΍ 
 ϩάऩूαʔόʹߴෛՙֻ͕͔ΔΑ͏ͳ݅਺Ͱ͸ͳ͍৔߹ͳͲͰ͸༗ޮ

Slide 53

Slide 53 text

Spring Boot + TempoͰ
 τϨʔεऩू

Slide 54

Slide 54 text

Spring BootͰτϨʔεऩू • τϨʔεऩूͷͨΊʹ͸࣍ͷ2͕ͭඞཁ • ᶃτϨʔεIDͷ఻೻ • ʮͲͷϦΫΤετʹඥ෇͘ॲཧ͔ʯʮ୭͔Βݺ͹Ε͔ͨʯͱ͍͏ ৘ใΛΞϓϦέʔγϣϯؒͰ఻೻ͤ͞Δ • ᶄτϨʔε৘ใͷ஝ੵ • τϨʔε৘ใΛσʔλετΞʹૹͬͯ஝ੵ͢Δ

Slide 55

Slide 55 text

Spring BootͰτϨʔεIDͷ఻೻ • Spring BootΞϓϦέʔγϣϯͷؒͷ௨৴ʢHTTP΍AMQPʣͷ 
 ϔομʹτϨʔεIDΛೖΕͯૹ৴ɺड৴͢Δ Spring Boot 
 Application1 Spring Boot 
 Application2 Spring Boot 
 Application4 Spring Boot 
 Application3 traceparent 
 = req001-01 traceparent 
 = req001-02 traceparent 
 = req001-01

Slide 56

Slide 56 text

Spring BootͰτϨʔεIDͷ఻೻ • τϨʔεIDͷૹड৴ʹ͸Spring Boot 2.x·Ͱ͸Spring Cloud Sleuth Λ༻͍͍͕ͯͨɺSpring Boot 3.0͔ΒMicrometer Tracing͕ 
 ར༻Ͱ͖ΔΑ͏ʹͳͬͨ Spring Boot 
 Application Micrometer 
 Tracing RestTemplate Spring Boot 
 Application Micrometer 
 Tracing Rest Controller HTTPϔομʹ 
 τϨʔεIDͷ෇༩ 
 ʢࣗಈʣ HTTPϔομ͔Β 
 τϨʔεIDͷऔಘ 
 ʢࣗಈʣ

Slide 57

Slide 57 text

Micrometer Tracingͷ௥Ճ • pom.xml ͷdependencyʹ micrometer-tracing-bridge-otel ͔ 
 -brave Λ௥Ճ͢ΔʢͲͪΒͰ΋ৼΔ෣͍ʹҧ͍͸ͳ͍ʣ • Spring BootͷRestTemplate΍Spring Cloud StreamΛ༻͍ͨ௨৴Ͱ 
 τϨʔεIDͷ఻೻͕ߦΘΕΔʢࣗಈʣ io.micrometer micrometer-tracing-bridge-otel runtime

Slide 58

Slide 58 text

Spring BootͰτϨʔεऩू • Spring BootΞϓϦέʔγϣϯͷτϨʔε৘ใͷه࿥Λ 
 Open Telemetry͔BraveͷϥΠϒϥϦΛ༻͍ͯTempoʹૹ৴͢Δ • Grafana͕Tempo͔ΒτϨʔε৘ใΛऔಘͯ͠ՄࢹԽ͢Δ Spring Boot 
 Application Open 
 Telemetry Tempo ఆظతʹૹ৴ Grafana औಘͯ͠ՄࢹԽ

Slide 59

Slide 59 text

Open Telemetryͷ௥Ճ • Spring Boot͔ΒτϨʔε৘ใΛૹ৴͢Δํ๏ • pom.xml ͷdependencyʹ opentelemetry-exporter-zipkin ͔ 
 zipkin-reporter-brave Λ௥Ճ͢ΔʢৼΔ෣͍ʹҧ͍͸ͳ͍ʣ • ZipkinʹτϨʔε৘ใΛૹΔϥΠϒϥϦ 
 ʢTempo͕Zipkinޓ׵APIΛඋ͍͑ͯΔͨΊར༻Ͱ͖Δʣ io.opentelemetry opentelemetry-exporter-zipkin runtime

Slide 60

Slide 60 text

LogbackͷઃఆϑΝΠϧΛ༗ޮʹ͢Δ • application.properties ʹͯɺτϨʔε৘ใΛͲΕ͘Β͍ͷׂ߹Ͱ TempoʹૹΔ͔Λઃఆ͢ΔʢσϑΥϧτ͸10%ʣ • ։ൃத΍σϞͰ͸100%ʹͨ͠ํ͕ྑ͍ • ӡ༻தͷΤϥʔൃੜ࣌ʹ༨ͣ͞τϨʔεΛऔಘ͢ΔͨΊʹ͸ 
 100%ʹઃఆ͓ͯ͘͠ͱྑ͍ management.tracing.sampling.probability=1.0

Slide 61

Slide 61 text

TempoͰτϨʔεऩू • ʢTempoࣗମͷߏஙʹ͍ͭͯ͸ׂѪʣ • TempoͷઃఆϑΝΠϧʹͯɺZipkinޓ׵APIΛ༗ޮʹ͢Δ server: http_listen_port: 3200 distributor: receivers: zipkin:

Slide 62

Slide 62 text

σϞ

Slide 63

Slide 63 text

τϨʔε͸pushͷΈ • Zipkinʢޓ׵APIؚΉʣ͸τϨʔε৘ใΛड͚औΔͨΊͷAPIΛ 
 ެ։͓ͯ͠Γ֤ΞϓϦέʔγϣϯ͔Βͦ͜ʹτϨʔε৘ใΛૹΔ 
 ʮPushܕʯͷΞʔΩςΫνϟ͕࠾༻͞Ε͍ͯΔ • ͍ͣΕPullܕͷΤʔδΣϯτ͕ग़ͯ͘Δ͔΋஌Εͳ͍

Slide 64

Slide 64 text

Spring Boot 3.0 + k8sͷ
 ϞχλϦϯά

Slide 65

Slide 65 text

k8s্ͰPullܕΛӡ༻͢Δ՝୊ • PrometheusͰ͸ϞχλϦϯάର৅ͷΞυϨεΛྻڍͨ͠ • k8sͷΑ͏ʹಈతʹϞχλϦϯάର৅͕૿ݮɾมԽ͢Δ৔߹ʹ͸ 
 ͲͷΑ͏ʹऩू͢Δͷ͔ʁ

Slide 66

Slide 66 text

αʔϏεσΟεΧόϦ • ΞϓϦέʔγϣϯͷϧʔςΟϯά΍ϩʔυόϥϯγϯάͷͨΊʹ 
 ༻͍ΒΕΔαʔϏεσΟεΧόϦͷ࢓૊ΈΛϞχλϦϯάͰ΋ར༻ ͢Δ • k8sͰ͸CoreDNSΛ༻͍ͨαʔϏεσΟεΧόϦΛߦ͏

Slide 67

Slide 67 text

αʔϏεσΟεΧόϦ • 20ඵͰ෼͔ΔαʔϏεσΟεΧόϦ Spring Boot 
 Application1 Service 
 Registry Spring Boot 
 Application1 Spring Boot 
 Application1 Spring Boot 
 Application1 ᶃΞυϨεͷొ࿥ ᶄݺͼग़͍ͨ͠αʔϏεͷΞυϨεΛऔಘ ᶅݺͼग़͠ Spring Boot 
 Application2

Slide 68

Slide 68 text

αʔϏεσΟεΧόϦ • αʔϏεσΟεΧόϦΛར༻ͨ͠ϞχλϦϯά Spring Boot 
 Application1 Prometheus Promtail Service 
 Registry Spring Boot 
 Application1 Spring Boot 
 Application1 Spring Boot 
 Application ᶃΞυϨεͷొ࿥ ᶄϞχλϦϯά͍ͨ͠αʔϏεͷΞυϨεΛऔಘ ᶅݺͼग़͠

Slide 69

Slide 69 text

PrometheusͷαʔϏεσΟεΧόϦઃఆ • PrometheusͷઃఆϑΝΠϧʹͯɺkubernetes_sd_con fi g ͰαʔϏεσΟεΧόϦΛઃఆ͢Δ scrape_con fi gs: - job_name: "apps" metrics_path: "/actuator/prometheus" kubernetes_sd_con fi g: - role: pod 
 relabel_con fi gs: - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape] action: keep regex: true - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path] action: replace target_label: __metrics_path__ regex: (.+) - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port] action: replace target_label: __address__ regex: (.+?)(?::\d+)?;(\d+) replacement: $1:$2 ଞʹ΋৭ʑͱઃఆ͕ඞཁ͔ͩΒ 
 Prometheus΍Promtail͸ 
 helmͰΠϯετʔϧ͢Δͷ͕Φεεϝ

Slide 70

Slide 70 text

PrometheusͷαʔϏεσΟεΧόϦઃఆ • ΞϓϦέʔγϣϯΛσϓϩΠ͢ΔϚχϑΣετϑΝΠϧʹͯ 
 annotationsͰϙʔτ൪߸΍ActuatorͷύεΛࢦఆ͢Δ spec: template: metadata: labels: app: bff annotations: prometheus.io/scrape: "true" prometheus.io/port: "9000" prometheus.io/path: /actuator/prometheus

Slide 71

Slide 71 text

σϞ

Slide 72

Slide 72 text

ίϯςφ࣌୅Ͱ΋PullͰେৎ෉ • ϚΠΫϩαʔϏεͷΑ͏ʹಈతʹ૿ݮ͢ΔΞϓϦέʔγϣϯ܈Λ 
 ϞχλϦϯά͢ΔࡍɺαʔϏεσΟεΧόϦͷ࢓૊Έͱซ༻͢Ε͹ 
 PullܕͰ΋ಛʹ໰୊ͳ͘ϞχλϦϯά؀ڥΛߏஙͰ͖Δ

Slide 73

Slide 73 text

·ͱΊ

Slide 74

Slide 74 text

·ͱΊ • ϝτϦΫεɺϩάɺτϨʔε͸औΕΔ͚ͩऔΓ·͠ΐ͏ • GrafanaελοΫͰ͋ΔɺGrafanaɺPrometheusɺLokiɺTempoͰ 
 ·ͱΊͯϞχλϦϯάͰ͖Δ • Spring Boot 3.0Ͱ͸ϞχλϦϯάपΓͷϥΠϒϥϦͳͲ΋੔ཧ͞Ε 
 GrafanaελοΫͳͲͰϞχλϦϯά͠΍͘͢ͳ͍ͬͯΔ • ϩά͔ΒτϨʔε΁ͷભҠͳͲɺ͜ΕΒͷ3ͭΛͦΕͧΕ 
 ભҠͰ͖ΔΑ͏ʹ͢ΔτϨϯυ͕͋Δ

Slide 75

Slide 75 text

·ͱΊ • PushܕɺPullܕͷ͍ͣΕͰ΋໰୊ͳ͘σʔλΛऩूͰ͖Δ • ૯ͯ͡ɺPushܕͷํ͕༰қɺPullܕͷํ͕ॊೈͱͳ͍ͬͯΔ • DataDog΍AWS CloudWatchͳͲ͸جຊతʹPushܕ • ΞϓϦέʔγϣϯଆͷιʔείʔυ΍ઃఆϑΝΠϧ͕ 
 Πϯϑϥʹґଘ͠ͳ͍Α͏ཹҙ͢Δͱ։ൃ͠΍͍͢Α

Slide 76

Slide 76 text

Resources • ιʔείʔυ • https://github.com/cero-t/spring-store-2022 • υΩϡϝϯτ • Spring Boot 3.0ΞϓϦέʔγϣϯΛGrafanaελοΫͰՄࢹԽͯ͠Έͨɻ • https://cero-t.hatenadiary.jp/entry/2023/01/10/163547 • GrafanaελοΫʹΑΔSpring BootΞϓϦέʔγϣϯ؂ࢹͷৄࡉʢͦͷ1 Grafana + Prometheusฤʣ • https://cero-t.hatenadiary.jp/entry/2023/01/11/181715 • GrafanaελοΫʹΑΔSpring BootΞϓϦέʔγϣϯ؂ࢹͷৄࡉʢͦͷ2 Grafana + Lokiฤʣ • https://cero-t.hatenadiary.jp/entry/2023/01/12/130022 • GrafanaελοΫʹΑΔSpring BootΞϓϦέʔγϣϯ؂ࢹͷৄࡉʢͦͷ3 Grafana + Tempoฤʣ • https://cero-t.hatenadiary.jp/entry/2023/01/15/203442