Suggested Topicの質問に可能な限り答えてみた

Slide 1

Slide 1 text

Japan Datadog User Group Meetup#1 ABEJA, Inc. Shogo Muranushi Suggested Topicͷ࣭໰ʹՄೳͳݶΓ౴͑ͯΈͨ

Slide 2

Slide 2 text

@rwle1212 2

Slide 3

Slide 3 text

@rwle1212 3 ໊લ ɿ ଜओ ૖ޛʢΉΒ͵͠ ͠ΐ͏͝ʣ ձࣾ ɿ ABEJA, Inc. ۀ຿ ɿΠϯϑϥΤϯδχΞ -> ϓϩμΫτΦʔφʔ -> SRE

Slide 4

Slide 4 text

@rwle1212 4 Datadogྐྵ 2೥΄ͲɻInfra ͱ Logs ͷར༻͕ϝΠϯɻ Datadog͸ࡉ͔͍ػೳ͕ଟ͘ɺͿͬͪΌ͚Α͘෼͔ͬͯͳ͍ɻ ࠓ೔࿩͢͜ͱ΋͋͘·Ͱฐࣾͷ࢖͍ํͳͷͰਖ਼ղͰ͸ͳ͍ɻ ৭ʑ஌Γ͍ͨͷͰΈΜͳͰ৘ใ࣋ͪدΓͨ͘ɺ  ·ͣ͸ొஃͯ͠Έͨɻ

Slide 5

Slide 5 text

@rwle1212 5

Slide 6

Slide 6 text

@rwle1212 6

Slide 7

Slide 7 text

@rwle1212 7 Datadogར༻ن໛ײ docker.containers.running Datadog Logs by weekly

Slide 8

Slide 8 text

@rwle1212 8 ຊ୊ʹೖΓ·͢

Slide 9

Slide 9 text

@rwle1212 9 ࢖͍ͬͯΔػೳ΍ͦͷλΠϛϯά  (ʮৼΓฦΓͰ͜ΜͳμογϡϘʔυΛ࢖͍ͬͯΔʯͳͲ) • Α͘͞ΘΔػೳ͸LogsͱNotebooks • ཧ༝΍λΠϛϯά͸ޙड़

Slide 10

Slide 10 text

@rwle1212 10 μογϡϘʔυͲ͏΍ͬͯઃܭͯ͠·͔͢ʁ ΅͘ͷ͔Μ͕͍͖͑ͨ͞ΐ͏ͷμογϡϘʔυ ͲΜͳDashboardͰӡ༻͍ͯ͠Δ͔ɻ εϚʔτͳμογϡϘʔυͷ࡞Γํ μογϡϘʔυ͸׶͑ͯ࢖ͬͯ·ͤΜɻ ཧ༝͸ʮ΅͘ͷ͔Μ͕͍͖͑ͨ͞ΐ͏ͷμογϡϘʔυʯʹͳͬͯ͠·͏͔ΒͰ͢ɻ

Slide 11

Slide 11 text

@rwle1212 11 ࣦഊྫ1: ݁ہCloudWatchΛݟ͍ͯͨ ALB -> Container -> EC2 -> RDSͱ͍͏ϦΫΤετͷྲྀΕͰμογϡϘʔυΛ࡞ͬ ͯΈͨɻ μογϡϘʔυͷେ͖͞͸ܾ·͍ͬͯΔɺ׌ͭάϥϑ͕ଟ͗ͯ͢΋ෳࡶʹͳΔͷ ͰɺεϚʔτʹগͳΊʹάϥϑΛ഑ஔͨ͠ɻ ͔͠͠ো֐࣌ʹ৘ใ͕଍Γͳ͍ͷͰɺ݁ہCloudWatchΛோΊ͍ͯͨ͜ͱʹؾ෇͍ ͨɻ ͔ͦ͜ΒඞཁͳάϥϑΛ଍͍ͯ͘͠ͱͲΜͲΜෳࡶʹͳΓɺଞͷϝϯόʔ͸ʮԿΛ Ͳ͏ݟͨΒྑ͍ͷ͔Θ͔Βͳ͍ʯঢ়ଶʹͳͬͨɻ ݁ہCloudWatchΛோΊ͍ͯͨ͜ͱʹؾ෇͍ͨɻ

Slide 12

Slide 12 text

@rwle1212 12 ࣦഊྫ2:   μογϡϘʔυͷछྨ͕૿͑ΔͱΑ͘Θ͔Βͳ͘ͳΔ

Slide 13

Slide 13 text

@rwle1212 13 ղܾྫ  NotebookΛ࢖ͬͯΈͨ • ো֐࣌ɺௐࠪ࣌ʹඞཁͳಛఆͷάϥϑ ͕഑ஔ͞Ε͍ͯΔෛՙ֬ೝ༻ͷ NotebookΛ࡞੒ • ܦա؍࡯ΛݟΔͨΊͷάϥϑ͕഑ஔ͞ Ε͍ͯΔNotebookΛ࡞੒ ͳͲ

Slide 14

Slide 14 text

@rwle1212 14 ղܾྫ  NotebookΛ࢖ͬͯΈͨ • Jupyter Notebook Likeͳ࢖͍ํ͕Ͱ͖ ͯɺίϝϯτͱάϥϑΛॻ͚Δ • ಛఆͷো֐ʹରͯ͠ௐ΂ͨ͜ͱɺਪଌ ͳͲͷίϝϯτΛॻ͖ͳ͕Βه࿥͕࢒ ͤΔ • ϩάͷάϥϑԽ΋༰қ • Issueʹը૾ΛషΓ෇͚ΔΑΓεϜʔ ζʹ·ͱΊΒΕΔͷͰڞ༗ָ͕͘͢͝

Slide 15

Slide 15 text

@rwle1212 15 ղܾྫ  NotebookΛ࢖ͬͯΈͨ • ͭ·Γ൚༻తͳμογϡϘʔυ͸࡞੒ͤͣʹɺCloudWatchͳͲͷશମΛ၆ᛌͰ ͖ΔϝτϦΫεΛݟͭͭɺϩάΛάϥϑԽͨ͠Γͯ͠ɺಛఆͷো֐ʹ߹Θͤͯ ඞཁͳάϥϑΛ഑ஔͨ͠NotebookͰௐࠪ΍ܦա؍࡯Λߦ͍ͬͯΔ

Slide 16

Slide 16 text

@rwle1212 16 ͦ΋ͦ΋Έͳ͞ΜͲͷػೳԡ͠? ͱ͍͏͜ͱͰNotebooksΛਪͯ͠·͢ɻ ͋ͱLogs΋ྑ͍ࢠͰ͢ɻޙड़Ͱ·ͱ·͍ͬͯ·͢ɻ

Slide 17

Slide 17 text

@rwle1212 17 ௿ϦΫΤετ࣌ͷrateΞϥʔτ཈੍ํ๏ʹ͍ͭͯ ௿ϦΫΤετͷ؂ࢹ͸ͯ͠·ͤΜ

Slide 18

Slide 18 text

@rwle1212 18 terraformҎ֎Ͱͷ؅ཧํ๏ ओʹຊ൪༻͸TerraformɻͱΓ͋͑ͣͷ࣌͸खͰ࡞Δɻ खͰ࡞ͬͨͷͱTerraformͰ࡞ͬͨ΋ͷ͕ࠞࡏ͢Δͱɺຊ൪༻ͳͷ͔ɺΞϥʔτΛ ์ஔͯ͠ྑ͍ͷ͔Α͘෼͔Βͳ͘ͳΔͷͰʮCreateByTerraformʯTagΛ෇༩ͯ͠൑ ผ͍ͯ͠Δ

Slide 19

Slide 19 text

@rwle1212 19 ՝ۚ؅ཧʹؔ͢Δେٕখٕ ಛʹେٕ͸ແ͠ɻখωλ͕͋Δͱͨ͠Β • Multi Org؀ڥ͸Billing͕ू໿͞Εͯ͠·͏ɻʮLast Month’s UsageʯͰར༻ྔʢྉ ۚͰ͸ͳ͍ʣ͕Orgຖʹग़ྗ͞ΕΔͷͰ҈෼͍ͯ͠ΔɻӦۀ͞Μʹݴ͑͹Billing΋ ෼͚ͯ͘ΕΔ͔΋ʁखؒͳͷͰͦ͜·Ͱ΍ͬͯͳ͍ • AWSΠϯςάϨʔγϣϯͰͨ͘͞ΜͷαʔϏεͷϝτϦΫεΛऔΕΔΑ͏ʹ͢Δ ͱɺCloudWatch ͷ GetMetricData API ͕ΊͪΌͪ͘Όݺͼग़͞ΕΔͷͰ՝͕ۚ େ͖͘૿͑Δɻ·ͣ͸ඞཁͳαʔϏε͚ͩߜͬͯऔΓࠐΉͷ͕٢ɻ  SQSͰࢮ๢ͨ͠ܦݧ༗Γ

Slide 20

Slide 20 text

@rwle1212 20 ՝ۚ؅ཧʹؔ͢Δେٕখٕ • Logs͸ʮͲΕ͕Կ݅ʯͱ͔ݟΕΔͷͰɺͨ·ʹෆཁͳϩά͕ແ͍͔֬ೝͯ͠ ʮIndexsʯͰআ֎ϑΟϧλΛ࡞੒ͯ͠ϩάͷ՝ۚΛ཈͑ͨΓ͢Δ  ʢIndexed Logsͷ՝ۚ͸ࢭ·Δ͚ͲɺIngested Logsͷ՝ۚ͸ࢭ·Βͳ͍ͷͰ஫ҙɻ͍͔҆Βؾʹ͢Δ ΄ͲͰ͸ͳ͍͚Ͳʣ

Slide 21

Slide 21 text

@rwle1212 21 DBϞχλϦϯάͷTips͋Ε͜Ε CloudWatchΛIntegrationͯ͠ΔҐͰͦΕҎ্ͷ͜ͱ͸͓ͯ͠Γ·ͤΜ

Slide 22

Slide 22 text

@rwle1212 22 GoogleAnalyticsAPIར༻ͨ͠ݱࡏϢʔβʔ਺ͱ͔࠾औͯ͠ ͍Δࣄྫ͕͋Ε͹ GoogleAnalyticsAPI͸࢖༻͓ͯ͠Γ·ͤΜ

Slide 23

Slide 23 text

@rwle1212 ࡢ೥ͷ12݄ʹҎԼͷهࣄΛॻ͖·ͨ͠ 23 ଞͷ؂ࢹSaaS౳ʢࣗ෼͕஌ΔൣғͰ͸Zabbixɾ PrometheusɾNew RelicɾMackerelʣͱͷൺֱ

Slide 24

Slide 24 text

@rwle1212 24 ʲཁ໿ʳͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ • Datadog͔ΒPrometheus΁৐Γ׵͑ • ݩʑDatadogΛ࢖͍ͬͯͨɻ$18/host͕ߴ͔ͬͨ͠ɺKubernetesͷ؂ࢹͷσϑΝΫ τʹͳΓͭͭ͋ΔPrometheusΛࢼ͢໨త΋ؚΊͯ࢖͍࢝Ίͨɻ • ൒೥ܦաɻPrometheusࣗମͷো֐͕νϥϗϥ • OSSͰ৘ใগͳ͍৔߹͸ௐࠪʹ͕͔͔࣌ؒΔɻଟ͍࣌͸िʹ1,2ճ΄Ͳ໰୊͕ൃੜ ͠1,2ਓ͕਺࣌ؒ/ि΄ͲऔΒΕ͍ͯͨ • ελʔτΞοϓ͸ਓ͕͍ͳ͍ɻຊۀͷ։ൃʹूத͍ͨ͠ • πʔϧͷ͓कΓʹ͕࣌ؒऔΒΕ͍ͯͨɻPrometheusͷӡ༻Λ֎෦ʹҕୗ͍ͨ͠ɻ ӡ༻Λ֎෦ʹҕୗ͢Δͱ͍͘ΒʹͳΔ͔෼͔Βͳ͍͚Ͳɺࠓͷن໛ΛDatadogʹม ͑Δͱ10ສ/݄͘Β͍ͩͬͨɻɾɾDatadogͰΑ͘Ͷʁͱࢥ͍࢝ΊΔ

Slide 25

Slide 25 text

@rwle1212 25 ʲཁ໿ʳͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ • ϩάͷऩूͲ͏͢Δ໰୊ • CloudWatch LogsΛ࢖͍͕ͬͯͨݕࡧੜօແɻʢ౰࣌͸ʣPrometheus͸ϩάऩू ͢Δػೳ͸ͳ͍ɻσϑΝΫτతʹ͸ ElasticSearch + Kibana ͩΖ͏ • ͔͠͠ɺͦͷ࣌఺Ͱϩάྔ͕5ԯ/݄ɻঃʑʹର৅ͷΞϓϦ΍νʔϜΛ૿΍ͦ͏ͱࢥ ͏ͱ਺ϲ݄Ͱഒʑʹ૿͍͑ͯ͘͜ͱ͕૝ఆ͞Ε͍ͯͨ • ͦΕΛElasticSearchͰड͚ͳ͕Βӡ༻͢Δ͜ͱΛ૝ఆ͢Δͱɺख͕औΒΕͦ͏ײ͕ ൒୺ͳ͔ͬͨ • Datadog΋ϩάͷαʔϏεΛఏڙ࢝͠Ί͍ͯͨͷͰ࢖ͬͯΈΔ͜ͱʹ

Slide 26

Slide 26 text

@rwle1212 26 ʲཁ໿ʳͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ • ϩάͲ͏͢Δ͔ͷ݁࿦ • Facets ͕ڧྗͰɺಋೖָ͕ɺίετίϯτϩʔϧ΋͠΍͘͢ɺԿ͔͋Ε͹શϩά Λ௥͑Δɻͦͷ্Ͱɺϩάऩू͢Δ্ͰΠϯϑϥΛؾʹ͠ͳͯ͘ྑ͍ͷ͕ؾʹೖ ΓɺঃʑʹϩάΛ Datadog Logs ʹدͤ·ͨ͠ • ͦͯ͠ɺϞχλϦϯά΋ Promethes ͔Β Datadog ΁Ҡߦ • ͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ɺͷ·ͱΊ • ϩά΋ϞχλϦϯά΋ ͦΕ͕ࣄۀͷίΞͰ͋Ε͹ͪΌΜͱߏங͠ӡ༻͢Δ͜ͱࣗ ମ͸ԀΘͳ͍ͷͰ͕͢ɺίΞͰͳ͍ͳΒ࣌ؒΛׂ͖ͨ͘ͳ͍

Slide 27

Slide 27 text

@rwle1212 ZabbixɺPrometheus ๻ͷ؍఺͔Β͢ΔͱɺػೳੑͰ͸ͳ͘ӡ༻ੑͰ൑அ͍ͯ͠ΔͷͰOSSܥ͸Α΄ Ͳಛ௃తͳػೳ΍ཁ͕݅ͳ͍ݶΓੵۃతͳ࠾༻͸ආ͚͍ͨͰ͢ 27 ଞͷ؂ࢹSaaS౳ʢࣗ෼͕஌ΔൣғͰ͸Zabbixɾ PrometheusɾNew RelicɾMackerelʣͱͷൺֱ

Slide 28

Slide 28 text

@rwle1212 New RelicɺMackerel ྆ํͱ΋௚઀ݕূ͸͍ͯ͠ͳ͍ͷͰൺֱʹ͸ͳΒͳ͍Ͱ͕͢ɺݕূ͠ͳ͔ͬͨ ཧ༝ͷҰͭͱͯ͠ɺDatadogͰ͸ʮMonitoring Modern Infrastructureʯͱ͍͏ه ࣄ͕ެ։͞Ε͍ͯ·͢ɻ  ͜͜ʹॻ͔Ε͍ͯΔ಺༰͕ͱͯ΋ཧʹ͔ͳ͍ͬͯͨͷͰɺ͜ͷΑ͏ͳࢥ૝ʹ ΑͬͯαʔϏε͕ઃܭ͞Εࠓޙ΋αʔϏεΛ௥Ճ͍ͯ͘͠ΜͩΖ͏ͳͱࢥ͍ɺ ࠾༻ͨ͠എܠ΋͋Γ·͢ɻ 28 ଞͷ؂ࢹSaaS౳ʢࣗ෼͕஌ΔൣғͰ͸Zabbixɾ PrometheusɾNew RelicɾMackerelʣͱͷൺֱ

Slide 29

Slide 29 text

@rwle1212 29 SLOؔ࿈ͷϞχλʔ࡞੒ݕ౼Ͱࠔͬͨ͜ͱɾ޻෉ͨ͜͠ͱ ͋·Γ೉͍͠ػೳͰ͸ͳ͍ͷͰಛʹͳ͠ʁ ڧ͍ͯ͋͛ΔͳΒʮԿΛSLOͱͯ͠ܭଌ͢Δ͔ʯͷɺͦ΋ͦ΋ͷSLOͷఆ͕ٛ೉͍͠ ͘Β͍Ͱ͢ɻ

Slide 30

Slide 30 text

@rwle1212 30 Datadogͷ෍ڭ׆ಈ ʢͲͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ʁʣ • ޮՌ͋ͬͨ͜ͱ • Dev/Stg/ProdͷશΞϓϦͷϩάΛશ෦ू໿ͨ͠ɻ1ͭͷWebαΠτΛݟʹ ͍͘͜ͱͰશ෦ͷϩά͕ݟΕΔঢ়ଶʹͨ͠ɻࠓ·Ͱ͸؀ڥຖʹݟΔ৔ॴ͕ ෼͔ΕͯͨΓɺग़ͯͨΓग़ͯͳ͔ͬͨΓ • ͔ͦ͜ΒঃʑʹDatadogͷ࢖͍ํΛೝࣝ͠ɺMonitors΍NotebooksɺAPM ʹϝϯόʔͷڵຯ͕޿͕ͬͯߦͬͨ

Slide 31

Slide 31 text

@rwle1212 31 Datadogͷ෍ڭ׆ಈ ʢͲͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ʁʣ • ޮՌ͋·Γͳ͔ͬͨ͜ͱ • Datadogͷ࢖͍ํษڧձɻʮ΁ʔɻ͜Μͳ͜ͱͰ͖ΔΜͩʯͱ͍͏͚ͩͰखΛ ಈ͔͞ͳ͍ͷͰฉ͍ͯ΋๨ΕΔ • ϩάɺϝτϦΫεΛՄೳͳݶΓू໿ͯ͠ɺඞཁʹۦΒΕͯࣗ෼Ͱௐ΂ͯ޻ ෉ͯ͠΋Β͏ํ͕͕֮͑ྑͦ͞͏ͳؾ͕ͨ͠ɻݴ͏ͯ΋೉͘͠ͳ͍ͷͰ • ͋Δఔ౓ͨ͠Βʮ͓Εͷ͔Μ͕͍͖͑ͨ͞ΐ͏ͷDatadogڞ༗ձʯΛ֤ϝ ϯόʔͰখωλతʹ΍Δͷ΋ѱ͘͸ͳ͔ͬͨ

Slide 32

Slide 32 text

@rwle1212 32 Datadogͷ෍ڭ׆ಈ ʢͲͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ʁʣ • Ͳͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ • ࢖͍ͬͯΔͷ͸ΠϯϑϥɺόοΫΤϯυ • ϑϩϯτΤϯυͷE2E؂ࢹܥ͸গ͠ߴ͍ͷͰ࢖͍ͬͯͳ͍ • Real User MonitoringɺSynthetic Monitoring (Browser)

Slide 33

Slide 33 text

@rwle1212 33 ΠϯςάϨʔγϣϯͰ޻෉͍ͯ͠Δ͜ͱʢSlack / JIRAͱ૊ Έ߹ΘͤͯɺΠϯγσϯτ؅ཧɾ໰୊؅ཧʹར༻ͳͲʣ Warning͸ʮSlack௨஌ʯ Alert͸ʮSlack௨஌ʯ+ʮPagerDutyͰి࿩ʯ+ʮGitHub Issue࡞੒(Zapierܦ༝)ʯ

Slide 34

Slide 34 text

@rwle1212 34 Ξϥʔτͷ௨஌Ϩϕϧ؅ཧ ʢฐࣾ͸warn / error / fatal ͳͲʣ • WarnͱErrorͷΈ • ʮαʔϏεʹ௚઀Өڹ͕Ͱ͍ͯΔϞϊɺͰ͍ͯΔՄೳੑ͕͋ΔϞϊʯΛΞϥʔτ ͱͯ͠ઃఆ͍ͯ͠Δ • ྫʣCPU΍ϝϞϦͷ࢖༻཰Λ؂ࢹͱ͔͸͍ͯ͠ͳ͍ɻͦΕΒͷӨڹʹΑͬ ͯALBͰ5xx͕සൃ͢Ε͹Ξϥʔτ͕ඈͿΑ͏ʹ͍ͯ͠ΔɻDynamoDBͷ εϩοτϦϯάͱ͔

Slide 35

Slide 35 text

@rwle1212 35 datadog logsʹ͍ͭͯ࢖༻ײ΍ࣄྫ

Slide 36

Slide 36 text

@rwle1212 36 facets • JSONͰૹΕ͹উखʹύʔεͯ͘͠ΕΔ • ύʔε͞Εͨ஋Λfacets(index)Ͱ͖ΔͷͰɺϩ άͷQuery΍Group ByͳͲʹར༻Ͱ͖Δ

Slide 37

Slide 37 text

@rwle1212 37 facets • Կ͕ѱ͍ͷ͔Θ͔Βͳ͍͚ͲɺindexͰ͖ͳ͍ ͕࣌͋Δɻ୭͔ղܾํ๏Λڭ͑ͯཉ͍͠ • ͜ͷέʔεͷ৔߹͸਺஋Ҏ֎ͷจࣈྻ͕ೖͬ ͪΌͬͯΔͷ͔ͳɾɾʁͬ͟ͱோΊͨײͦ͡Μ ͳ͜ͱͳ͍͚ͲɺͦΕΛ؆୯ʹݟΔʹ͸Ͳ͏͠ ͨΒྑ͍ΜͩΖ͏ɾɾ

Slide 38

Slide 38 text

@rwle1212 38 Graph by Logs • ϩάΛάϥϑԽͰ͖Δ • facetsʹొ࿥ͨ͠஋ΛGroup Byʹࢦఆͨ͠Γɺ CountʹೖΕΔ͜ͱ΋Ͱ͖Δ • ࠨਤͷྫͰ͸ɺOrganization NameຖͷϦΫΤ ετ਺΍ฏۉԠ౴࣌ؒͳͲΛදࣔɻίϯςφID ຖʹݟͯภΓ͕ແ͍͔ݟͨΓ΋ͯ͠Δ • γεςϜܥ͚ͩͰ͸ͳ͘ΞϓϦͷར༻ঢ়گ෼ੳ ʹ΋࢖༻ՄೳͳͷͰɺΞϓϦଆͰ޷͖ͳ஋Λग़ ͤ͹৭ʑͳ؍఺Ͱ෼ੳͰ͖Δ • ᮢ஋Λઃ͚ͯΞϥʔτൃ๒Ͱ͖ΔͷͰ1ͭͷγ εςϜͰϝτϦΫεɾϩάͷ؂ࢹΛ؅ཧͰ͖Δ

Slide 39

Slide 39 text

@rwle1212 39 Graph by Logs • େ͖͍มߋΛͨ࣌͠͸εςʔλείʔυͷׂ߹ Λ֬ೝ • ઃఆํ๏ • Group By: @http.status_code • Display: Areas • Stacked as: % of overall count: *

Slide 40

Slide 40 text

@rwle1212 40 Pipelines • JSON͡Όͳ͍ϩάΛύʔεͨ͠ΓɺDatadog ্Ͱʮ͜ͷϑΟʔϧυ͸εςʔλεѻ͍͢Δʯ ͱ͍͏Α͏ͳϚοϐϯάΛߦ͏ • Nginx΍RedisͳͲҰൠతͳϛυϧ΢ΣΞͷϩά ͸IntegrationΛೖΕΕ͹উखʹύʔεͯ͘͠Ε ΔͷͰࣗ෼Ͱgrokॻ͘ඞཁ͕ͳ͘ศར • ԼਤͷΑ͏ʹܭࢉࣜΛೖΕͨΓɺผ໊ʹม͑ͨ Γ͢Δ͜ͱ΋Ͱ͖Δ

Slide 41

Slide 41 text

@rwle1212 41 Indexes • ϩάऩूͷআ֎ϑΟϧλ͕ॻ͚ΔɻIndexed LogsͰ՝ۚ͞Εͳ͘ͳΔ • 10%͚ͩऔಘ͢Δͱ͍͏͜ͱ΋ՄೳɻΫΤϦͰ ͔ͳΓߜΓࠐΉ͜ͱ͕Ͱ͖Δ • ϩάΛૹ৴͢ΔAgent/ClientଆͰίϯτϩʔϧ ͢Δ͜ͱ΋Մೳ͕ͩɺઃఆϑΝΠϧͷ഑෍౳͕ ໘౗ͳͷͰ͍ͬͨΜશ෦Datadogʹ౤͛ͯɺ IndexsͰϩάΛམͱ͢ͱ͍͏࢖͍ํ͕Φεε ϝɻʢ҆Ձ͚ͩͲIngested Logsͷ՝ۚ͸ൃੜ͢ ΔͷͰ஫ҙʣ

Slide 42

Slide 42 text

@rwle1212 42 Archives • Datadogͷϩά͸ܖ໿ظؒʢ7೔ͱ͔15೔ͱ ͔ʣ͔͠ݟΕͳ͍ • աڈϩά΋௕ظؒอଘ͍ͨ͠৔߹͸ArchivesͰ S3΍GCSʹஷΊΔ͜ͱ͕Ͱ͖ΔͷͰɺAthena ΍BigQueryͰݟΔ͜ͱ΋Ͱ͖Δ • Fluentdͱ͔ͰS3ʹૹΔ͜ͱ΋Ͱ͖Δ͚Ͳɺ Datadogͷ৔߹͸ͱΓ͋͑ͣऩू͢ΔઃఆϑΝ ΠϧΛૹ৴͢Δ͚ͩͰྑ͍ͷͰɺઃఆϑΝΠϧ ͷ͹Β·͖΍ߋ৽͕ͳ͍෼ָ

Slide 43

Slide 43 text

@rwle1212 43 Archives • Rehydrate From Archivesͱ͍͏ػೳΛ࢖͑͹ ArchivesʹೖͬͯΔσʔλΛDatadog Logsʹ໭ ͢͜ͱ͕Ͱ͖ΔΈ͍ͨʁ͚ͩͲ࢖ͬͨ͜ͱͳ͍ ͷͰ஫ҙ఺͕Θ͔ͬͯͳ͍ɻ୭͔ڭ͑ͯ

Slide 44

Slide 44 text

@rwle1212 44 datadog logsʹ͍ͭͯ࢖༻ײ΍ࣄྫ ΞϓϦͷϩάΛDatadog LogsʹೖΕ͍ͯΔͷͰɺ΄΅ຖ೔౷ܭతʹݟͯҟৗ͕ແ͞ ͦ͏͔βοͱݟ͍ͯΔɻ ඞཁ͋Β͹ਂງΓ͢ΔͨΊʹ Graph Ͱ৭ʑͳ؍఺Ͱ෼ੳͨ͠Γɺه࿥࢒࣌͢ɺܦ ա؍࡯͢Δ͸NotebooksΛ࡞ͬͯܧଓతʹ༷ࢠΛݟͨΓ͍ͯ͠Δɻ

Slide 45

Slide 45 text

@rwle1212 45 Πϕϯτ৘ใʹσϓϩΠΠϕϯτͳͲɺͲΜͳ΋ͷΛૹͬ ͍ͯΔ͔ɻ ࿈ܞͤ͞ΔπʔϧΛ૿΍͢ͷ͕ݏͳͷͰ࿈ܞ͍ͤͯ͞·ͤΜ

Slide 46

Slide 46 text

@rwle1212 46 APMؔ࿈ɺAnomalyϞχλʔͷ༗༻ͳ࢖͍ํ APMɺAnomaly͸࢖͍͜ͳ͓ͤͯΓ·ͤΜ

Slide 47

Slide 47 text

@rwle1212 47 LogsઃఆͷϕετϓϥΫςΟε • ͱΓ͋͑ͣ • શ෦ͷϩάΛ1Օॴʹू໿͢Δɻѹ౗తޮ཰্͕Δ • Pipeline • ύʔεͰ͖Δ͚ͲɺͰ͖ΔݶΓJSONͰૹ৴ͯࣗ͠ಈతʹύʔεͯ͠΋Β͏ • εςʔλεͱ͔͸Ϛοϐϯάͯ͠ɺϏϡʔϫʔͷ੨ɾԫɾ੺৭දࣔͰҰ໨ྎવʹ͢Δ • Indexes • ίετ࡟ݮ͢ΔͨΊʹݟͳ͍ϩά͸আ֎͢Δ • Archives • ೦ͷͨΊաڈϩά͸S3/GCSʹอଘ͢Δ • S3/GCSଆͰҰఆظؒޙʹࣗಈ࡟আ͢ΔΑ͏ʹϥΠϑαΠΫϧΛೖΕΔ

Slide 48

Slide 48 text

48 ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠