Suggested Topicの質問に可能な限り答えてみた

Transcript

1. Japan Datadog User Group Meetup#1 ABEJA, Inc. Shogo Muranushi Suggested

   Topicͷ࣭໰ʹՄೳͳݶΓ౴͑ͯΈͨ

2. @rwle1212 2

3. @rwle1212 3 ໊લ ɿ ଜओ ૖ޛʢΉΒ͵͠ ͠ΐ͏͝ʣ ձࣾ ɿ ABEJA,

   Inc. ۀ຿ ɿΠϯϑϥΤϯδχΞ -> ϓϩμΫτΦʔφʔ -> SRE

4. @rwle1212 4 Datadogྐྵ 2೥΄ͲɻInfra ͱ Logs ͷར༻͕ϝΠϯɻ Datadog͸ࡉ͔͍ػೳ͕ଟ͘ɺͿͬͪΌ͚Α͘෼͔ͬͯͳ͍ɻ ࠓ೔࿩͢͜ͱ΋͋͘·Ͱฐࣾͷ࢖͍ํͳͷͰਖ਼ղͰ͸ͳ͍ɻ ৭ʑ஌Γ͍ͨͷͰΈΜͳͰ৘ใ࣋ͪدΓͨ͘ɺ


   ·ͣ͸ొஃͯ͠Έͨɻ

5. @rwle1212 5

6. @rwle1212 6

7. @rwle1212 7 Datadogར༻ن໛ײ docker.containers.running Datadog Logs by weekly

8. @rwle1212 8 ຊ୊ʹೖΓ·͢

9. @rwle1212 9 ࢖͍ͬͯΔػೳ΍ͦͷλΠϛϯά
 (ʮৼΓฦΓͰ͜ΜͳμογϡϘʔυΛ࢖͍ͬͯΔʯͳͲ) • Α͘͞ΘΔػೳ͸LogsͱNotebooks • ཧ༝΍λΠϛϯά͸ޙड़

10. @rwle1212 10 μογϡϘʔυͲ͏΍ͬͯઃܭͯ͠·͔͢ʁ ΅͘ͷ͔Μ͕͍͖͑ͨ͞ΐ͏ͷμογϡϘʔυ ͲΜͳDashboardͰӡ༻͍ͯ͠Δ͔ɻ εϚʔτͳμογϡϘʔυͷ࡞Γํ μογϡϘʔυ͸׶͑ͯ࢖ͬͯ·ͤΜɻ ཧ༝͸ʮ΅͘ͷ͔Μ͕͍͖͑ͨ͞ΐ͏ͷμογϡϘʔυʯʹͳͬͯ͠·͏͔ΒͰ͢ɻ

11. @rwle1212 11 ࣦഊྫ1: ݁ہCloudWatchΛݟ͍ͯͨ ALB -> Container -> EC2 ->

    RDSͱ͍͏ϦΫΤετͷྲྀΕͰμογϡϘʔυΛ࡞ͬ ͯΈͨɻ μογϡϘʔυͷେ͖͞͸ܾ·͍ͬͯΔɺ׌ͭάϥϑ͕ଟ͗ͯ͢΋ෳࡶʹͳΔͷ ͰɺεϚʔτʹগͳΊʹάϥϑΛ഑ஔͨ͠ɻ ͔͠͠ো֐࣌ʹ৘ใ͕଍Γͳ͍ͷͰɺ݁ہCloudWatchΛோΊ͍ͯͨ͜ͱʹؾ෇͍ ͨɻ ͔ͦ͜ΒඞཁͳάϥϑΛ଍͍ͯ͘͠ͱͲΜͲΜෳࡶʹͳΓɺଞͷϝϯόʔ͸ʮԿΛ Ͳ͏ݟͨΒྑ͍ͷ͔Θ͔Βͳ͍ʯঢ়ଶʹͳͬͨɻ ݁ہCloudWatchΛோΊ͍ͯͨ͜ͱʹؾ෇͍ͨɻ

12. @rwle1212 12 ࣦഊྫ2: 
 μογϡϘʔυͷछྨ͕૿͑ΔͱΑ͘Θ͔Βͳ͘ͳΔ

13. @rwle1212 13 ղܾྫ
 NotebookΛ࢖ͬͯΈͨ • ো֐࣌ɺௐࠪ࣌ʹඞཁͳಛఆͷάϥϑ ͕഑ஔ͞Ε͍ͯΔෛՙ֬ೝ༻ͷ NotebookΛ࡞੒ • ܦա؍࡯ΛݟΔͨΊͷάϥϑ͕഑ஔ͞

    Ε͍ͯΔNotebookΛ࡞੒ ͳͲ

14. @rwle1212 14 ղܾྫ
 NotebookΛ࢖ͬͯΈͨ • Jupyter Notebook Likeͳ࢖͍ํ͕Ͱ͖ ͯɺίϝϯτͱάϥϑΛॻ͚Δ •

    ಛఆͷো֐ʹରͯ͠ௐ΂ͨ͜ͱɺਪଌ ͳͲͷίϝϯτΛॻ͖ͳ͕Βه࿥͕࢒ ͤΔ • ϩάͷάϥϑԽ΋༰қ • Issueʹը૾ΛషΓ෇͚ΔΑΓεϜʔ ζʹ·ͱΊΒΕΔͷͰڞ༗ָ͕͘͢͝

15. @rwle1212 15 ղܾྫ
 NotebookΛ࢖ͬͯΈͨ • ͭ·Γ൚༻తͳμογϡϘʔυ͸࡞੒ͤͣʹɺCloudWatchͳͲͷશମΛ၆ᛌͰ ͖ΔϝτϦΫεΛݟͭͭɺϩάΛάϥϑԽͨ͠Γͯ͠ɺಛఆͷো֐ʹ߹Θͤͯ ඞཁͳάϥϑΛ഑ஔͨ͠NotebookͰௐࠪ΍ܦա؍࡯Λߦ͍ͬͯΔ

16. @rwle1212 16 ͦ΋ͦ΋Έͳ͞ΜͲͷػೳԡ͠? ͱ͍͏͜ͱͰNotebooksΛਪͯ͠·͢ɻ ͋ͱLogs΋ྑ͍ࢠͰ͢ɻޙड़Ͱ·ͱ·͍ͬͯ·͢ɻ

17. @rwle1212 17 ௿ϦΫΤετ࣌ͷrateΞϥʔτ཈੍ํ๏ʹ͍ͭͯ ௿ϦΫΤετͷ؂ࢹ͸ͯ͠·ͤΜ

18. @rwle1212 18 terraformҎ֎Ͱͷ؅ཧํ๏ ओʹຊ൪༻͸TerraformɻͱΓ͋͑ͣͷ࣌͸खͰ࡞Δɻ खͰ࡞ͬͨͷͱTerraformͰ࡞ͬͨ΋ͷ͕ࠞࡏ͢Δͱɺຊ൪༻ͳͷ͔ɺΞϥʔτΛ ์ஔͯ͠ྑ͍ͷ͔Α͘෼͔Βͳ͘ͳΔͷͰʮCreateByTerraformʯTagΛ෇༩ͯ͠൑ ผ͍ͯ͠Δ

19. @rwle1212 19 ՝ۚ؅ཧʹؔ͢Δେٕখٕ ಛʹେٕ͸ແ͠ɻখωλ͕͋Δͱͨ͠Β • Multi Org؀ڥ͸Billing͕ू໿͞Εͯ͠·͏ɻʮLast Month’s UsageʯͰར༻ྔʢྉ ۚͰ͸ͳ͍ʣ͕Orgຖʹग़ྗ͞ΕΔͷͰ҈෼͍ͯ͠ΔɻӦۀ͞Μʹݴ͑͹Billing΋

    ෼͚ͯ͘ΕΔ͔΋ʁखؒͳͷͰͦ͜·Ͱ΍ͬͯͳ͍ • AWSΠϯςάϨʔγϣϯͰͨ͘͞ΜͷαʔϏεͷϝτϦΫεΛऔΕΔΑ͏ʹ͢Δ ͱɺCloudWatch ͷ GetMetricData API ͕ΊͪΌͪ͘Όݺͼग़͞ΕΔͷͰ՝͕ۚ େ͖͘૿͑Δɻ·ͣ͸ඞཁͳαʔϏε͚ͩߜͬͯऔΓࠐΉͷ͕٢ɻ
 SQSͰࢮ๢ͨ͠ܦݧ༗Γ

20. @rwle1212 20 ՝ۚ؅ཧʹؔ͢Δେٕখٕ • Logs͸ʮͲΕ͕Կ݅ʯͱ͔ݟΕΔͷͰɺͨ·ʹෆཁͳϩά͕ແ͍͔֬ೝͯ͠ ʮIndexsʯͰআ֎ϑΟϧλΛ࡞੒ͯ͠ϩάͷ՝ۚΛ཈͑ͨΓ͢Δ
 ʢIndexed Logsͷ՝ۚ͸ࢭ·Δ͚ͲɺIngested Logsͷ՝ۚ͸ࢭ·Βͳ͍ͷͰ஫ҙɻ͍͔҆Βؾʹ͢Δ ΄ͲͰ͸ͳ͍͚Ͳʣ

21. @rwle1212 21 DBϞχλϦϯάͷTips͋Ε͜Ε CloudWatchΛIntegrationͯ͠ΔҐͰͦΕҎ্ͷ͜ͱ͸͓ͯ͠Γ·ͤΜ

22. @rwle1212 22 GoogleAnalyticsAPIར༻ͨ͠ݱࡏϢʔβʔ਺ͱ͔࠾औͯ͠ ͍Δࣄྫ͕͋Ε͹ GoogleAnalyticsAPI͸࢖༻͓ͯ͠Γ·ͤΜ

23. @rwle1212 ࡢ೥ͷ12݄ʹҎԼͷهࣄΛॻ͖·ͨ͠ 23 ଞͷ؂ࢹSaaS౳ʢࣗ෼͕஌ΔൣғͰ͸Zabbixɾ PrometheusɾNew RelicɾMackerelʣͱͷൺֱ

24. @rwle1212 24 ʲཁ໿ʳͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ • Datadog͔ΒPrometheus΁৐Γ׵͑ • ݩʑDatadogΛ࢖͍ͬͯͨɻ$18/host͕ߴ͔ͬͨ͠ɺKubernetesͷ؂ࢹͷσϑΝΫ τʹͳΓͭͭ͋ΔPrometheusΛࢼ͢໨త΋ؚΊͯ࢖͍࢝Ίͨɻ • ൒೥ܦաɻPrometheusࣗମͷো֐͕νϥϗϥ

    • OSSͰ৘ใগͳ͍৔߹͸ௐࠪʹ͕͔͔࣌ؒΔɻଟ͍࣌͸िʹ1,2ճ΄Ͳ໰୊͕ൃੜ ͠1,2ਓ͕਺࣌ؒ/ि΄ͲऔΒΕ͍ͯͨ • ελʔτΞοϓ͸ਓ͕͍ͳ͍ɻຊۀͷ։ൃʹूத͍ͨ͠ • πʔϧͷ͓कΓʹ͕࣌ؒऔΒΕ͍ͯͨɻPrometheusͷӡ༻Λ֎෦ʹҕୗ͍ͨ͠ɻ ӡ༻Λ֎෦ʹҕୗ͢Δͱ͍͘ΒʹͳΔ͔෼͔Βͳ͍͚Ͳɺࠓͷن໛ΛDatadogʹม ͑Δͱ10ສ/݄͘Β͍ͩͬͨɻɾɾDatadogͰΑ͘Ͷʁͱࢥ͍࢝ΊΔ

25. @rwle1212 25 ʲཁ໿ʳͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ • ϩάͷऩूͲ͏͢Δ໰୊ • CloudWatch LogsΛ࢖͍͕ͬͯͨݕࡧੜօແɻʢ౰࣌͸ʣPrometheus͸ϩάऩू ͢Δػೳ͸ͳ͍ɻσϑΝΫτతʹ͸ ElasticSearch

    + Kibana ͩΖ͏ • ͔͠͠ɺͦͷ࣌఺Ͱϩάྔ͕5ԯ/݄ɻঃʑʹର৅ͷΞϓϦ΍νʔϜΛ૿΍ͦ͏ͱࢥ ͏ͱ਺ϲ݄Ͱഒʑʹ૿͍͑ͯ͘͜ͱ͕૝ఆ͞Ε͍ͯͨ • ͦΕΛElasticSearchͰड͚ͳ͕Βӡ༻͢Δ͜ͱΛ૝ఆ͢Δͱɺख͕औΒΕͦ͏ײ͕ ൒୺ͳ͔ͬͨ • Datadog΋ϩάͷαʔϏεΛఏڙ࢝͠Ί͍ͯͨͷͰ࢖ͬͯΈΔ͜ͱʹ

26. @rwle1212 26 ʲཁ໿ʳͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ • ϩάͲ͏͢Δ͔ͷ݁࿦ • Facets ͕ڧྗͰɺಋೖָ͕ɺίετίϯτϩʔϧ΋͠΍͘͢ɺԿ͔͋Ε͹શϩά Λ௥͑Δɻͦͷ্Ͱɺϩάऩू͢Δ্ͰΠϯϑϥΛؾʹ͠ͳͯ͘ྑ͍ͷ͕ؾʹೖ ΓɺঃʑʹϩάΛ

    Datadog Logs ʹدͤ·ͨ͠ • ͦͯ͠ɺϞχλϦϯά΋ Promethes ͔Β Datadog ΁Ҡߦ • ͳͥPrometheusΛࣙΊͯDatadogΛ࠾༻ͨ͠ͷ͔ɺͷ·ͱΊ • ϩά΋ϞχλϦϯά΋ ͦΕ͕ࣄۀͷίΞͰ͋Ε͹ͪΌΜͱߏங͠ӡ༻͢Δ͜ͱࣗ ମ͸ԀΘͳ͍ͷͰ͕͢ɺίΞͰͳ͍ͳΒ࣌ؒΛׂ͖ͨ͘ͳ͍

27. @rwle1212 ZabbixɺPrometheus ๻ͷ؍఺͔Β͢ΔͱɺػೳੑͰ͸ͳ͘ӡ༻ੑͰ൑அ͍ͯ͠ΔͷͰOSSܥ͸Α΄ Ͳಛ௃తͳػೳ΍ཁ͕݅ͳ͍ݶΓੵۃతͳ࠾༻͸ආ͚͍ͨͰ͢ 27 ଞͷ؂ࢹSaaS౳ʢࣗ෼͕஌ΔൣғͰ͸Zabbixɾ PrometheusɾNew RelicɾMackerelʣͱͷൺֱ

28. @rwle1212 New RelicɺMackerel ྆ํͱ΋௚઀ݕূ͸͍ͯ͠ͳ͍ͷͰൺֱʹ͸ͳΒͳ͍Ͱ͕͢ɺݕূ͠ͳ͔ͬͨ ཧ༝ͷҰͭͱͯ͠ɺDatadogͰ͸ʮMonitoring Modern Infrastructureʯͱ͍͏ه ࣄ͕ެ։͞Ε͍ͯ·͢ɻ
 ͜͜ʹॻ͔Ε͍ͯΔ಺༰͕ͱͯ΋ཧʹ͔ͳ͍ͬͯͨͷͰɺ͜ͷΑ͏ͳࢥ૝ʹ ΑͬͯαʔϏε͕ઃܭ͞Εࠓޙ΋αʔϏεΛ௥Ճ͍ͯ͘͠ΜͩΖ͏ͳͱࢥ͍ɺ

    ࠾༻ͨ͠എܠ΋͋Γ·͢ɻ 28 ଞͷ؂ࢹSaaS౳ʢࣗ෼͕஌ΔൣғͰ͸Zabbixɾ PrometheusɾNew RelicɾMackerelʣͱͷൺֱ

29. @rwle1212 29 SLOؔ࿈ͷϞχλʔ࡞੒ݕ౼Ͱࠔͬͨ͜ͱɾ޻෉ͨ͜͠ͱ ͋·Γ೉͍͠ػೳͰ͸ͳ͍ͷͰಛʹͳ͠ʁ ڧ͍ͯ͋͛ΔͳΒʮԿΛSLOͱͯ͠ܭଌ͢Δ͔ʯͷɺͦ΋ͦ΋ͷSLOͷఆ͕ٛ೉͍͠ ͘Β͍Ͱ͢ɻ

30. @rwle1212 30 Datadogͷ෍ڭ׆ಈ ʢͲͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ʁʣ • ޮՌ͋ͬͨ͜ͱ • Dev/Stg/ProdͷશΞϓϦͷϩάΛશ෦ू໿ͨ͠ɻ1ͭͷWebαΠτΛݟʹ ͍͘͜ͱͰશ෦ͷϩά͕ݟΕΔঢ়ଶʹͨ͠ɻࠓ·Ͱ͸؀ڥຖʹݟΔ৔ॴ͕ ෼͔ΕͯͨΓɺग़ͯͨΓग़ͯͳ͔ͬͨΓ

    • ͔ͦ͜ΒঃʑʹDatadogͷ࢖͍ํΛೝࣝ͠ɺMonitors΍NotebooksɺAPM ʹϝϯόʔͷڵຯ͕޿͕ͬͯߦͬͨ

31. @rwle1212 31 Datadogͷ෍ڭ׆ಈ ʢͲͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ʁʣ • ޮՌ͋·Γͳ͔ͬͨ͜ͱ • Datadogͷ࢖͍ํษڧձɻʮ΁ʔɻ͜Μͳ͜ͱͰ͖ΔΜͩʯͱ͍͏͚ͩͰखΛ ಈ͔͞ͳ͍ͷͰฉ͍ͯ΋๨ΕΔ •

    ϩάɺϝτϦΫεΛՄೳͳݶΓू໿ͯ͠ɺඞཁʹۦΒΕͯࣗ෼Ͱௐ΂ͯ޻ ෉ͯ͠΋Β͏ํ͕͕֮͑ྑͦ͞͏ͳؾ͕ͨ͠ɻݴ͏ͯ΋೉͘͠ͳ͍ͷͰ • ͋Δఔ౓ͨ͠Βʮ͓Εͷ͔Μ͕͍͖͑ͨ͞ΐ͏ͷDatadogڞ༗ձʯΛ֤ϝ ϯόʔͰখωλతʹ΍Δͷ΋ѱ͘͸ͳ͔ͬͨ

32. @rwle1212 32 Datadogͷ෍ڭ׆ಈ ʢͲͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ʁʣ • Ͳͷ໾ׂͷΤϯδχΞ·Ͱ࢖͍ͬͯΔ͔ • ࢖͍ͬͯΔͷ͸ΠϯϑϥɺόοΫΤϯυ • ϑϩϯτΤϯυͷE2E؂ࢹܥ͸গ͠ߴ͍ͷͰ࢖͍ͬͯͳ͍

    • Real User MonitoringɺSynthetic Monitoring (Browser)

33. @rwle1212 33 ΠϯςάϨʔγϣϯͰ޻෉͍ͯ͠Δ͜ͱʢSlack / JIRAͱ૊ Έ߹ΘͤͯɺΠϯγσϯτ؅ཧɾ໰୊؅ཧʹར༻ͳͲʣ Warning͸ʮSlack௨஌ʯ Alert͸ʮSlack௨஌ʯ+ʮPagerDutyͰి࿩ʯ+ʮGitHub Issue࡞੒(Zapierܦ༝)ʯ

34. @rwle1212 34 Ξϥʔτͷ௨஌Ϩϕϧ؅ཧ ʢฐࣾ͸warn / error / fatal ͳͲʣ •

    WarnͱErrorͷΈ • ʮαʔϏεʹ௚઀Өڹ͕Ͱ͍ͯΔϞϊɺͰ͍ͯΔՄೳੑ͕͋ΔϞϊʯΛΞϥʔτ ͱͯ͠ઃఆ͍ͯ͠Δ • ྫʣCPU΍ϝϞϦͷ࢖༻཰Λ؂ࢹͱ͔͸͍ͯ͠ͳ͍ɻͦΕΒͷӨڹʹΑͬ ͯALBͰ5xx͕සൃ͢Ε͹Ξϥʔτ͕ඈͿΑ͏ʹ͍ͯ͠ΔɻDynamoDBͷ εϩοτϦϯάͱ͔

35. @rwle1212 35 datadog logsʹ͍ͭͯ࢖༻ײ΍ࣄྫ

36. @rwle1212 36 facets • JSONͰૹΕ͹উखʹύʔεͯ͘͠ΕΔ • ύʔε͞Εͨ஋Λfacets(index)Ͱ͖ΔͷͰɺϩ άͷQuery΍Group ByͳͲʹར༻Ͱ͖Δ

37. @rwle1212 37 facets • Կ͕ѱ͍ͷ͔Θ͔Βͳ͍͚ͲɺindexͰ͖ͳ͍ ͕࣌͋Δɻ୭͔ղܾํ๏Λڭ͑ͯཉ͍͠ • ͜ͷέʔεͷ৔߹͸਺஋Ҏ֎ͷจࣈྻ͕ೖͬ ͪΌͬͯΔͷ͔ͳɾɾʁͬ͟ͱோΊͨײͦ͡Μ ͳ͜ͱͳ͍͚ͲɺͦΕΛ؆୯ʹݟΔʹ͸Ͳ͏͠

    ͨΒྑ͍ΜͩΖ͏ɾɾ

38. @rwle1212 38 Graph by Logs • ϩάΛάϥϑԽͰ͖Δ • facetsʹొ࿥ͨ͠஋ΛGroup Byʹࢦఆͨ͠Γɺ

    CountʹೖΕΔ͜ͱ΋Ͱ͖Δ • ࠨਤͷྫͰ͸ɺOrganization NameຖͷϦΫΤ ετ਺΍ฏۉԠ౴࣌ؒͳͲΛදࣔɻίϯςφID ຖʹݟͯภΓ͕ແ͍͔ݟͨΓ΋ͯ͠Δ • γεςϜܥ͚ͩͰ͸ͳ͘ΞϓϦͷར༻ঢ়گ෼ੳ ʹ΋࢖༻ՄೳͳͷͰɺΞϓϦଆͰ޷͖ͳ஋Λग़ ͤ͹৭ʑͳ؍఺Ͱ෼ੳͰ͖Δ • ᮢ஋Λઃ͚ͯΞϥʔτൃ๒Ͱ͖ΔͷͰ1ͭͷγ εςϜͰϝτϦΫεɾϩάͷ؂ࢹΛ؅ཧͰ͖Δ

39. @rwle1212 39 Graph by Logs • େ͖͍มߋΛͨ࣌͠͸εςʔλείʔυͷׂ߹ Λ֬ೝ • ઃఆํ๏

    • Group By: @http.status_code • Display: Areas • Stacked as: % of overall count: *

40. @rwle1212 40 Pipelines • JSON͡Όͳ͍ϩάΛύʔεͨ͠ΓɺDatadog ্Ͱʮ͜ͷϑΟʔϧυ͸εςʔλεѻ͍͢Δʯ ͱ͍͏Α͏ͳϚοϐϯάΛߦ͏ • Nginx΍RedisͳͲҰൠతͳϛυϧ΢ΣΞͷϩά ͸IntegrationΛೖΕΕ͹উखʹύʔεͯ͘͠Ε

    ΔͷͰࣗ෼Ͱgrokॻ͘ඞཁ͕ͳ͘ศར • ԼਤͷΑ͏ʹܭࢉࣜΛೖΕͨΓɺผ໊ʹม͑ͨ Γ͢Δ͜ͱ΋Ͱ͖Δ

41. @rwle1212 41 Indexes • ϩάऩूͷআ֎ϑΟϧλ͕ॻ͚ΔɻIndexed LogsͰ՝ۚ͞Εͳ͘ͳΔ • 10%͚ͩऔಘ͢Δͱ͍͏͜ͱ΋ՄೳɻΫΤϦͰ ͔ͳΓߜΓࠐΉ͜ͱ͕Ͱ͖Δ •

    ϩάΛૹ৴͢ΔAgent/ClientଆͰίϯτϩʔϧ ͢Δ͜ͱ΋Մೳ͕ͩɺઃఆϑΝΠϧͷ഑෍౳͕ ໘౗ͳͷͰ͍ͬͨΜશ෦Datadogʹ౤͛ͯɺ IndexsͰϩάΛམͱ͢ͱ͍͏࢖͍ํ͕Φεε ϝɻʢ҆Ձ͚ͩͲIngested Logsͷ՝ۚ͸ൃੜ͢ ΔͷͰ஫ҙʣ

42. @rwle1212 42 Archives • Datadogͷϩά͸ܖ໿ظؒʢ7೔ͱ͔15೔ͱ ͔ʣ͔͠ݟΕͳ͍ • աڈϩά΋௕ظؒอଘ͍ͨ͠৔߹͸ArchivesͰ S3΍GCSʹஷΊΔ͜ͱ͕Ͱ͖ΔͷͰɺAthena ΍BigQueryͰݟΔ͜ͱ΋Ͱ͖Δ

    • Fluentdͱ͔ͰS3ʹૹΔ͜ͱ΋Ͱ͖Δ͚Ͳɺ Datadogͷ৔߹͸ͱΓ͋͑ͣऩू͢ΔઃఆϑΝ ΠϧΛૹ৴͢Δ͚ͩͰྑ͍ͷͰɺઃఆϑΝΠϧ ͷ͹Β·͖΍ߋ৽͕ͳ͍෼ָ

43. @rwle1212 43 Archives • Rehydrate From Archivesͱ͍͏ػೳΛ࢖͑͹ ArchivesʹೖͬͯΔσʔλΛDatadog Logsʹ໭ ͢͜ͱ͕Ͱ͖ΔΈ͍ͨʁ͚ͩͲ࢖ͬͨ͜ͱͳ͍

    ͷͰ஫ҙ఺͕Θ͔ͬͯͳ͍ɻ୭͔ڭ͑ͯ

44. @rwle1212 44 datadog logsʹ͍ͭͯ࢖༻ײ΍ࣄྫ ΞϓϦͷϩάΛDatadog LogsʹೖΕ͍ͯΔͷͰɺ΄΅ຖ೔౷ܭతʹݟͯҟৗ͕ແ͞ ͦ͏͔βοͱݟ͍ͯΔɻ ඞཁ͋Β͹ਂງΓ͢ΔͨΊʹ Graph Ͱ৭ʑͳ؍఺Ͱ෼ੳͨ͠Γɺه࿥࢒࣌͢ɺܦ

    ա؍࡯͢Δ͸NotebooksΛ࡞ͬͯܧଓతʹ༷ࢠΛݟͨΓ͍ͯ͠Δɻ

45. @rwle1212 45 Πϕϯτ৘ใʹσϓϩΠΠϕϯτͳͲɺͲΜͳ΋ͷΛૹͬ ͍ͯΔ͔ɻ ࿈ܞͤ͞ΔπʔϧΛ૿΍͢ͷ͕ݏͳͷͰ࿈ܞ͍ͤͯ͞·ͤΜ

46. @rwle1212 46 APMؔ࿈ɺAnomalyϞχλʔͷ༗༻ͳ࢖͍ํ APMɺAnomaly͸࢖͍͜ͳ͓ͤͯΓ·ͤΜ

47. @rwle1212 47 LogsઃఆͷϕετϓϥΫςΟε • ͱΓ͋͑ͣ • શ෦ͷϩάΛ1Օॴʹू໿͢Δɻѹ౗తޮ཰্͕Δ • Pipeline •

    ύʔεͰ͖Δ͚ͲɺͰ͖ΔݶΓJSONͰૹ৴ͯࣗ͠ಈతʹύʔεͯ͠΋Β͏ • εςʔλεͱ͔͸Ϛοϐϯάͯ͠ɺϏϡʔϫʔͷ੨ɾԫɾ੺৭දࣔͰҰ໨ྎવʹ͢Δ • Indexes • ίετ࡟ݮ͢ΔͨΊʹݟͳ͍ϩά͸আ֎͢Δ • Archives • ೦ͷͨΊաڈϩά͸S3/GCSʹอଘ͢Δ • S3/GCSଆͰҰఆظؒޙʹࣗಈ࡟আ͢ΔΑ͏ʹϥΠϑαΠΫϧΛೖΕΔ

48. 48 ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠