$30 off During Our Annual Pro Sale. View Details »

CloudWatch から Mackerel にメトリックを送る方法 2019年版 / Mackerel Meetup 13

CloudWatch から Mackerel にメトリックを送る方法 2019年版 / Mackerel Meetup 13

FUJIWARA Shunichiro

March 01, 2019
Tweet

More Decks by FUJIWARA Shunichiro

Other Decks in Technology

Transcript

  1. CloudWatch ͔Β
    Mackerel ʹϝτϦοΫ
    ΛૹΔํ๏ 2019೥൛
    2019-03-01 Mackerel Meetup #13
    @fujiwara

    View Slide

  2. ΞϯόαμʔʹͳΓ·ͨ͠

    View Slide

  3. CloudWatch → Mackerel
    ϝτϦοΫΛసૹ͍ͨ͠
    AWS ΠϯςάϨʔγϣϯͰαϙʔτ͞Ε͍ͯͳ͍΋ͷ
    άϥϑɾΞϥʔτ͸ Mackerel ଆͰݟ͍ͨ

    View Slide

  4. fluentd + plugin
    AWSͷCloudWatchͷϝτϦοΫ͔ΒELBؔ࿈ͷϝτϦοΫΛ
    αʔϏεϝτϦοΫʹ౤ߘ͢Δ
    mackerel.io/ja/docs/entry/advanced/fluentd#example-elb
    fluent-plugin-cloudwatch (input plugin) Ͱऔಘ
    fluent-plugin-mackerel (output plugin) Ͱૹ৴
    Ͳ͜Ͱಈ͔͢? 1୆͚ͩઃఆ͢Δ? ͱ͔໘౗͕͍Ζ͍Ζ

    View Slide

  5. Metrin - AWS CloudWatchͷ൚༻؂ࢹϓϥάΠϯ
    github.com/y13i/metrin
    $ metrin --namespace AWS/EC2 \
    --metric-name CPUUtilization \
    --statistic Average --dimension InstanceId:i-xxxxxxxxx \
    --start-time -600 print --last-value-only
    CloudWatch.InstanceId.i-xxxxxxxxx.CPUUtilization.Average 1.468 1482596160
    mackerel-agent ͕ѻ͑ΔܗࣜͰprint͢ΔͷͰ agent ͔Βୟ͘
    1ϝτϦοΫ͝ͱʹίϚϯυ࣮ߦ
    αʔϏεϝτϦοΫ͸ mackerel-agent Ͱ͸౤ߘͰ͖ͳ͍

    View Slide

  6. mackerel-cloudwatch-forwarder
    github.com/shogo82148/mackerel-cloudwatch-forwarder
    Lambda function. JSONͰऔಘ͢ΔϝτϦοΫΛఆٛ
    {
    "service_metrics": [
    {
    "service": "your service name",
    "name": "metric name on mackerel",
    "metric": [ "Namespace", "MetricName", "Dimension1Name", "Dimension1Value", {} ],
    "stat": "Sum"
    }
    ],
    "host_metrics": [
    {
    "hostId": "host id",
    "name": "metric name on mackerel",
    "metric": [ "Namespace", "MetricName", "Dimension1Name", "Dimension1Value", {} ],
    "stat": "Sum"
    }
    ]
    }

    View Slide

  7. ͜Ε·ͰͷιϦϡʔγϣϯ
    CloudWatch ͷ GetMetricStatistics API Λ࢖͍ͬͯΔ
    → 1ϝτϦοΫ͝ͱʹ 1 API call ͕ൃੜ͢Δ
    ઃఆͷ֬ೝ͕೉͍͠
    → ಈ͔ͯ͠Έͳ͍ͱҙਤͨ͠஋͕సૹͰ͖͍ͯΔ͔෼͔ΓͮΒ
    ͍
    (Mackerelʹؒҧͬͨ஋ΛೖΕͯ͠·͏ͱফͮ͠Β͍)

    View Slide

  8. cloudwatch-to-mackerel
    ෳ਺ͷϝτϦοΫΛҰׅऔಘͰ͖Δ GetMetricData API ͕
    2018೥9݄ʹϦϦʔε
    ϝτϦοΫసૹπʔϧ & Go ϥΠϒϥϦΛ࡞Γ·ͨ͠
    github.com/fujiwara/cloudwatch-to-mackerel
    MetricDataQuery JSON ͰऔಘɾసૹΛߦ͏΋ͷ

    View Slide

  9. MetricDataQuery JSON
    [{
    "Id": "m1",
    "Label": "service=MyService:alb.my-alb.response-time.p99",
    "MetricStat": {
    "Metric": {
    "Namespace": "AWS/ApplicationELB",
    "MetricName": "TargetResponseTime",
    "Dimensions": [{
    "Name": "LoadBalancer",
    "Value": "app/my-alb/8e0641feccf3491c"
    }]
    },
    "Period": 60,
    "Stat": "p99"
    }
    },
    {
    "Id": "m2",
    "Label": "service=MyService:alb.my-alb.response-time.p90",
    "MetricStat": {
    "Metric": {
    "Namespace": "AWS/ApplicationELB",
    "MetricName": "TargetResponseTime",
    "Dimensions": [{
    "Name": "LoadBalancer",
    "Value": "app/my-alb/8e0641feccf3491c"
    }]
    },
    "Period": 60,
    "Stat": "p90"
    }
    }]

    View Slide

  10. CLI
    $ cw2mkr metric-data-query.json
    ؀ڥม਺Ͱೝূ৘ใ͕ద੾ʹઃఆͯ͋͠Ε͹͜Ε͚ͩ
    σϑΥϧτͰ௚ۙ3෼ͷϝτϦοΫΛసૹ
    -start-time -end-time Λࢦఆ͢Δͱաڈ෼΋্ॻ͖Ͱ͖Δ
    (Mackerel͸աڈ24࣌ؒ෼·Ͱड͚෇͚)

    View Slide

  11. Go ϥΠϒϥϦ
    import (
    "github.com/fujiwara/cloudwatch-to-mackerel/agent"
    )
    query := []byte(`[ ... ]`) // MetricDataQuery JSON
    err := agent.Run(agent.Option{Query: query})
    ؀ڥม਺Ͱೝূ৘ใ͕ద੾ʹઃఆͯ͋͠Ε͹͜Ε͚ͩ
    Lambda Ͱ΋ಈ͔ͤ·͢

    View Slide

  12. ͍͍ͱ͜Ζ
    MetricDataQuery JSON ͸ aws-cli Ͱ࢖͑ΔܗࣜͱಉҰ
    → aws cloudwatch get-metric-data Ͱઃఆ֬ೝ͕༰қ
    ϗετɾαʔϏεϝτϦοΫ྆ରԠ
    Metric Math ʹ΋ରԠ
    → ෳ਺ϝτϦοΫͷԋࢉ݁ՌͰΞϥʔτͱ͔
    Enjoy!

    View Slide