CloudWatch から Mackerel にメトリックを送る方法 2019年版 / Mackerel Meetup 13

Transcript

1. CloudWatch ͔Β
   Mackerel ʹϝτϦοΫ
   ΛૹΔํ๏ 2019೥൛
   2019-03-01 Mackerel Meetup #13
   @fujiwara
   

   View Slide

2. ΞϯόαμʔʹͳΓ·ͨ͠
   

   View Slide

3. CloudWatch → Mackerel
   ϝτϦοΫΛసૹ͍ͨ͠
   AWS ΠϯςάϨʔγϣϯͰαϙʔτ͞Ε͍ͯͳ͍΋ͷ
   άϥϑɾΞϥʔτ͸ Mackerel ଆͰݟ͍ͨ
   

   View Slide

4. fluentd + plugin
   AWSͷCloudWatchͷϝτϦοΫ͔ΒELBؔ࿈ͷϝτϦοΫΛ
   αʔϏεϝτϦοΫʹ౤ߘ͢Δ
   mackerel.io/ja/docs/entry/advanced/fluentd#example-elb
   fluent-plugin-cloudwatch (input plugin) Ͱऔಘ
   fluent-plugin-mackerel (output plugin) Ͱૹ৴
   Ͳ͜Ͱಈ͔͢? 1୆͚ͩઃఆ͢Δ? ͱ͔໘౗͕͍Ζ͍Ζ
   

   View Slide

5. Metrin - AWS CloudWatchͷ൚༻؂ࢹϓϥάΠϯ
   github.com/y13i/metrin
   $ metrin --namespace AWS/EC2 \
   --metric-name CPUUtilization \
   --statistic Average --dimension InstanceId:i-xxxxxxxxx \
   --start-time -600 print --last-value-only
   CloudWatch.InstanceId.i-xxxxxxxxx.CPUUtilization.Average 1.468 1482596160
   mackerel-agent ͕ѻ͑ΔܗࣜͰprint͢ΔͷͰ agent ͔Βୟ͘
   1ϝτϦοΫ͝ͱʹίϚϯυ࣮ߦ
   αʔϏεϝτϦοΫ͸ mackerel-agent Ͱ͸౤ߘͰ͖ͳ͍
   

   View Slide

6. mackerel-cloudwatch-forwarder
   github.com/shogo82148/mackerel-cloudwatch-forwarder
   Lambda function. JSONͰऔಘ͢ΔϝτϦοΫΛఆٛ
   {
   "service_metrics": [
   {
   "service": "your service name",
   "name": "metric name on mackerel",
   "metric": [ "Namespace", "MetricName", "Dimension1Name", "Dimension1Value", {} ],
   "stat": "Sum"
   }
   ],
   "host_metrics": [
   {
   "hostId": "host id",
   "name": "metric name on mackerel",
   "metric": [ "Namespace", "MetricName", "Dimension1Name", "Dimension1Value", {} ],
   "stat": "Sum"
   }
   ]
   }
   

   View Slide

7. ͜Ε·ͰͷιϦϡʔγϣϯ
   CloudWatch ͷ GetMetricStatistics API Λ࢖͍ͬͯΔ
   → 1ϝτϦοΫ͝ͱʹ 1 API call ͕ൃੜ͢Δ
   ઃఆͷ֬ೝ͕೉͍͠
   → ಈ͔ͯ͠Έͳ͍ͱҙਤͨ͠஋͕సૹͰ͖͍ͯΔ͔෼͔ΓͮΒ
   ͍
   (Mackerelʹؒҧͬͨ஋ΛೖΕͯ͠·͏ͱফͮ͠Β͍)
   

   View Slide

8. cloudwatch-to-mackerel
   ෳ਺ͷϝτϦοΫΛҰׅऔಘͰ͖Δ GetMetricData API ͕
   2018೥9݄ʹϦϦʔε
   ϝτϦοΫసૹπʔϧ & Go ϥΠϒϥϦΛ࡞Γ·ͨ͠
   github.com/fujiwara/cloudwatch-to-mackerel
   MetricDataQuery JSON ͰऔಘɾసૹΛߦ͏΋ͷ
   

   View Slide

9. MetricDataQuery JSON
   [{
   "Id": "m1",
   "Label": "service=MyService:alb.my-alb.response-time.p99",
   "MetricStat": {
   "Metric": {
   "Namespace": "AWS/ApplicationELB",
   "MetricName": "TargetResponseTime",
   "Dimensions": [{
   "Name": "LoadBalancer",
   "Value": "app/my-alb/8e0641feccf3491c"
   }]
   },
   "Period": 60,
   "Stat": "p99"
   }
   },
   {
   "Id": "m2",
   "Label": "service=MyService:alb.my-alb.response-time.p90",
   "MetricStat": {
   "Metric": {
   "Namespace": "AWS/ApplicationELB",
   "MetricName": "TargetResponseTime",
   "Dimensions": [{
   "Name": "LoadBalancer",
   "Value": "app/my-alb/8e0641feccf3491c"
   }]
   },
   "Period": 60,
   "Stat": "p90"
   }
   }]
   

   View Slide

10. CLI
    $ cw2mkr metric-data-query.json
    ؀ڥม਺Ͱೝূ৘ใ͕ద੾ʹઃఆͯ͋͠Ε͹͜Ε͚ͩ
    σϑΥϧτͰ௚ۙ3෼ͷϝτϦοΫΛసૹ
    -start-time -end-time Λࢦఆ͢Δͱաڈ෼΋্ॻ͖Ͱ͖Δ
    (Mackerel͸աڈ24࣌ؒ෼·Ͱड͚෇͚)
    

    View Slide

11. Go ϥΠϒϥϦ
    import (
    "github.com/fujiwara/cloudwatch-to-mackerel/agent"
    )
    query := []byte(`[ ... ]`) // MetricDataQuery JSON
    err := agent.Run(agent.Option{Query: query})
    ؀ڥม਺Ͱೝূ৘ใ͕ద੾ʹઃఆͯ͋͠Ε͹͜Ε͚ͩ
    Lambda Ͱ΋ಈ͔ͤ·͢
    

    View Slide

12. ͍͍ͱ͜Ζ
    MetricDataQuery JSON ͸ aws-cli Ͱ࢖͑ΔܗࣜͱಉҰ
    → aws cloudwatch get-metric-data Ͱઃఆ֬ೝ͕༰қ
    ϗετɾαʔϏεϝτϦοΫ྆ରԠ
    Metric Math ʹ΋ରԠ
    → ෳ਺ϝτϦοΫͷԋࢉ݁ՌͰΞϥʔτͱ͔
    Enjoy!
    

    View Slide