AdServer Casual Monitoring by fluentd, Mackerel

941d3ec0c92d7b43d43c1baa81d4d82d?s=47 Takumi KANZAKI
November 10, 2015

AdServer Casual Monitoring by fluentd, Mackerel

Fashion Tech meetup #1 LT

941d3ec0c92d7b43d43c1baa81d4d82d?s=128

Takumi KANZAKI

November 10, 2015
Tweet

Transcript

  1. FasionTech Meetup LT AdServer Casual Monitoring by fluentd, Mackerel 1

    VASILY Inc. 2015
  2. Me • Takumi Kanzaki (@tknzk) • VASILY Inc. Backend Engineer

    • 2015.04~ • iQONADΛϝΠϯͰ୲౰ 2 VASILY Inc. 2015
  3. iQONAD • iQON͓Αͼ֎෦ΞϓϦʹ޿ࠂ഑৴Λߦ͍ͬͯΔADωοτϫʔΫ • 2014.10~ 3 VASILY Inc. 2015

  4. iQONAD • software, middleware • Ruby, Rails, sinatra • MySQL,

    memcached, Redis • infra • EC2, RDS • docker, Elasticbeanstalk (2015.06~) • auto-scaling 4 VASILY Inc. 2015
  5. Why Casual Monitoring • ॳظ։ൃ2िؒͰϩʔϯν • طଘͷ࢓૊Έ • ੜϩάΛऩूͯ͠ूܭ •

    fluentdͰlog-serverʹू໿͸͍ͯ͠Δ͕ະूܭ 5 VASILY Inc. 2015
  6. Casual Monitoring? • ͓͓·͔ͳঢ়ଶΛ؂ࢹ͍ͨ͠ • աڈͲΜͳঢ়گ͔ͩͬͨΛ೺Ѳ͍ͨ͠ 6 VASILY Inc. 2015

  7. Casual Monitoring? • ഑৴਺ͷਪҠ • impression/clickͳͲͷਪҠ • ഑৴ΤϥʔͷਪҠ • http

    status codeͷਪҠ • etc.. 7 VASILY Inc. 2015
  8. without Casual Monitoring? • ϩά෼ੳ • BigQuery • latency monitoring

    • error rate • NewRelic 8 VASILY Inc. 2015
  9. Log Collect • fluentd • mackerel 9 VASILY Inc. 2015

  10. fluentd • Fluentd is an open source data collector for

    unified logging layer. 10 VASILY Inc. 2015
  11. Mackerel • A RevolutionaryNew Kind ofApplication PerformanceManagement 11 VASILY Inc.

    2015
  12. plugin • fluent-plugin-datacounter • fluent-plugin-mackerel • (fluent-plugin-zabbix) 12 VASILY Inc.

    2015
  13. fluent-plugin-datacounter • ಛఆͷkeyͷ਺ΛಛఆͷִؒͰूܭͯ͘͠ΕΔ <store> type datacounter count_interval 1m count_key publisher_id

    aggregate all tag stats.delivery.publisher pattern1 publisher_id_1 ^1$ pattern2 publisher_id_2 ^2$ pattern3 publisher_id_3 ^3$ pattern4 publisher_id_4 ^4$ </store> 13 VASILY Inc. 2015
  14. fluent-plugin-mackerel <match stats.delivery.publisher> type mackerel api_key MACKEREL_API_KEY service iqon-ad metrics_name

    ad_server_delivery_publisher.${out_key} out_keys unmatched_count,publisher_id_1_count,publisher_id_2_count,publisher_id_3_count,publisher_id_4_count </match> 14 VASILY Inc. 2015
  15. metrics 15 VASILY Inc. 2015

  16. spike • ֎෦഑৴͍ͯ͠ΔͷͰ഑৴ઌΞϓϦͰpush഑৴͕͋Δspike • աڈͷspike৘ใΛ͋ͭΊ͓͓ͯ·͔ʹ࣌ؒΛ೺Ѳ • time scheduled base auto-scalingͰରԠ

    16 VASILY Inc. 2015
  17. metrics page • iQONADͷ಺෦πʔϧʹmackerelͷάϥϑΛ͸Γ͚ͭ • iframe, markdown ͳͲͰ֎෦ʹshareͰ͖Δ • ΋ͪΖΜmackerelͷೝূ͸ඞཁ

    17 VASILY Inc. 2015
  18. 18 VASILY Inc. 2015

  19. Alert • custom metricsͷঢ়گ΋؂ࢹର৅ʹઃఆͰ͖Δ • nginxͷstatusͷerror countͳͲ 19 VASILY Inc.

    2015
  20. Future • norikra • norikra-listener-mackerel 20 VASILY Inc. 2015

  21. Conclusion • fluentdͱmackerelΛ͔ͭͬͯ؆୯ʹσʔλͷϞχλϦϯά • fluentdͷconfigΛগ͠ॻ͍ͯɺmackerelͷ؅ཧπʔϧ͔ΒΆͪ ΆͪઃఆͰ׬ྃ • mackerel͍͢͝ศར!!! • άϥϑΛ੾Γग़ͯ͠ϖϥΠνͷϖʔδΛͭ͘Δ͜ͱͰΤϯδχ

    ΞҎ֎ʹ΋ • ϞχλϦϯά͍ͯ͠Δσʔλ΋؂ࢹର৅ʹઃఆ 21 VASILY Inc. 2015
  22. We are hiring! • info@vasily.jp 22 VASILY Inc. 2015

  23. end ͋Γ͕ͱ͏͍͟͝·ͨ͠ 23 VASILY Inc. 2015