AdServer Casual Monitoring by fluentd, Mackerel

Transcript

1. FasionTech Meetup LT AdServer Casual Monitoring by fluentd, Mackerel 1

   VASILY Inc. 2015

2. Me • Takumi Kanzaki (@tknzk) • VASILY Inc. Backend Engineer

   • 2015.04~ • iQONADΛϝΠϯͰ୲౰ 2 VASILY Inc. 2015

3. iQONAD • iQON͓Αͼ֎෦ΞϓϦʹ޿ࠂ഑৴Λߦ͍ͬͯΔADωοτϫʔΫ • 2014.10~ 3 VASILY Inc. 2015

4. iQONAD • software, middleware • Ruby, Rails, sinatra • MySQL,

   memcached, Redis • infra • EC2, RDS • docker, Elasticbeanstalk (2015.06~) • auto-scaling 4 VASILY Inc. 2015

5. Why Casual Monitoring • ॳظ։ൃ2िؒͰϩʔϯν • طଘͷ࢓૊Έ • ੜϩάΛऩूͯ͠ूܭ •

   fluentdͰlog-serverʹू໿͸͍ͯ͠Δ͕ະूܭ 5 VASILY Inc. 2015

6. Casual Monitoring? • ͓͓·͔ͳঢ়ଶΛ؂ࢹ͍ͨ͠ • աڈͲΜͳঢ়گ͔ͩͬͨΛ೺Ѳ͍ͨ͠ 6 VASILY Inc. 2015

7. Casual Monitoring? • ഑৴਺ͷਪҠ • impression/clickͳͲͷਪҠ • ഑৴ΤϥʔͷਪҠ • http

   status codeͷਪҠ • etc.. 7 VASILY Inc. 2015

8. without Casual Monitoring? • ϩά෼ੳ • BigQuery • latency monitoring

   • error rate • NewRelic 8 VASILY Inc. 2015

9. Log Collect • fluentd • mackerel 9 VASILY Inc. 2015

10. fluentd • Fluentd is an open source data collector for

    unified logging layer. 10 VASILY Inc. 2015

11. Mackerel • A RevolutionaryNew Kind ofApplication PerformanceManagement 11 VASILY Inc.

    2015

12. plugin • fluent-plugin-datacounter • fluent-plugin-mackerel • (fluent-plugin-zabbix) 12 VASILY Inc.

    2015

13. fluent-plugin-datacounter • ಛఆͷkeyͷ਺ΛಛఆͷִؒͰूܭͯ͘͠ΕΔ <store> type datacounter count_interval 1m count_key publisher_id

    aggregate all tag stats.delivery.publisher pattern1 publisher_id_1 ^1$ pattern2 publisher_id_2 ^2$ pattern3 publisher_id_3 ^3$ pattern4 publisher_id_4 ^4$ </store> 13 VASILY Inc. 2015

14. fluent-plugin-mackerel <match stats.delivery.publisher> type mackerel api_key MACKEREL_API_KEY service iqon-ad metrics_name

    ad_server_delivery_publisher.${out_key} out_keys unmatched_count,publisher_id_1_count,publisher_id_2_count,publisher_id_3_count,publisher_id_4_count </match> 14 VASILY Inc. 2015

15. metrics 15 VASILY Inc. 2015

16. spike • ֎෦഑৴͍ͯ͠ΔͷͰ഑৴ઌΞϓϦͰpush഑৴͕͋Δspike • աڈͷspike৘ใΛ͋ͭΊ͓͓ͯ·͔ʹ࣌ؒΛ೺Ѳ • time scheduled base auto-scalingͰରԠ

    16 VASILY Inc. 2015

17. metrics page • iQONADͷ಺෦πʔϧʹmackerelͷάϥϑΛ͸Γ͚ͭ • iframe, markdown ͳͲͰ֎෦ʹshareͰ͖Δ • ΋ͪΖΜmackerelͷೝূ͸ඞཁ

    17 VASILY Inc. 2015

18. 18 VASILY Inc. 2015

19. Alert • custom metricsͷঢ়گ΋؂ࢹର৅ʹઃఆͰ͖Δ • nginxͷstatusͷerror countͳͲ 19 VASILY Inc.

    2015

20. Future • norikra • norikra-listener-mackerel 20 VASILY Inc. 2015

21. Conclusion • fluentdͱmackerelΛ͔ͭͬͯ؆୯ʹσʔλͷϞχλϦϯά • fluentdͷconfigΛগ͠ॻ͍ͯɺmackerelͷ؅ཧπʔϧ͔ΒΆͪ ΆͪઃఆͰ׬ྃ • mackerel͍͢͝ศར!!! • άϥϑΛ੾Γग़ͯ͠ϖϥΠνͷϖʔδΛͭ͘Δ͜ͱͰΤϯδχ

    ΞҎ֎ʹ΋ • ϞχλϦϯά͍ͯ͠Δσʔλ΋؂ࢹର৅ʹઃఆ 21 VASILY Inc. 2015

22. We are hiring! • info@vasily.jp 22 VASILY Inc. 2015

23. end ͋Γ͕ͱ͏͍͟͝·ͨ͠ 23 VASILY Inc. 2015