AdServer Casual Monitoring by fluentd, Mackerel

Fashion Tech meetup #1 LT

Transcript

1. FasionTech Meetup LT AdServer Casual Monitoring by fluentd, Mackerel 1 VASILY Inc. 2015
2. Me • Takumi Kanzaki (@tknzk) • VASILY Inc. Backend Engineer • 2015.04~ • iQONADΛϝΠϯͰ୲౰ 2 VASILY Inc. 2015
3. iQONAD • iQON͓Αͼ֎෦ΞϓϦʹ޿ࠂ഑৴Λߦ͍ͬͯΔADωοτϫʔΫ • 2014.10~ 3 VASILY Inc. 2015
4. iQONAD • software, middleware • Ruby, Rails, sinatra • MySQL, memcached, Redis • infra • EC2, RDS • docker, Elasticbeanstalk (2015.06~) • auto-scaling 4 VASILY Inc. 2015
5. Why Casual Monitoring • ॳظ։ൃ2िؒͰϩʔϯν • طଘͷ࢓૊Έ • ੜϩάΛऩूͯ͠ूܭ • fluentdͰlog-serverʹू໿͸͍ͯ͠Δ͕ະूܭ 5 VASILY Inc. 2015
6. Casual Monitoring? • ͓͓·͔ͳঢ়ଶΛ؂ࢹ͍ͨ͠ • աڈͲΜͳঢ়گ͔ͩͬͨΛ೺Ѳ͍ͨ͠ 6 VASILY Inc. 2015
7. Casual Monitoring? • ഑৴਺ͷਪҠ • impression/clickͳͲͷਪҠ • ഑৴ΤϥʔͷਪҠ • http status codeͷਪҠ • etc.. 7 VASILY Inc. 2015
8. without Casual Monitoring? • ϩά෼ੳ • BigQuery • latency monitoring • error rate • NewRelic 8 VASILY Inc. 2015
9. Log Collect • fluentd • mackerel 9 VASILY Inc. 2015
10. fluentd • Fluentd is an open source data collector for unified logging layer. 10 VASILY Inc. 2015
11. Mackerel • A RevolutionaryNew Kind ofApplication PerformanceManagement 11 VASILY Inc. 2015
12. plugin • fluent-plugin-datacounter • fluent-plugin-mackerel • (fluent-plugin-zabbix) 12 VASILY Inc. 2015
13. fluent-plugin-datacounter • ಛఆͷkeyͷ਺ΛಛఆͷִؒͰूܭͯ͘͠ΕΔ <store> type datacounter count_interval 1m count_key publisher_id aggregate all tag stats.delivery.publisher pattern1 publisher_id_1 ^1$ pattern2 publisher_id_2 ^2$ pattern3 publisher_id_3 ^3$ pattern4 publisher_id_4 ^4$ </store> 13 VASILY Inc. 2015
14. fluent-plugin-mackerel <match stats.delivery.publisher> type mackerel api_key MACKEREL_API_KEY service iqon-ad metrics_name ad_server_delivery_publisher.${out_key} out_keys unmatched_count,publisher_id_1_count,publisher_id_2_count,publisher_id_3_count,publisher_id_4_count </match> 14 VASILY Inc. 2015
15. metrics 15 VASILY Inc. 2015
16. spike • ֎෦഑৴͍ͯ͠ΔͷͰ഑৴ઌΞϓϦͰpush഑৴͕͋Δspike • աڈͷspike৘ใΛ͋ͭΊ͓͓ͯ·͔ʹ࣌ؒΛ೺Ѳ • time scheduled base auto-scalingͰରԠ 16 VASILY Inc. 2015
17. metrics page • iQONADͷ಺෦πʔϧʹmackerelͷάϥϑΛ͸Γ͚ͭ • iframe, markdown ͳͲͰ֎෦ʹshareͰ͖Δ • ΋ͪΖΜmackerelͷೝূ͸ඞཁ 17 VASILY Inc. 2015
18. 18 VASILY Inc. 2015
19. Alert • custom metricsͷঢ়گ΋؂ࢹର৅ʹઃఆͰ͖Δ • nginxͷstatusͷerror countͳͲ 19 VASILY Inc. 2015
20. Future • norikra • norikra-listener-mackerel 20 VASILY Inc. 2015
21. Conclusion • fluentdͱmackerelΛ͔ͭͬͯ؆୯ʹσʔλͷϞχλϦϯά • fluentdͷconfigΛগ͠ॻ͍ͯɺmackerelͷ؅ཧπʔϧ͔ΒΆͪ ΆͪઃఆͰ׬ྃ • mackerel͍͢͝ศར!!! • άϥϑΛ੾Γग़ͯ͠ϖϥΠνͷϖʔδΛͭ͘Δ͜ͱͰΤϯδχ ΞҎ֎ʹ΋ • ϞχλϦϯά͍ͯ͠Δσʔλ΋؂ࢹର৅ʹઃఆ 21 VASILY Inc. 2015
22. We are hiring! • info@vasily.jp 22 VASILY Inc. 2015
23. end ͋Γ͕ͱ͏͍͟͝·ͨ͠ 23 VASILY Inc. 2015