Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mapion Log/Monitoring

Mapion Log/Monitoring

2017.03.24 WOVN x Mapion 勉強会

One compath

March 24, 2017
Tweet

More Decks by One compath

Other Decks in Technology

Transcript

  1. Mapion
    Log/Monitoring
    ByɹSuto
    Mapion
    Infrastructure Group
    Hironori Suto.

    View full-size slide

  2. Logs.
    (Access log.)

    View full-size slide

  3. Mapion Log.
    ֤γεςϜͷཁٻʹԠͯ͡ෳ਺ύλʔϯ
    ݹ͘ͳΓ͍͗ͯ͢Δ෦෼΋͋ΔͷͰ௚͍͖͍ͯͨ͠
    ͱ͜Ζ΋͋Γ·͢ɻ

    View full-size slide

  4. ELB+S3+Redshift
    ELB EC2
    Access
    Logs
    S3 Redshift
    Other data

    View full-size slide

  5. ELB+S3+Redshift
    Pros
    - Less changes.
    - Sql like search.
    Cons
    - Cost.

    View full-size slide

  6. ELB+S3+Athena
    ELB EC2
    Access
    Logs
    S3 Athena

    View full-size slide

  7. ELB+S3+Athena
    Pros
    - Less changes.
    - Sql like search.
    - Low cost.
    Cons
    - Only Access Logs.

    View full-size slide

  8. ELB+S3+Athena+ re-dash
    ELB EC2
    Access
    Logs
    S3 Athena
    Other data

    View full-size slide

  9. Re-dash
    Visualizing tool for any data
    sources.
    MySQL,PostgreSQL,Elastic search
    mongoDB, Athena and so on.

    View full-size slide

  10. EC2(or nova) + fluentd+kibana
    ELB EC2
    Access
    Proxy Server
    Fluentd(td-agent)
    EC2
    Elastic search +
    Elastic search service

    View full-size slide

  11. Pros
    - Search.
    - Any logs.
    - Realtime Watch.
    Cons
    - Cost.
    EC2(or nova) + fluentd+kibana

    View full-size slide

  12. Legacy…
    Maintenance cost…
    SCP + awstats.

    View full-size slide

  13. Monitoring
    - Zabbix (Helion & AWS)
    1 Zabbix Instance → AWS(vpc) ≒100 instance
    1 Zabbix Instance → Helion ≒100 instance
    3 Zabbix Instance → AWS(classic) ≒80 instance

    View full-size slide

  14. Monitoring
    - Cloudwatch
    for AWS system (ELB etc)

    View full-size slide

  15. AutoScale + Zabbix

    Zabbix
    Zabbix API
    Chef
    +Script
    New Server

    View full-size slide

  16. AutoScale + Zabbix

    Zabbix
    Zabbix API
    Script
    Delete Server

    View full-size slide

  17. AutoScale + Zabbix
    Zabbix
    Alert Log.

    View full-size slide

  18. Service Check(Nagios)
    ELB
    AWS
    Public cloud
    Nagios

    View full-size slide