Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elastic Stackでマイクロサービス運用を
楽にするには? / Monitoring Microservices with Elastic Stack

Jun Ohtani
January 25, 2019

Elastic Stackでマイクロサービス運用を
楽にするには? / Monitoring Microservices with Elastic Stack

OSC 大阪 2019でのセッションで使用した資料です。
「マイクロサービス」とありますが、アプリケーションの監視の話がメインですので、アプリケーションの監視やログ解析などに興味のある方にも読んでいただける内容となっています。

Jun Ohtani

January 25, 2019
Tweet

More Decks by Jun Ohtani

Other Decks in Technology

Transcript

  1. !1
    2019/01/25
    Community Engineer @Elastic

    Jun Ohtani @johtani
    Elastic StackͰϚΠΫϩαʔϏεӡ༻Λ

    ָʹ͢Δʹ͸ʁ - Elastic Stackͷೖ໳ͱ׆༻ -

    View full-size slide

  2. !2
    about
    • Me, Jun Ohtani / Community Engineer
    ‒ lucene-gosenίϛολʔ
    ‒ σʔλ෼ੳج൫ߏஙೖ໳ ڞஶ
    ‒ http://blog.johtani.info

    • Elastic, founded in 2012
    ‒ Products: Elasticsearch, Logstash, Kibana, Beats 

    Elastic APM, 

    Elastic Cloud, Swiftype 

    Professional services: Support & development subscriptions

    Trainings, Consulting, SaaS

    View full-size slide

  3. !3
    ΞδΣϯμ
    • ϚΠΫϩαʔϏεͱ͸ʁ
    • Elastic Stackͱ͸ʁ
    • ༷ʑͳ؍఺͔ΒͷΞϓϦέʔγϣϯͷ؂ࢹ
    • ͞Βʹ৭ʑࢼͯ͠ΈΔʹ͸ʁ

    View full-size slide

  4. !4
    ϚΠΫϩαʔϏεͱ͸

    View full-size slide

  5. !5
    ϚΠΫϩαʔϏε (Wikipedia)
    https://ja.wikipedia.org/wiki/ϚΠΫϩαʔϏε

    View full-size slide

  6. !6
    ϞϊϦγοΫ v.s. ϚΠΫϩαʔϏε

    View full-size slide

  7. !7
    ϚΠΫϩαʔϏεʁʁʁ

    View full-size slide

  8. !8
    Elastic Stackͱ͸ʁ

    View full-size slide

  9. Elastic
    Stack
    อଘɺݕࡧɺ෼ੳ
    Elasticsearch
    ՄࢹԽɺ؅ཧ
    Kibana
    Beats ΠϯδΣετ
    Logstash

    View full-size slide

  10. Metrics
    Logging
    APM
    Site

    Search
    Application
    Search
    Business

    Analytics
    Enterprise

    Search
    Security

    Analytics
    Future ιϦϡʔγϣϯ
    อଘɺݕࡧɺ෼ੳ
    ՄࢹԽɺ؅ཧ
    ΠϯδΣετ
    Kibana
    Elasticsearch
    Beats Logstash
    Elastic
    Stack

    View full-size slide

  11. Metrics
    Logging
    APM
    Site

    Search
    App

    Search
    Business

    Analytics
    Enterprise

    Search
    Security

    Analytics
    Future ιϦϡʔγϣϯ
    SaaS
    Elastic Cloud
    Self Managed
    Elastic Cloud

    Enterprise Standalone
    σϓϩΠ
    อଘɺݕࡧɺ෼ੳ
    ՄࢹԽɺ؅ཧ
    ΠϯδΣετ
    Kibana
    Elasticsearch
    Beats Logstash
    Elastic
    Stack

    View full-size slide

  12. อଘɺݕࡧɺ෼ੳ
    Elasticsearch
    ՄࢹԽɺ؅ཧ
    Kibana
    Beats ΠϯδΣετ
    Logstash
    Metrics
    Logging
    APM
    Site

    Search
    Application
    Search
    Business

    Analytics
    Enterprise

    Search
    Security

    Analytics
    Future ιϦϡʔγϣϯ
    SaaS
    Elastic Cloud
    Self Managed
    Elastic Cloud

    Enterprise Standalone
    σϓϩΠ
    Elastic
    Stack

    View full-size slide

  13. 14
    Beats
    ܰྔσʔλγούʔ
    ιʔε͔ΒσʔλΛసૹ సૹ͠Elasticsearchʹू໿ ม׵ͱύʔεͷͨΊ
    Logstashʹసૹ
    Elastic Cloudʹసૹ
    Libbeat: ΧελϜbeatsͷͨ
    ΊͷAPIϑϨʔϜϫʔΫ
    30Ҏ্ͷίϛϡχςΟbeats

    View full-size slide

  14. The Beats family
    Heartbeat
    Uptime monitoring
    Filebeat
    Log files
    Winlogbeat
    Windows Event Logs
    Packetbeat
    Network data
    +40
    community
    Beats
    Metricbeat
    Metrics
    Auditbeat
    Audit data

    View full-size slide

  15. 17
    Logstash
    σʔλՃ޻ύΠϓϥΠϯ
    શͯͷܗࣜɺαΠζͱσʔλιʔ
    εͷ౤ೖ
    ύʔεͱಈతͳ
    σʔλม׵
    ͋ΒΏΔग़ྗʹ
    σʔλసૹ
    ҆શͰ҉߸Խ͞Εͨ

    σʔλೖྗ
    ಠࣗͷύΠϓϥΠϯॲཧ
    ͷ࡞੒
    200Ҏ্ͷϓϥάΠϯ

    View full-size slide

  16. 19
    Elasticsearch
    Heart of the Elastic Stack
    ෼ࢄܕɺεέʔϥϒϧ ߴՄ༻ੑ Ϛϧνςφϯτ
    ։ൃऀϑϨϯυϦʔ ϦΞϧλΠϜɺશจݕࡧ ΞάϦήʔγϣϯ

    View full-size slide

  17. Elasticsearchͱ͸ʁ

    View full-size slide

  18. ϑϦʔϫʔυݕࡧ
    !21

    View full-size slide

  19. ߜΓࠐΈ
    !22

    View full-size slide

  20. ϋΠϥΠτ
    !23

    View full-size slide

  21. ϖʔδϯά
    !25

    View full-size slide

  22. αδΣετ
    !27

    View full-size slide

  23. Elasticsearch in 10 seconds
    • εΩʔϚϑϦʔɺ෼ࢄυΩϡϝϯτετΞɺREST & JSON
    • Φʔϓϯιʔε: Apache License 2.0
    • ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ
    • JavaͰ࣮૷ɻ֦ு΋༰қ
    !28

    View full-size slide

  24. σʔλొ࿥
    30
    curl -XPUT localhost:9200/books/book/1 -d '
    {
    "title" : "Elasticsearch - The definitive guide",
    "authors" : "Clinton Gormley",
    "started" : "2013-02-04",
    "pages" : 230
    }'

    View full-size slide

  25. σʔλߋ৽
    31
    curl -XPUT localhost:9200/books/book/1 -d '
    {
    "title" : "Elasticsearch - The definitive guide",
    "authors" : [ "Clinton Gormley", "Zachary Tong" ],
    "started" : "2013-02-04",
    "pages" : 230
    }'

    View full-size slide

  26. σʔλ࡟আ
    !32
    curl -X DELETE localhost:9200/books/book/1
    σʔλͷऔಘ
    curl —X GET localhost:9200/books/book/1
    curl —X GET localhost:9200/books/book/1/_source

    View full-size slide

  27. ݕࡧ - Query DSL
    !33
    curl -XGET ‘localhost:9200/books/doc/_search' -d '{
    "query": {
    "bool": {
    "must": [
    { "match": { "title": "Search" }},
    { "match": { "content": "Elasticsearch" }}
    ],
    "filter": [
    { "term": { "status": "published" }},
    { "range": { "publish_date": { "gte": "2015-01-01" }}}
    ]
    }
    }
    }'

    View full-size slide

  28. ෼ࢄߏ੒ɺ

    εέʔϧ

    View full-size slide

  29. γϟʔυͱϨϓϦΧ
    !35
    node 1
    orders
    products
    1
    4
    1 2
    2
    3
    curl -X PUT localhost:9200/orders -d '{
    "settings.index.number_of_shards" : 4
    "settings.index.number_of_replicas" : 1
    }'
    curl -X PUT localhost:9200/products -d '{
    "settings.index.number_of_shards" : 2
    "settings.index.number_of_replicas" : 0
    }'

    View full-size slide

  30. γϟʔυͱϨϓϦΧ
    !36
    node 1
    orders
    products
    1
    4
    1
    node 2
    orders
    products
    2
    2
    3 4
    1 2
    3

    View full-size slide

  31. ࣗಈతͳ෼ࢄ
    !37
    node 1
    orders
    products
    2
    1
    4
    1
    node 2
    orders
    products
    2
    2
    node 3
    orders
    products
    3 4
    1
    3

    View full-size slide

  32. 39
    Kibana
    Window into the Elastic Stack
    ՄࢹԽͱ෼ੳ ஍ཧۭؒ ΧελϚΠζͱ
    Ϩϙʔτͷڞ༗
    άϥϑ୳ࡧ Elastic Stack΁ͷ
    ηΩϡΞͳΞΫηεͱ؅ཧ
    ΧελϜAppsͷ࡞੒

    View full-size slide

  33. !41
    Elastic Stackͷߏ੒
    Beats
    Log
    Files
    Metrics
    Wire
    Data
    your{beat}
    Kibana
    Instances
    Kafka
    Distributed
    Message
    Queue
    Notification
    Queues Storage Metrics
    Data
    Store
    Web
    APIs
    Social Sensors
    Elasticsearch
    Nodes
    Logstash
    Nodes

    View full-size slide

  34. !42
    ΞϓϦέʔγϣϯͷ

    ؂ࢹϙΠϯτ

    View full-size slide

  35. !43
    ؂ࢹϙΠϯτ
    • ֎ܗ؂ࢹ
    • ϝτϦοΫʢϝτϦΫεʣ
    • αʔόʔɺΞϓϦέʔγϣϯ
    • ϩά
    • ΞϓϦέʔγϣϯͷϦϦʔελΠϛϯά
    • ෼ࢄτϨʔγϯά

    View full-size slide

  36. !44
    ֎ܗ؂ࢹ
    • ࢮ׆؂ࢹ
    • ϓϩηε
    • HTTPαʔόʔ
    • TCP
    • ICMP

    View full-size slide

  37. Lightweight
    Shipper for
    Uptime
    Monitoring
    Heartbeat

    View full-size slide

  38. !46
    ϝτϦοΫ
    • αʔόʔ
    • CPUɺϝϞϦɺσΟεΫɺωοτϫʔΫI/Oɺϓϩηε਺
    • ΞϓϦέʔγϣϯ
    • ϦΫΤετ਺ɺίωΫγϣϯ਺ɺॲཧ࣌ؒ
    • ίϯςφʔ
    • ίϯςφ਺

    View full-size slide

  39. Collect system
    and application
    metrics
    Metricbeat

    View full-size slide

  40. lots of modules
    Metricbeat

    View full-size slide

  41. !49
    Metricbeat Ϟδϡʔϧ
    ● Aerospike module
    ● Apache module
    ● Ceph module
    ● Couchbase module
    ● Docker module
    ● Dropwizard module
    ● Elasticsearch module
    ● Etcd module
    ● Golang module
    ● Graphite module
    ● HAProxy module
    ● HTTP module
    ● Jolokia module
    ● Kafka module
    ● Kibana module
    ● Kubernetes module
    ● kvm module
    ● Logstash module
    ● Memcached module
    ● MongoDB module
    ● Munin module
    ● MySQL module
    ● Nginx module

    ● PHP_FPM module
    ● PostgreSQL module
    ● Prometheus module
    ● RabbitMQ module
    ● Redis module
    ● System module
    ● uwsgi module
    ● vSphere module
    ● Windows module
    ● ZooKeeper module


    View full-size slide

  42. !50
    ϩά
    • ΞΫηεϩά
    • γεςϜϩά
    • ೝূϩά
    • εϩʔϩά
    • ΞϓϦέʔγϣϯϩά

    View full-size slide

  43. tail log from
    file
    Filebeat

    View full-size slide

  44. many modules
    Filebeat

    View full-size slide

  45. Filebeat modules - v6.4.2
    • Apache2 module
    • Auditd module
    • Icinga module
    • IIS module
    • Kafka module
    • Logstash module
    • MongoDB module
    • MySQL module
    • Nginx module
    • Osquery module
    • PostgreSQL module
    • Redis module
    • System module
    • Traefik module

    View full-size slide

  46. Welcome
    to 1998
    winlogbeat

    View full-size slide

  47. Now
    winlogbeat

    View full-size slide

  48. Capture the
    Packet
    Packetbeat

    View full-size slide

  49. Capture the
    Packet
    Packetbeat

    View full-size slide

  50. !58
    ΞϓϦέʔγϣϯͷϦϦʔελΠϛϯά
    • όάϑΟοΫεϦϦʔε
    • ৽ػೳϦϦʔε
    • ৽αʔϏε։࢝
    • αʔόʔ૿ڧ

    View full-size slide

  51. !59
    ෼ࢄτϨʔγϯά
    • ϚΠΫϩαʔϏε
    • 1ͭͷϦΫΤετʹରͯ͠ෳ਺ͷϓϩηε͕ؔ܎
    • ΞϓϦέʔγϣϯύϑΥʔϚϯεϞχλϦϯάͷ1ͭ

    View full-size slide

  52. !61
    Elastic APM

    View full-size slide

  53. Distributed Tracing
    Beta | Basic (free)
    શͯͷܭଌ͞ΕͨαʔϏεΛݟΔͨΊͷ
    ౷߹͞ΕͨϏϡʔ
    αϒίϯςΩετ಺ͷτϨʔεʹભҠ
    OpenTracing ޓ׵

    View full-size slide

  54. !63
    ͦͷ΄͔ͷศརͳػೳ
    • Infra UI
    • Logs UI
    • Machine Learning
    • Alerting

    View full-size slide

  55. Infrastructure Solution
    Beta | Basic (free)
    ΠϯϑϥΦϖϨʔλʔ޲͚ʹಛԽ
    Λ௒͑ΔΠϯϑϥͷߏ੒Λ၆ᛌ
    ,VCFSOFUFTɺ%PDLFSͷωΠςΟϒαϙʔ
    τ
    ϝτϦοΫɺϩάɺ"1.Ϗϡʔ΁ͷ

    υϦϧɾμ΢ϯ
    ΞυϗοΫ͓Αͼߏ଄Խݕࡧ

    View full-size slide

  56. Logs Solution
    Beta | Basic (free)
    ϥΠϒͰϩάͷτϥϒϧγϡʔςΟϯά
    Λॿ͚ΔܰྔͳϩάϏϡʔΞʔ
    ίϯιʔϧͷΑ͏ͳදࣔ
    UBJMGͷΑ͏ͳ
    ϥΠϒɾϩάɾετϦʔ
    ϛϯά
    ཤྺϩάͷແݶεΫϩʔϧ
    ΞυϗοΫ͓Αͼߏ଄Խݕࡧ

    View full-size slide

  57. !69
    ͞Βʹ׆༻͢Δʹ͸ʁ

    View full-size slide

  58. ͦͷଞͷ࢖͍ํ
    !72

    View full-size slide

  59. !73
    σʔλͷొ࿥ํ๏
    • Kibanaͷαϯϓϧσʔλʢ6.4͔Βʣ
    • LogstashͰJDBC input
    • LogstashͰCSV
    • FilebeatͰΞΫηεϩά
    • MetricbeatͰϝτϦοΫ
    • PacketbeatͰMySQL/PostgreSQLͷύέοτղੳ

    View full-size slide

  60. !74
    Kibanaͷαϯϓϧσʔλʢ>= 6.4.0ʣ

    View full-size slide

  61. !75
    ϫϯΫϦοΫͰσʔλొ࿥

    View full-size slide

  62. !76
    LogstashͰJDBC Input
    Kibana
    Instances
    Data
    Store
    Elasticsearch
    Nodes
    Logstash
    Nodes

    View full-size slide

  63. !77
    JDBC Input

    View full-size slide

  64. !78
    LogstashͰCSV
    Kibana
    Instances
    CSV

    File
    Elasticsearch
    Nodes
    Logstash
    Nodes

    View full-size slide

  65. !79
    CSV filter

    View full-size slide

  66. !80
    FilebeatͰΞΫηεϩά
    Beats
    Log
    Files
    Kibana
    Instances
    Elasticsearch
    Nodes

    View full-size slide

  67. • 2ͭͷElasticsearchϓϥάΠϯΛΠϯετʔϧͯ͠ElasticsearchΛىಈ
    • Filebeatͷapache2ϞδϡʔϧΛ༗ޮԽ
    • modules.d/apache2.ymlʹΞΫηεϩάͷύεΛઃఆ
    • setupίϚϯυΛ࣮ߦ͔ͯ͠ΒFilebeatΛىಈ
    !81
    FilebeatͰΞΫηεϩά

    View full-size slide

  68. MetricbeatͰϝτϦοΫ
    Beats
    Metrics
    Kibana
    Instances
    Elasticsearch
    Nodes

    View full-size slide

  69. • MetricbeatͷsystemϞδϡʔϧΛ༗ޮԽ
    • setupίϚϯυΛ࣮ߦ͔ͯ͠ΒFilebeatΛىಈ
    !83
    MetricbeatͰϝτϦοΫ

    View full-size slide

  70. !84
    PacketbeatͰMySQLɺPostgreSQLͷύέοτղੳ
    Beats
    Wire
    Data
    Kibana
    Instances
    Elasticsearch
    Nodes

    View full-size slide

  71. !85
    ࢀߟจݙ
    • Elasticsearch - The Definitive guide
    ‒ http://www.elastic.co/guide/en/elasticsearch/guide/current/
    index.html
    • ॻ੶ʢ೔ຊޠʣ
    ‒ σʔλ෼ੳج൫ߏஙೖ໳
    ‒ Elasticsearch࣮ફΨΠυ

    View full-size slide

  72. !86
    ࢀߟจݙ
    • ೖ໳ ؂ࢹ

    ―ϞμϯͳϞχλϦϯάͷͨΊͷσβΠϯύλʔϯ

    Mike JulianɹஶɺদӜ ൏ਓɹ༁


    https://www.oreilly.co.jp/books/9784873118642/

    View full-size slide

  73. !87
    ࢀߟαΠτ
    • Ϣʔεέʔε
    • https://www.elastic.co/use-cases
    • DiscussʢWebϑΥʔϥϜʣ
    • https://discuss.elastic.co
    • Elastic{ON}ͷϏσΦͱࢿྉ
    • https://www.elastic.co/elasticon/videos
    • αϙʔτϝχϡʔ
    • https://www.elastic.co/subscriptions

    View full-size slide

  74. Thank you!
    ● Web : https://www.elastic.co/jp/
    ● Forums : https://discuss.elastic.co/
    ● Twitter : @johtani

    View full-size slide