Elastic Stackでマイクロサービス運用を 楽にするには？ / Monitoring Microservices with Elastic Stack

Slide 1

Slide 1 text

!1 2019/01/25 Community Engineer @Elastic  Jun Ohtani @johtani Elastic StackͰϚΠΫϩαʔϏεӡ༻Λ  ָʹ͢Δʹ͸ʁ - Elastic Stackͷೖ໳ͱ׆༻ -

Slide 2

Slide 2 text

!2 about • Me, Jun Ohtani / Community Engineer ‒ lucene-gosenίϛολʔ ‒ σʔλ෼ੳج൫ߏஙೖ໳ ڞஶ ‒ http://blog.johtani.info  • Elastic, founded in 2012 ‒ Products: Elasticsearch, Logstash, Kibana, Beats   Elastic APM,   Elastic Cloud, Swiftype   Professional services: Support & development subscriptions  Trainings, Consulting, SaaS

Slide 3

Slide 3 text

!3 ΞδΣϯμ • ϚΠΫϩαʔϏεͱ͸ʁ • Elastic Stackͱ͸ʁ • ༷ʑͳ؍఺͔ΒͷΞϓϦέʔγϣϯͷ؂ࢹ • ͞Βʹ৭ʑࢼͯ͠ΈΔʹ͸ʁ

Slide 4

Slide 4 text

!4 ϚΠΫϩαʔϏεͱ͸

Slide 5

Slide 5 text

!5 ϚΠΫϩαʔϏε (Wikipedia) https://ja.wikipedia.org/wiki/ϚΠΫϩαʔϏε

Slide 6

Slide 6 text

!6 ϞϊϦγοΫ v.s. ϚΠΫϩαʔϏε

Slide 7

Slide 7 text

!7 ϚΠΫϩαʔϏεʁʁʁ

Slide 8

Slide 8 text

!8 Elastic Stackͱ͸ʁ

Slide 9

Slide 9 text

Elastic Stack อଘɺݕࡧɺ෼ੳ Elasticsearch ՄࢹԽɺ؅ཧ Kibana Beats ΠϯδΣετ Logstash

Slide 10

Slide 10 text

Metrics Logging APM Site  Search Application Search Business  Analytics Enterprise  Search Security  Analytics Future ιϦϡʔγϣϯ อଘɺݕࡧɺ෼ੳ ՄࢹԽɺ؅ཧ ΠϯδΣετ Kibana Elasticsearch Beats Logstash Elastic Stack

Slide 11

Slide 11 text

Metrics Logging APM Site  Search App  Search Business  Analytics Enterprise  Search Security  Analytics Future ιϦϡʔγϣϯ SaaS Elastic Cloud Self Managed Elastic Cloud  Enterprise Standalone σϓϩΠ อଘɺݕࡧɺ෼ੳ ՄࢹԽɺ؅ཧ ΠϯδΣετ Kibana Elasticsearch Beats Logstash Elastic Stack

Slide 12

Slide 12 text

อଘɺݕࡧɺ෼ੳ Elasticsearch ՄࢹԽɺ؅ཧ Kibana Beats ΠϯδΣετ Logstash Metrics Logging APM Site  Search Application Search Business  Analytics Enterprise  Search Security  Analytics Future ιϦϡʔγϣϯ SaaS Elastic Cloud Self Managed Elastic Cloud  Enterprise Standalone σϓϩΠ Elastic Stack

Slide 13

Slide 13 text

!13

Slide 14

Slide 14 text

14 Beats ܰྔσʔλγούʔ ιʔε͔ΒσʔλΛసૹ సૹ͠Elasticsearchʹू໿ ม׵ͱύʔεͷͨΊ Logstashʹసૹ Elastic Cloudʹసૹ Libbeat: ΧελϜbeatsͷͨ ΊͷAPIϑϨʔϜϫʔΫ 30Ҏ্ͷίϛϡχςΟbeats

Slide 15

Slide 15 text

The Beats family Heartbeat Uptime monitoring Filebeat Log files Winlogbeat Windows Event Logs Packetbeat Network data +40 community Beats Metricbeat Metrics Auditbeat Audit data

Slide 16

Slide 16 text

!16

Slide 17

Slide 17 text

17 Logstash σʔλՃ޻ύΠϓϥΠϯ શͯͷܗࣜɺαΠζͱσʔλιʔ εͷ౤ೖ ύʔεͱಈతͳ σʔλม׵ ͋ΒΏΔग़ྗʹ σʔλసૹ ҆શͰ҉߸Խ͞Εͨ  σʔλೖྗ ಠࣗͷύΠϓϥΠϯॲཧ ͷ࡞੒ 200Ҏ্ͷϓϥάΠϯ

Slide 18

Slide 18 text

!18

Slide 19

Slide 19 text

19 Elasticsearch Heart of the Elastic Stack ෼ࢄܕɺεέʔϥϒϧ ߴՄ༻ੑ Ϛϧνςφϯτ ։ൃऀϑϨϯυϦʔ ϦΞϧλΠϜɺશจݕࡧ ΞάϦήʔγϣϯ

Slide 20

Slide 20 text

Elasticsearchͱ͸ʁ

Slide 21

Slide 21 text

ϑϦʔϫʔυݕࡧ !21

Slide 22

Slide 22 text

ߜΓࠐΈ !22

Slide 23

Slide 23 text

ϋΠϥΠτ !23

Slide 24

Slide 24 text

ιʔτ !24

Slide 25

Slide 25 text

ϖʔδϯά !25

Slide 26

Slide 26 text

ूܭ !26

Slide 27

Slide 27 text

αδΣετ !27

Slide 28

Slide 28 text

Elasticsearch in 10 seconds • εΩʔϚϑϦʔɺ෼ࢄυΩϡϝϯτετΞɺREST & JSON • Φʔϓϯιʔε: Apache License 2.0 • ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ • JavaͰ࣮૷ɻ֦ு΋༰қ !28

Slide 29

Slide 29 text

؆୯ͳCRUD

Slide 30

Slide 30 text

σʔλొ࿥ 30 curl -XPUT localhost:9200/books/book/1 -d ' { "title" : "Elasticsearch - The definitive guide", "authors" : "Clinton Gormley", "started" : "2013-02-04", "pages" : 230 }'

Slide 31

Slide 31 text

σʔλߋ৽ 31 curl -XPUT localhost:9200/books/book/1 -d ' { "title" : "Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : "2013-02-04", "pages" : 230 }'

Slide 32

Slide 32 text

σʔλ࡟আ !32 curl -X DELETE localhost:9200/books/book/1 σʔλͷऔಘ curl —X GET localhost:9200/books/book/1 curl —X GET localhost:9200/books/book/1/_source

Slide 33

Slide 33 text

ݕࡧ - Query DSL !33 curl -XGET ‘localhost:9200/books/doc/_search' -d '{ "query": { "bool": { "must": [ { "match": { "title": "Search" }}, { "match": { "content": "Elasticsearch" }} ], "filter": [ { "term": { "status": "published" }}, { "range": { "publish_date": { "gte": "2015-01-01" }}} ] } } }'

Slide 34

Slide 34 text

෼ࢄߏ੒ɺ  εέʔϧ

Slide 35

Slide 35 text

γϟʔυͱϨϓϦΧ !35 node 1 orders products 1 4 1 2 2 3 curl -X PUT localhost:9200/orders -d '{ "settings.index.number_of_shards" : 4 "settings.index.number_of_replicas" : 1 }' curl -X PUT localhost:9200/products -d '{ "settings.index.number_of_shards" : 2 "settings.index.number_of_replicas" : 0 }'

Slide 36

Slide 36 text

γϟʔυͱϨϓϦΧ !36 node 1 orders products 1 4 1 node 2 orders products 2 2 3 4 1 2 3

Slide 37

Slide 37 text

ࣗಈతͳ෼ࢄ !37 node 1 orders products 2 1 4 1 node 2 orders products 2 2 node 3 orders products 3 4 1 3

Slide 38

Slide 38 text

!38

Slide 39

Slide 39 text

39 Kibana Window into the Elastic Stack ՄࢹԽͱ෼ੳ ஍ཧۭؒ ΧελϚΠζͱ Ϩϙʔτͷڞ༗ άϥϑ୳ࡧ Elastic Stack΁ͷ ηΩϡΞͳΞΫηεͱ؅ཧ ΧελϜAppsͷ࡞੒

Slide 40

Slide 40 text

!40 Kibana 6

Slide 41

Slide 41 text

!41 Elastic Stackͷߏ੒ Beats Log Files Metrics Wire Data your{beat} Kibana Instances Kafka Distributed Message Queue Notification Queues Storage Metrics Data Store Web APIs Social Sensors Elasticsearch Nodes Logstash Nodes

Slide 42

Slide 42 text

!42 ΞϓϦέʔγϣϯͷ  ؂ࢹϙΠϯτ

Slide 43

Slide 43 text

!43 ؂ࢹϙΠϯτ • ֎ܗ؂ࢹ • ϝτϦοΫʢϝτϦΫεʣ • αʔόʔɺΞϓϦέʔγϣϯ • ϩά • ΞϓϦέʔγϣϯͷϦϦʔελΠϛϯά • ෼ࢄτϨʔγϯά

Slide 44

Slide 44 text

!44 ֎ܗ؂ࢹ • ࢮ׆؂ࢹ • ϓϩηε • HTTPαʔόʔ • TCP • ICMP

Slide 45

Slide 45 text

Lightweight Shipper for Uptime Monitoring Heartbeat

Slide 46

Slide 46 text

!46 ϝτϦοΫ • αʔόʔ • CPUɺϝϞϦɺσΟεΫɺωοτϫʔΫI/Oɺϓϩηε਺ • ΞϓϦέʔγϣϯ • ϦΫΤετ਺ɺίωΫγϣϯ਺ɺॲཧ࣌ؒ • ίϯςφʔ • ίϯςφ਺

Slide 47

Slide 47 text

Collect system and application metrics Metricbeat

Slide 48

Slide 48 text

lots of modules Metricbeat

Slide 49

Slide 49 text

!49 Metricbeat Ϟδϡʔϧ ● Aerospike module ● Apache module ● Ceph module ● Couchbase module ● Docker module ● Dropwizard module ● Elasticsearch module ● Etcd module ● Golang module ● Graphite module ● HAProxy module ● HTTP module ● Jolokia module ● Kafka module ● Kibana module ● Kubernetes module ● kvm module ● Logstash module ● Memcached module ● MongoDB module ● Munin module ● MySQL module ● Nginx module ● ● PHP_FPM module ● PostgreSQL module ● Prometheus module ● RabbitMQ module ● Redis module ● System module ● uwsgi module ● vSphere module ● Windows module ● ZooKeeper module 

Slide 50

Slide 50 text

!50 ϩά • ΞΫηεϩά • γεςϜϩά • ೝূϩά • εϩʔϩά • ΞϓϦέʔγϣϯϩά

Slide 51

Slide 51 text

tail log from ﬁle Filebeat

Slide 52

Slide 52 text

many modules Filebeat

Slide 53

Slide 53 text

Filebeat modules - v6.4.2 • Apache2 module • Auditd module • Icinga module • IIS module • Kafka module • Logstash module • MongoDB module • MySQL module • Nginx module • Osquery module • PostgreSQL module • Redis module • System module • Traefik module

Slide 54

Slide 54 text

Welcome to 1998 winlogbeat

Slide 55

Slide 55 text

Now winlogbeat

Slide 56

Slide 56 text

Capture the Packet Packetbeat

Slide 57

Slide 57 text

Capture the Packet Packetbeat

Slide 58

Slide 58 text

!58 ΞϓϦέʔγϣϯͷϦϦʔελΠϛϯά • όάϑΟοΫεϦϦʔε • ৽ػೳϦϦʔε • ৽αʔϏε։࢝ • αʔόʔ૿ڧ

Slide 59

Slide 59 text

!59 ෼ࢄτϨʔγϯά • ϚΠΫϩαʔϏε • 1ͭͷϦΫΤετʹରͯ͠ෳ਺ͷϓϩηε͕ؔ܎ • ΞϓϦέʔγϣϯύϑΥʔϚϯεϞχλϦϯάͷ1ͭ

Slide 60

Slide 60 text

!60

Slide 61

Slide 61 text

!61 Elastic APM

Slide 62

Slide 62 text

Distributed Tracing Beta | Basic (free) શͯͷܭଌ͞ΕͨαʔϏεΛݟΔͨΊͷ ౷߹͞ΕͨϏϡʔ αϒίϯςΩετ಺ͷτϨʔεʹભҠ OpenTracing ޓ׵

Slide 63

Slide 63 text

!63 ͦͷ΄͔ͷศརͳػೳ • Infra UI • Logs UI • Machine Learning • Alerting

Slide 64

Slide 64 text

Infrastructure Solution Beta | Basic (free) ΠϯϑϥΦϖϨʔλʔ޲͚ʹಛԽ Λ௒͑ΔΠϯϑϥͷߏ੒Λ၆ᛌ ,VCFSOFUFTɺ%PDLFSͷωΠςΟϒαϙʔ τ ϝτϦοΫɺϩάɺ"1.Ϗϡʔ΁ͷ  υϦϧɾμ΢ϯ ΞυϗοΫ͓Αͼߏ଄Խݕࡧ

Slide 65

Slide 65 text

Logs Solution Beta | Basic (free) ϥΠϒͰϩάͷτϥϒϧγϡʔςΟϯά Λॿ͚ΔܰྔͳϩάϏϡʔΞʔ ίϯιʔϧͷΑ͏ͳදࣔ UBJMGͷΑ͏ͳ ϥΠϒɾϩάɾετϦʔ ϛϯά ཤྺϩάͷແݶεΫϩʔϧ ΞυϗοΫ͓Αͼߏ଄Խݕࡧ

Slide 66

Slide 66 text

!66

Slide 67

Slide 67 text

!67

Slide 68

Slide 68 text

!68

Slide 69

Slide 69 text

!69 ͞Βʹ׆༻͢Δʹ͸ʁ

Slide 70

Slide 70 text

!70

Slide 71

Slide 71 text

!71

Slide 72

Slide 72 text

ͦͷଞͷ࢖͍ํ !72

Slide 73

Slide 73 text

!73 σʔλͷొ࿥ํ๏ • Kibanaͷαϯϓϧσʔλʢ6.4͔Βʣ • LogstashͰJDBC input • LogstashͰCSV • FilebeatͰΞΫηεϩά • MetricbeatͰϝτϦοΫ • PacketbeatͰMySQL/PostgreSQLͷύέοτղੳ

Slide 74

Slide 74 text

!74 Kibanaͷαϯϓϧσʔλʢ>= 6.4.0ʣ

Slide 75

Slide 75 text

!75 ϫϯΫϦοΫͰσʔλొ࿥

Slide 76

Slide 76 text

!76 LogstashͰJDBC Input Kibana Instances Data Store Elasticsearch Nodes Logstash Nodes

Slide 77

Slide 77 text

!77 JDBC Input

Slide 78

Slide 78 text

!78 LogstashͰCSV Kibana Instances CSV  File Elasticsearch Nodes Logstash Nodes

Slide 79

Slide 79 text

!79 CSV filter

Slide 80

Slide 80 text

!80 FilebeatͰΞΫηεϩά Beats Log Files Kibana Instances Elasticsearch Nodes

Slide 81

Slide 81 text

• 2ͭͷElasticsearchϓϥάΠϯΛΠϯετʔϧͯ͠ElasticsearchΛىಈ • Filebeatͷapache2ϞδϡʔϧΛ༗ޮԽ • modules.d/apache2.ymlʹΞΫηεϩάͷύεΛઃఆ • setupίϚϯυΛ࣮ߦ͔ͯ͠ΒFilebeatΛىಈ !81 FilebeatͰΞΫηεϩά

Slide 82

Slide 82 text

MetricbeatͰϝτϦοΫ Beats Metrics Kibana Instances Elasticsearch Nodes

Slide 83

Slide 83 text

• MetricbeatͷsystemϞδϡʔϧΛ༗ޮԽ • setupίϚϯυΛ࣮ߦ͔ͯ͠ΒFilebeatΛىಈ !83 MetricbeatͰϝτϦοΫ

Slide 84

Slide 84 text

!84 PacketbeatͰMySQLɺPostgreSQLͷύέοτղੳ Beats Wire Data Kibana Instances Elasticsearch Nodes

Slide 85

Slide 85 text

!85 ࢀߟจݙ • Elasticsearch - The Definitive guide ‒ http://www.elastic.co/guide/en/elasticsearch/guide/current/ index.html • ॻ੶ʢ೔ຊޠʣ ‒ σʔλ෼ੳج൫ߏஙೖ໳ ‒ Elasticsearch࣮ફΨΠυ

Slide 86

Slide 86 text

!86 ࢀߟจݙ • ೖ໳ ؂ࢹ  ―ϞμϯͳϞχλϦϯάͷͨΊͷσβΠϯύλʔϯ  Mike JulianɹஶɺদӜ ൏ਓɹ༁    https://www.oreilly.co.jp/books/9784873118642/

Slide 87

Slide 87 text

!87 ࢀߟαΠτ • Ϣʔεέʔε • https://www.elastic.co/use-cases • DiscussʢWebϑΥʔϥϜʣ • https://discuss.elastic.co • Elastic{ON}ͷϏσΦͱࢿྉ • https://www.elastic.co/elasticon/videos • αϙʔτϝχϡʔ • https://www.elastic.co/subscriptions

Slide 88

Slide 88 text

Thank you! ● Web : https://www.elastic.co/jp/ ● Forums : https://discuss.elastic.co/ ● Twitter : @johtani