Elastic Stack Workshop

Elastic Stack Workshop

One of the hottest and most widely used tools for log monitoring and analytics at the moment is the Elastic Stack:
• Elasticsearch doing the hard work of analyzing and searching large amounts of data.
• Logstash and Beats for collecting them.
• Kibana for powerful visualizations.

This workshop gives you an overview of the four technologies, how they are working together, and how they can solve your problems.

Demo: https://github.com/xeraa/vagrant-elastic-stack

Ce4685da897c912aa41a815435b40a5a?s=128

Philipp Krenn

June 28, 2017
Tweet

Transcript

  1. Awesome Logging Infrastructure Using The Stack Philipp Krenn@xeraa 1

  2. Infrastructure | Developer Advocate 2

  3. Disclaimer This is not a training https://www.elastic.co/training 3

  4. Who is using Elasticsearch Logstash and Kibana Beats X-Pack 4

  5. Starting point https://github.com/xeraa/vagrant-elastic-stack 5

  6. USB Sticks 6

  7. Box Vagrant Ansible Provisioner 7

  8. Credentials vagrant & vagrant 8

  9. SSH $ ssh vagrant@127.0.0.1 -p 2222 -o PreferredAuthentications=password Windows: http://www.putty.org

    9
  10. Ansible $ cd /elastic-stack/ $ ls 10

  11. 11

  12. 12

  13. REST $ curl -XGET -u "elastic:changeme" http://localhost:9200/ 13

  14. 14

  15. Login http://localhost:5601 Login: elastic & changeme 15

  16. Overview GET / GET _cat GET _cat/indices?v 16

  17. Insert data PUT /movies/movie/1 { "title": "The Godfather", "director": "Francis

    Ford Coppola", "year": 1972 } GET /movies/movie/1 GET /movies/_mapping 17
  18. Replace data PUT /movies/movie/1 { "title": "The Godfather", "director": "Francis

    Ford Coppola", "year": 1972, "genres": ["Crime", "Drama"] } GET /movies/movie/1 18
  19. More data PUT /movies/movie/2 { "title": "Lawrence of Arabia", "director":

    "David Lean", "year": 1962, "genres": ["Adventure", "Biography", "Drama"] } PUT /movies/movie/3 { "title": "Apocalypse Now", "director": "Francis Ford Coppola", "year": 1979, "genres": ["Drama", "War"] } 19
  20. Query endpoints /_search /movies/_search /movies/movie/_search 20

  21. Queries POST /movies/_search { "query": { "query_string": { "query": "ford"

    } } } 21
  22. Filter POST /movies/_search { "query": { "bool": { "filter": {

    "term": { "year": 1972 } } } } } 22
  23. Cleanup DELETE /movies GET /movies/movie/1 23

  24. Insert test data $ java -jar /opt/injector.jar 100000 1000 24

  25. Overview GET _cat/indices?v 25

  26. Search GET /person/person/_search { "query": { "match": { "address.country": "Germany"

    } } } 26
  27. More complex search GET /person/person/_search { "query": { "bool": {

    "must": [ { "match": { "address.country": "Germany" } }, { "range": { "dateOfBirth": { "from": "1970", "to": "1971" } } } ] } } } 27
  28. Aggregation GET /person/person/_search { "size": 0, "aggs": { "by_country": {

    "terms": { "field": "address.country" } } } } 28
  29. Index pattern in Kibana Index name person Time-field name dateOfBirth

    29
  30. Kibana Discover 30

  31. Kibana Visualize Vertical bar chart with a date histogram Save

    31
  32. Kibana Visualize Pie chart split on the gender Save 32

  33. Kibana Visualize Pie chart split on the country and then

    city Save 33
  34. Kibana Visualize Tile map Save 34

  35. Kibana Dashboard Combine all the saved visualizations 35

  36. 36

  37. Filebeat 37

  38. Filebeat Modules 38

  39. /var/log/kibana/kibana.log JSON messages Limit Kibana view to its type 39

  40. Kibana Discover Limit Kibana view to the nginx-access type 40

  41. 41

  42. /var/log/syslog Logstash filter 42

  43. Collect nginx in /etc/filebeat/filebeat.yml - input_type: log paths: - /var/log/nginx/access.log

    document_type: nginx-access 43
  44. Disable Filebeat modules #filebeat.modules: #- module: system #- module: nginx

    44
  45. Enable the Logstash output output.logstash: hosts: ["localhost:5044"] username: "elastic" password:

    "changeme" 45
  46. Disable the Elasticsearch output #output.elasticsearch: # hosts: ["localhost:9200"] # username:

    "elastic" # password: "changeme" 46
  47. Filebeat Restart $ sudo service filebeat restart 47

  48. Logstash pattern /opt/logstash/patterns/nginx 48

  49. Logstash Filter $ sudo tee -a /etc/logstash/conf.d/11-nginx-filter.conf >/dev/null <<'EOF' filter

    { if [type] == "nginx-access" { grok { patterns_dir => ["/opt/logstash/patterns"] match => { "message" => "%{NGINXACCESS}" } } } } EOF $ sudo service logstash restart 49
  50. Debug Logstash $ less /var/log/logstash/logstash-plain.log 50

  51. Metricbeat 51

  52. Metricbeat System 52

  53. Metricbeat Service 53

  54. Visual Builder Docker network traffic 54

  55. Packetbeat 55

  56. Protocols 56

  57. Flows Application layer: Unsupported / encrypted (TLS) protocols IP /

    TCP / UDP Number of packets & bytes Retransmissions Temporal flow 57
  58. Heartbeat 58

  59. Heartbeat Open Heartbeat dashboard and set to auto-refresh $ sudo

    service nginx stop 59
  60. Winlogbeat 60

  61. libbeat https://github.com/elastic/beats/tree/master/generate/beat 61

  62. 62

  63. X-Pack Monitoring Graph Reporting Alerting Machine Learning 63

  64. X-Pack Basic 64

  65. Monitoring GET _nodes/stats { "_nodes" : { "total" : 1,

    "successful" : 1, "failed" : 0 }, "cluster_name" : "elasticsearch", "nodes" : { "Koy5OmQ5RoiuFlL_TQ2Ngg" : { "timestamp" : 1492549940504, "name" : "Koy5OmQ", ... 65
  66. 66

  67. Search Profiler GET /person "profile": true, "size": 0, "query": {

    ... } ... "searches" : [ { "query" : [ { "type" : "BooleanQuery", "description" : "...", "time" : "2.366942000ms", "time_in_nanos" : 2366942, "breakdown" : { "score" : 0, "build_scorer_count" : 5, ... 67
  68. 68

  69. 69

  70. Install $ bin/elasticsearch-plugin install x-pack $ bin/kibana-plugin install x-pack $

    bin/logstash-plugin install x-pack 70
  71. GET _xpack/license { "license" : { "status" : "active", "uid"

    : "...", "type" : "trial", "issue_date" : "2017-04-18T20:41:50.233Z", "issue_date_in_millis" : 1492548110233, "expiry_date" : "2017-05-18T20:41:50.233Z", "expiry_date_in_millis" : 1495140110233, "max_nodes" : 1000, "issued_to" : "elasticsearch", "issuer" : "elasticsearch", "start_date_in_millis" : -1 } } 71
  72. 72

  73. 73

  74. Install $ curl -XPUT -u elastic:changeme 'http://localhost:9200/_xpack/license?acknowledge=true' -H "Content-Type: application/json"

    -d @philipp-krenn-...-v5.json 74
  75. GET _xpack/license { "license" : { "status" : "active", "uid"

    : "...", "type" : "basic", "issue_date" : "2017-04-18T00:00:00.000Z", "issue_date_in_millis" : 1492473600000, "expiry_date" : "2018-04-18T23:59:59.999Z", "expiry_date_in_millis" : 1524095999999, "max_nodes" : 100, "issued_to" : "Philipp Krenn (Elastic)", "issuer" : "Web Form", "start_date_in_millis" : 1492473600000 } } 75
  76. Conclusion 76

  77. 77

  78. 78

  79. 79

  80. 80

  81. Opbeat 81

  82. PS: More Open Source https://www.elastic.co/training Development Support Consulting Production Support

    82
  83. Thanks! Questions? Philipp Krenn@xeraa 83

  84. Container ship: https://flic.kr/p/hjxW62 https://flic.kr/p/2AzAVJ Wooden logs: https://flic.kr/p/9vvbKE Files: https://flic.kr/p/2EFcQ Metric:

    https://flic.kr/p/9g5h3f Packages: https://flic.kr/p/cJFDLN Windows: https://flic.kr/p/94Z6y Library: https://flic.kr/p/fiXcBj 84