Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Security, Alerting, Monitoring, and More With the Elastic Stack

Elastic Co
February 18, 2016

Security, Alerting, Monitoring, and More With the Elastic Stack

Elastic's Jay Modi, Alexander Reelsen, Joe Fleming, Tanguy Leroux, and Tim Sullivan present X-Pack — and all the cool features it brings to the Elastic Stack — at Elastic{ON}16 in San Francisco on February 18, 2016.

Elastic Co

February 18, 2016
Tweet

More Decks by Elastic Co

Other Decks in Technology

Transcript

  1. ‹#› Security, Alerting, Monitoring and More with the Elastic Stack

  2. Monitoring Tanguy Leroux Tim Sullivan @tlrx @timsullivanyeah All the Things

  3. ‹#› Logging and monitoring are at the heart of making

    sure your solutions are up and running to your expectations. - David Messina VP, Enterprise Marketing at Docker’s VP of Enterprise Marketing
  4. ‹#› And then Marvel 1.0 Arrived First commercial plugin of

    Elastic!
  5. Cluster Pulse 5

  6. ‹#›

  7. ‹#›

  8. ‹#› Monitoring for Elasticsearch 2.x Yes, we like to rename

    things :)
  9. Multi-Cluster Support 9

  10. Cluster Overview at a Glance 10

  11. Node Listing 11

  12. ‹#›

  13. 19 Kibana 5 New design Issues Cross-Stack Monitoring

  14. ‹#› Spotlight Theater @ 4:40pm Monitoring Elasticsearch

  15. Security Don’t Hack Me Bro! Jay Modi @jaymode2001

  16. 16 Security for the Elastic Stack • Username/password protection Simply

    Secure Elasticsearch Advanced Security When Needed • LDAP, Active Directory, and PKI integration • Role-based access control • Field and document level security • Encrypted communication • Auditing
  17. Adding Users (now) Command Line Utility 17 $ bin/shield/esusers useradd

    jaymode -r admin Enter new password: Retype new password:
  18. Security APIs User and Role management 18 curl –XPUT localhost:9200/_shield/user/jaymode

    –d ‘ { "roles" : ["engineering", "security"], "password" : "changeme" }’ curl –XPUT localhost:9200/_shield/role/security –d ‘ { "cluster": ["all"], "indices": [ { "names": ".shield_audit_log-*",
 "privileges": ["all"] } ] }’
  19. Kibana Sessions and Login Screen 19

  20. User and Roles UI 20

  21. ‹#› Security Configuration API & UI

  22. ‹#› Kibana Security

  23. ‹#› Built In Users

  24. ‹#› Spotlight Theater @ 2:40pm Securing Elasticsearch

  25. Alerting Watch This! Alexander Reelsen @spinscale

  26. ‹#› Notify me on chat, if we have over 1000

    orders per hour - The Startup CEO
  27. ‹#› Trigger an alert, when the same IP accesses all

    services in a certain interval - Your Admin
  28. ‹#› Email me when the product is back in stock!

    - Desperate Online Shopper
  29. ‹#› 5% traffic increase in the last 5 minutes. Ping

    folks on chat! - Your Loadbalancer
  30. ‹#› 5% traffic increase in the last 5 minutes at

    2am. Pager time! - Not your SO
  31. ‹#› Can you predict the additional system resources for the

    next two weeks? - Every system architect ever
  32. 32 A Watch consists of… • Trigger • Input •

    Condition • Actions • Metadata • Transformation
  33. { } 33 PUT /_watcher/watch/cluster_health

  34. { "trigger" : { "schedule" : { "interval" : "10s"

    } } } 34 PUT /_watcher/watch/cluster_health
  35. { "trigger" : { "schedule" : { "interval" : "10s"

    } }, "input" : { "http" : { "request" : { "url" : "http://localhost:9200/_cluster/health" } } } } 35 PUT /_watcher/watch/cluster_health
  36. { "trigger" : { "schedule" : { "interval" : "10s"

    } }, "input" : { "http" : { "request" : { "url" : "http://localhost:9200/_cluster/health" } } }, "condition" : { "compare" : { "ctx.payload.status" : { "eq" : "red" } } } } 36 PUT /_watcher/watch/cluster_health
  37. { "trigger" : { "schedule" : { "interval" : "10s"

    } }, "input" : { "http" : { "request" : { "url" : "http://localhost:9200/_cluster/health" } } }, "condition" : { "compare" : { "ctx.payload.status" : { "eq" : "red" } } }, "actions" : { "send_email" : { "email" : { "to" : "[email protected]", "subject" : "Cluster Status Warning", "body" : "Cluster status is RED" } } } } 37 PUT /_watcher/watch/cluster_health
  38. ‹#› Recap

  39. 2.0: Hipchat action 39 "actions" : { "notify-hipchat" : {

    "hipchat" : { "account" : "integration-account", "message" : { "body" : "@{{ctx.metadata.userOnDuty}} Encountered {{ctx.payload.hits.total}} errors in the last 5 minutes (facepalm)", "format" : "text", "color" : "red", "notify" : true } } } }
  40. 2.0: Slack action 40 "actions" : { "notify-slack" : {

    "slack" : { "message" : { "from" : "watcher", "to" : [ "#admins", "#errors" ] , "text" : "Monitoring incident", "attachments" : [ { "text" : "@{{ctx.metadata.userOnDuty}} Encountered {{ctx.payload.hits.total}} errors in the last 5 minutes (facepalm)", "title" : "text", "color" : "danger" ] } ...
  41. 2.0: Activate/Deactivate REST API 41 PUT /_watcher/watch/<watch_id>/_activate PUT /_watcher/watch/<watch_id>/_deactivate

  42. 2.0: Array Compare 42 "condition": { "array_compare": { "ctx.payload.aggregations.top_tweeters.buckets" :

    { "path": "doc_count", "gte": { "value": 25, "quantifier": "some" } } } }
  43. 2.1: Chained inputs 43 "input" : { "chain": { "inputs":

    [ { "first": { "simple" : { "path" : "/_search" } } }, { "second": { "http" : { "request" : { … } } } } ] } } ... }
  44. 2.1: Chained inputs 44 {{ctx.payload.second.hits.total}} {{ctx.payload.first.path}} "input" "chain" "inputs" "first"

    "second" } } ... }
  45. 2.3: PagerDuty action 45 "actions" : { "notify-pagerduty" : {

    "pagerduty" : { "message" : { "description" : "Main system down, please check! Happened at {{ctx.execution_time}}", "client" : "/foo/bar/{{ctx.watch_id}}", "attach_payload" : true, "context" : [ { "type" : "response", "href" : "http://www.test.de/foo" } ] ...
  46. 2.3: External email attachments 46 "actions" : { "email_admin" :

    { "email" : { "to" : "[email protected]", "attachments" : { "my_id" : { "http" : { "request" : { "url" : "http://example.org/daily-report.pdf" } } } } } ...
  47. ‹#› What's next?

  48. ‹#› Curator

  49. ‹#› Watcher UI

  50. ‹#› Actions

  51. ‹#› Friday, Lunch area @ 11:00am BoF: Alerting & Notifications

    Share Your Watcher Stories
  52. Reporting Kibana For Your Inbox Joe Fleming @w33ble

  53. ‹#› I need this information. Can you send me a

    report? - Every Manager Ever
  54. 54

  55. 55

  56. ‹#› Network’s down. Meeting’s over. - No Manager Ever

  57. None
  58. None
  59. Delivery with Alerting 59 "actions" : { "email_admin" : {

    "email" : { "to" : "[email protected]", "attachments" : { "my_id" : { "http" : { "request" : { "url" : "http://example.org/daily-report.pdf" } } } } } ...
  60. ‹#› 5.0 Alpha 1

  61. ‹#› Distributed Rendering

  62. ‹#› Administrative Control

  63. ‹#› Historical Archive

  64. ‹#› Spotlight Theater @ 3:40pm From Dashboard to PDF Generate

    Reports with the Elastic Stack
  65. 65

  66. ‹#› Monitoring Elasticsearch Securing Elasticsearch From Dashboard to PDF BoF:

    Alerting & Notifications Spotlight Theater: 2:40pm Spotlight Theater: 3:40pm Spotlight Theater: 4:40pm Lunch Area: Friday at 11:00am