Security, Alerting, Monitoring, and More With the Elastic Stack

Dd9d954997353b37b4c2684f478192d3?s=47 Elastic Co
February 18, 2016

Security, Alerting, Monitoring, and More With the Elastic Stack

Elastic's Jay Modi, Alexander Reelsen, Joe Fleming, Tanguy Leroux, and Tim Sullivan present X-Pack — and all the cool features it brings to the Elastic Stack — at Elastic{ON}16 in San Francisco on February 18, 2016.

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

February 18, 2016
Tweet

Transcript

  1. 3.

    ‹#› Logging and monitoring are at the heart of making

    sure your solutions are up and running to your expectations. - David Messina VP, Enterprise Marketing at Docker’s VP of Enterprise Marketing
  2. 6.
  3. 7.
  4. 12.
  5. 16.

    16 Security for the Elastic Stack • Username/password protection Simply

    Secure Elasticsearch Advanced Security When Needed • LDAP, Active Directory, and PKI integration • Role-based access control • Field and document level security • Encrypted communication • Auditing
  6. 17.

    Adding Users (now) Command Line Utility 17 $ bin/shield/esusers useradd

    jaymode -r admin Enter new password: Retype new password:
  7. 18.

    Security APIs User and Role management 18 curl –XPUT localhost:9200/_shield/user/jaymode

    –d ‘ { "roles" : ["engineering", "security"], "password" : "changeme" }’ curl –XPUT localhost:9200/_shield/role/security –d ‘ { "cluster": ["all"], "indices": [ { "names": ".shield_audit_log-*",
 "privileges": ["all"] } ] }’
  8. 26.

    ‹#› Notify me on chat, if we have over 1000

    orders per hour - The Startup CEO
  9. 27.

    ‹#› Trigger an alert, when the same IP accesses all

    services in a certain interval - Your Admin
  10. 29.

    ‹#› 5% traffic increase in the last 5 minutes. Ping

    folks on chat! - Your Loadbalancer
  11. 30.
  12. 31.

    ‹#› Can you predict the additional system resources for the

    next two weeks? - Every system architect ever
  13. 32.

    32 A Watch consists of… • Trigger • Input •

    Condition • Actions • Metadata • Transformation
  14. 34.

    { "trigger" : { "schedule" : { "interval" : "10s"

    } } } 34 PUT /_watcher/watch/cluster_health
  15. 35.

    { "trigger" : { "schedule" : { "interval" : "10s"

    } }, "input" : { "http" : { "request" : { "url" : "http://localhost:9200/_cluster/health" } } } } 35 PUT /_watcher/watch/cluster_health
  16. 36.

    { "trigger" : { "schedule" : { "interval" : "10s"

    } }, "input" : { "http" : { "request" : { "url" : "http://localhost:9200/_cluster/health" } } }, "condition" : { "compare" : { "ctx.payload.status" : { "eq" : "red" } } } } 36 PUT /_watcher/watch/cluster_health
  17. 37.

    { "trigger" : { "schedule" : { "interval" : "10s"

    } }, "input" : { "http" : { "request" : { "url" : "http://localhost:9200/_cluster/health" } } }, "condition" : { "compare" : { "ctx.payload.status" : { "eq" : "red" } } }, "actions" : { "send_email" : { "email" : { "to" : "admin@example.org", "subject" : "Cluster Status Warning", "body" : "Cluster status is RED" } } } } 37 PUT /_watcher/watch/cluster_health
  18. 39.

    2.0: Hipchat action 39 "actions" : { "notify-hipchat" : {

    "hipchat" : { "account" : "integration-account", "message" : { "body" : "@{{ctx.metadata.userOnDuty}} Encountered {{ctx.payload.hits.total}} errors in the last 5 minutes (facepalm)", "format" : "text", "color" : "red", "notify" : true } } } }
  19. 40.

    2.0: Slack action 40 "actions" : { "notify-slack" : {

    "slack" : { "message" : { "from" : "watcher", "to" : [ "#admins", "#errors" ] , "text" : "Monitoring incident", "attachments" : [ { "text" : "@{{ctx.metadata.userOnDuty}} Encountered {{ctx.payload.hits.total}} errors in the last 5 minutes (facepalm)", "title" : "text", "color" : "danger" ] } ...
  20. 42.

    2.0: Array Compare 42 "condition": { "array_compare": { "ctx.payload.aggregations.top_tweeters.buckets" :

    { "path": "doc_count", "gte": { "value": 25, "quantifier": "some" } } } }
  21. 43.

    2.1: Chained inputs 43 "input" : { "chain": { "inputs":

    [ { "first": { "simple" : { "path" : "/_search" } } }, { "second": { "http" : { "request" : { … } } } } ] } } ... }
  22. 45.

    2.3: PagerDuty action 45 "actions" : { "notify-pagerduty" : {

    "pagerduty" : { "message" : { "description" : "Main system down, please check! Happened at {{ctx.execution_time}}", "client" : "/foo/bar/{{ctx.watch_id}}", "attach_payload" : true, "context" : [ { "type" : "response", "href" : "http://www.test.de/foo" } ] ...
  23. 46.

    2.3: External email attachments 46 "actions" : { "email_admin" :

    { "email" : { "to" : "ceo@example.org", "attachments" : { "my_id" : { "http" : { "request" : { "url" : "http://example.org/daily-report.pdf" } } } } } ...
  24. 53.
  25. 54.

    54

  26. 55.

    55

  27. 57.
  28. 58.
  29. 59.

    Delivery with Alerting 59 "actions" : { "email_admin" : {

    "email" : { "to" : "ceo@example.org", "attachments" : { "my_id" : { "http" : { "request" : { "url" : "http://example.org/daily-report.pdf" } } } } } ...
  30. 65.

    65

  31. 66.

    ‹#› Monitoring Elasticsearch Securing Elasticsearch From Dashboard to PDF BoF:

    Alerting & Notifications Spotlight Theater: 2:40pm Spotlight Theater: 3:40pm Spotlight Theater: 4:40pm Lunch Area: Friday at 11:00am