Slide 1
Slide 1 text
Shield your cluster
Security with Elasticsearch
Alexander Reelsen
@spinscale
[email protected]
Slide 2
Slide 2 text
Agenda
Why?
How?
Next?
What?
Who?
Q & A
Slide 3
Slide 3 text
About
2012
Elasticsearch got founded
Series A investment
Trainings
Supports subscriptions
Slide 4
Slide 4 text
About
2012
Series B investment
Kibana
Elasticsearch for Apache Hadoop Integration
Logstash
Elasticsearch Clients
2013
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.
Slide 5
Slide 5 text
About
2012
Series C investment
Marvel released
2013 2014
Slide 6
Slide 6 text
About
2012
Shield goes GA
First user conference & rebrand
Found acquired
Packetbeat joins
Watcher in beta
2013 2014 2015
Slide 7
Slide 7 text
About
2012 2013 2014 2015
Joined in March 2013
Working on Elasticsearch & Shield
Development, Trainings, Conferences, Support, Blog posts
We're hiring...
Slide 8
Slide 8 text
Why?
How?
Next?
What?
Who?
Q & A
Slide 9
Slide 9 text
Why?
Elasticsearch: No security OOTB
No encrypted communication
No Authorization
No Authentication
No Audit Logging
Slide 10
Slide 10 text
ES
nginx
client
Filter by HTTP method, URI or IP
User management via basic auth
Use aliases & filters
nginx in front
Slide 11
Slide 11 text
ES
nginx
client
How to solve multi index operations?
nginx in front
GET /logs-2015.10.10,evil,logs-2015.10.11
{
"query" : { "match_all": {} }
}
Slide 12
Slide 12 text
ES
nginx
client
How to solve bulk/multi operations?
nginx in front
{ "index" : { "_index" : "test1", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test2", "_type" : "type1", "_id" : "2" } }
{ "create" : { "_index" : "test3", "_type" : "type1", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "test4"} }
{ "doc" : {"field2" : "value2"} }
Slide 13
Slide 13 text
ES
nginx
client
Prevent unwanted accesses
nginx in front
HTTP/Transport
Slide 14
Slide 14 text
ES
nginx
client
nginx in front
Firewall
Slide 15
Slide 15 text
Data
ACL
client
Configuration scattered across systems
operational overhead
IP Filtering
Slide 16
Slide 16 text
Data
ACL
client
operational overhead
IP Filtering Directory
Configuration scattered across systems
Slide 17
Slide 17 text
Why?
How?
Next?
What?
Who?
Q & A
Slide 18
Slide 18 text
How?
Elasticsearch modular & pluggable
Security as a plugin
HTTP + Transport protocols
Integration into the ELK stack!
Slide 19
Slide 19 text
How?
Elasticsearch
Elasticsearch
auth_token Authentication
Authorization
Slide 20
Slide 20 text
How?
Elasticsearch
Elasticsearch
auth_token
200 OK
Authentication
Authorization
Slide 21
Slide 21 text
How?
Elasticsearch
Elasticsearch
auth_token
401 Unauthorized
Authentication
Authorization
Slide 22
Slide 22 text
How?
Getting up and running is easy
Install elasticsearch 1.6
bin/plugin install elasticsearch/license/latest
bin/plugin install elasticsearch/shield/latest
Slide 23
Slide 23 text
Why?
How?
Next?
What?
Who?
Q & A
Slide 24
Slide 24 text
What?
IP Filtering
Encrypted communication
Authentication
Authorization
Audit Trail
Slide 25
Slide 25 text
Configurable in elasticsearch.yml
Can be updated dynamically via
cluster update settings API
IP Filtering
shield.transport.filter:
allow: "192.168.0.1"
deny: "192.168.0.0/24"
Slide 26
Slide 26 text
keystore required
different config for HTTP and
transport protocol (+profiles)
Encrypted communication
shield.ssl.keystore.path: /path/to/keystore.jks
shield.ssl.keystore.password: secret
shield.transport.ssl: true
shield.http.ssl: true
Slide 27
Slide 27 text
Authentication
"Who are you?"
Auth mechanisms are called realms
Available: esusers, ldap, ad, pki
Realms can be chained
Support for caching & API for clearing
Slide 28
Slide 28 text
Authentication
shield.authc:
realms:
esusers:
type: esusers
order: 0
ldap1:
type: ldap
order: 1
enabled: false
url: 'url_to_ldap1'
...
ad1:
type: active_directory
order: 3
url: 'url_to_ad'
Slide 29
Slide 29 text
ESusers realm
Local files, can be changed via CLI
Elasticsearch watches file changes &
reloads
config/shield/users
config/shield/users_roles
Slide 30
Slide 30 text
ESusers realm
bin/shield/esusers useradd alex
bin/shield/esusers roles alex -a
admin -r user
bin/shield/esusers list
bin/shield/esusers userdel alex
Slide 31
Slide 31 text
Fallback to configurable user
Disabled by default
Anonymous access
shield.authc:
anonymous:
username: anonymous_user
roles: role1, role2
Slide 32
Slide 32 text
Authorization
"Are you allowed to do that?"
File: config/shield/roles.yml
admin:
cluster: all
indices:
'*': all
Slide 33
Slide 33 text
Role Based Access Control
role
named set of permissions
permission
set of cluster wide privileges
set of indices/aliases specific privileges
privilege
set of one or more action names
/_search ⬌ indices:data/read/search
Slide 34
Slide 34 text
Role Based Access Control
admin:
cluster: all
indices:
'*': all
role permission
Slide 35
Slide 35 text
Authorization
user:
indices:
'*': read
events_user:
indices:
'events_*': read
Slide 36
Slide 36 text
Authorization
get_user:
indices:
'events_index': 'indices:data/read/get'
logfile_user_readonly:
indices:
"logstash-201?-*": read
Slide 37
Slide 37 text
Audit Trail
Writes an own audit log file
Implemented as logger
Logs different types of event based
on log level
(ip filtering, tampered requests, access denied, auth failed)
shield.audit.enabled: true
Slide 38
Slide 38 text
Integration
Transport Client
Logstash
Kibana 3/4
Watcher
Marvel
Slide 39
Slide 39 text
Transport Client
TransportClient client = new TransportClient(builder()
.put("cluster.name", "myClusterName")
.put("shield.user", "test_user:changeme")
.put("shield.ssl.keystore.path", "/path/to/client.jks")
.put("shield.ssl.keystore.password", "password")
.put("shield.transport.ssl", "true"))
.addTransportAddress(new
InetSocketTransportAddress("localhost", 9300));
Slide 40
Slide 40 text
Why?
How?
Next?
What?
Who?
Q & A
Slide 41
Slide 41 text
Who?
Use-case 1: Monitoring application
No write access
Cluster Health
Nodes stats/info
Indices Stats
Slide 42
Slide 42 text
Use-case 2: Logstash
No read access (unless input is used)
Indices: Indexing
Cluster: Index templates
Slide 43
Slide 43 text
Use-case 3: Marvel
marvel_user:
cluster: cluster:monitor/nodes/info,
cluster:admin/plugin/license/get
indices:
'.marvel-*': all
marvel_agent:
cluster: indices:admin/template/get,
indices:admin/template/put
indices:
'.marvel-*': indices:data/write/bulk, create_index
Slide 44
Slide 44 text
Use-case 4: Ecommerce
bulk:
indices:
'products_*': write, manage, read
updater:
indices:
'products': index, delete, indices:admin/optimize
webshop:
indices:
'products': search, get
Slide 45
Slide 45 text
Use-case 4: Ecommerce
monitoring:
cluster: monitor
indices:
'*': monitor
sales_rep :
indices:
'sales_*' : all
'social_events' : data_access, monitor
Slide 46
Slide 46 text
Why?
How?
Next?
What?
Who?
Q & A
Slide 47
Slide 47 text
Next?
Simplify SSL configuration
API driven user/role management
Open up realms API
Field-level security
Index Audit Trail into ES
Slide 48
Slide 48 text
Why?
How?
Next?
What?
Who?
Q & A
Slide 49
Slide 49 text
Q & A
Thanks for listening!
Alexander Reelsen
@spinscale
[email protected]
We're hiring
https://www.elastic.co/about/careers
We're helping
https://www.elastic.co/subscriptions
Slide 50
Slide 50 text
Resources
Shield documentation
https://www.elastic.co/guide/en/shield/current/index.html
Shield: Security in ELK
https://www.elastic.co/elasticon/2015/sf/security-in-elk
Shield and Beyond:
Recommendations for a Secure ELK
Environment
https://www.elastic.co/webinars/shield-and-beyond
Slide 51
Slide 51 text
Resources
https://discuss.elastic.co/c/shield
Slide 52
Slide 52 text
Resources
Slide 53
Slide 53 text
Resources
Slide 54
Slide 54 text
Q & A
Thanks for listening!
Alexander Reelsen
@spinscale
[email protected]
We're hiring
https://www.elastic.co/about/careers
We're helping
https://www.elastic.co/subscriptions