Securing Elasticsearch

Elastic's Jay Modi and Igor Motov discuss securing Elasticsearch at Elastic{ON}16 in San Francisco, February 18, 2016.

Transcript

1. ‹#› Igor Motov Jay Modi Securing Elasticsearch @imotov @jaymode2001
2. or why REST-based security solutions rarely work 2 Pitfalls in Securing Search
3. REST approach to securing elasticsearch • Most of the clients are using HTTP protocol • Easy to implement using standard HTTP reverse proxy • Many elasticsearch operation are REST-compliant • PUT /index/type/doc_id - indexes a document • GET /index/type/doc_id - gets a document 3
4. REST-based security architecture 4 node1 node2 transport client HTTP client
5. REST-based security architecture 5 node1 node2 transport client HTTP client Reverse Proxy
6. Security perimeter REST-based security architecture 6 node1 node2 transport client HTTP client Reverse Proxy
7. Security perimeter REST-based security architecture 7 node1 node2 transport client HTTP client Reverse Proxy
8. Works great for authentication 8
9. 9 Frequently fails for authorization
10. Limiting access to an index • Easy! Just setup the following url matching patterns: • GET /myindex/*/* • PUT /myindex/*/* • POST /myindex/*/* 10 by using url matching
11. URL doesn’t always limit the scope of all operation by default! • Easy! Just setup the following url matching patterns: • GET /myindex/*/* • PUT /myindex/*/* • POST /myindex/*/* • But what about? • POST /myindex/my_type/_bulk • GET /myindex/_msearch • GET /myindex/_mget 11
12. URL doesn’t always limit the scope of all operation by default! rest.action.multi.allow_explicit_index: false 12
13. Search is not always limited to the index on the URL • Cross index search operation: • More like this query • Terms lookup mechanism in the terms query • Percolation • …. 13 Source: Gray Arial 10pt
14. Be really careful with URL matching rules! 14
15. 15 Document - level security
16. A typical solution for document level access control • Add an ACL token(s) to each document • Create an alias with ACL filters for each user • Limit users access to their aliases 16 Source: Gray Arial 10pt
17. Filtered Alias Limitations • Get and Put operations don’t check filters at all: • GET /my_alias/doc/1 • PUT /my_alias/doc/1 • Aliases are not considered in • Suggest API • Children Aggs 17 Source: Gray Arial 10pt
18. Filtered aliases are not a security mechanism 18
19. 19 Field - level security
20. A naive solution - filtering of source in response { "id" : "123456", "name": "imotov", "secret_part": { "favorite_color": "green" } } 20 Source: Gray Arial 10pt
21. A naive solution - filtering of source in response /_search?_source_exclude=secret_part.* 21 Source: Gray Arial 10pt
22. If you can search it - you have access to it 22
23. Naive solution - filtering of source in response { "query": { "match": { "id": "123456" } }, "aggs" : { "catch_me_if_you_can": { "terms": { "field": "secret_part.favorite_color" } } } } 23
24. Naive solution - filtering of source in response { "aggregations" : { "catch_me_if_you_can" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 0, "buckets" : [ { "key" : "green", "doc_count" : 1 } ] } } } 24
25. The problem with source filtering • You can search a field in number of different ways • query string query (in the text of the query) • multi match query (with wildcards) • aggregations • suggestions • … 25 Source: Gray Arial 10pt
26. Securing search is difficult 26
27. Why is it difficult? • URL path is not a filter • Some operations are working on entire index • It’s a search and analytics engine, which makes frequency analysis attack so much easier. 27
28. 28 Is there a solution?
29. Security perimeter Solution 1: Implement security in the business logic layer 29 node1 node2 Front - end app Business Logic + Security
30. Solution 2: Move security to the transport level 30
31. 31 Securing Elasticsearch with X-Pack
32. Common Security Components 32 Authentication Authorization Auditing • Native authentication • LDAP • Active Directory • PKI • Custom integrations • Role-based access control • File output • Index output
33. Securing API Calls • Secure at the lowest level 33 APIs can perform multiple operations Rest Request Client Call Action 1+ 1 Action Client Call 0+ 0+ …
34. Secured Communication • Encryption with TLS • Message Signing • IP Filtering • Transport Filters 34 node1 node2 transport client HTTP client
35. customer_care: indices: '*': privileges: read query: term: department_id: 12 Document Level Security Managed using Roles 35
36. Field Level Security Implemented at the Lucene Level 36 customer_care: indices: '*': privileges: read fields: - issue_id - description - customer_handle - customer_email - customer_address - customer_phone
37. ‹#› Questions? See us at AMA https://discuss.elastic.co