Elasticsearch Tony Messias MaceioDevMeetup #5 

who am I? ➔ Tony Messias ~ @tony0x01 ➔ Building web stuff since ~2010 

No content

search 

what is “search”? “A search is the organized pursuit of information (...) that you want to find, but you have no idea where it is.” (NASA). 

how do we add search to our apps? ➔ SQL queries (not just LIKE queries); ➔ Elasticsearch; ➔ Lucene; ➔ Solr; ➔ … 

how do we add search to our apps? ➔ SQL queries (not just LIKE queries); ➔ Elasticsearch; ➔ Lucene; ➔ Solr; ➔ … 

who uses it? 

No content

what is Elasticsearch? ➔ database server; ➔ document based; ➔ based on Apache Lucene; ➔ schema-free*; 

why Elasticsearch? ➔ speaks HTTP/JSON; ➔ easy to setup; ➔ data replication; ➔ easily scalable; 

what can I use it for? 

No content

but it can do more than that… 

some analogy * Relational ES database index table type row document column field schema mapping index everything SQL QueryDSL 

REST(ish) $ curl -XPOST http://es-server.com/index/type/id -d '{}' 

REST(ish) $ curl -XPOST http://es-server.com/index/type/id -d '{}' 

REST(ish) $ curl -XPOST http://es-server.com/index/type/id -d '{}' 

REST(ish) $ curl -XPOST http://es-server.com/index/type/id -d '{}' 

REST(ish) $ curl -XPOST http://es-server.com/index/type/id -d '{}' 

enough with the talking… let’s do some searches! 

but first, let’s talk about CRUD 

No content

indexing 

{ "email": "[email protected]", "first_name": "John", "last_name": "Smith", "info": { "bio": "Eco-warrior and defender of the weak", "age": 25, "interests": [ "dolphins", "whales" ] }, "join_date": "2014/05/01" } 

$ curl -XPOST localhost:9200/my-app/users -d ‘{ "email": "[email protected]", "first_name": "John", "last_name": "Smith", "info": { "bio": "Eco-warrior and defender of the weak", "age": 25, "interests": [ "dolphins", "whales" ] }, "join_date": "2014/05/01" }’ 

{ "_index": "my-app", "_type": "users", "_id": "AUxOBeJbue-jeR3jNcHj", "_version": 1, "created": true } 

$ curl -XPUT localhost:9200/my-app/users/AUxOBeJbue-jeR3jNcHj -d ‘{ "email": "[email protected]", "first_name": "John" }’ 

{ "_index": "my-app", "_type": "users", "_id": "AUxOBeJbue-jeR3jNcHj", "_version": 2, "created": false } 

$ curl -XGET localhost:9200/my-app/users/AUxOBeJbue-jeR3jNcHj { "_index": "my-app", "_type": "users", "_id": "AUxOJe-4ue-jeR3jNcHo", "_version": 2, "found": true, "_source": { "email": "[email protected]", "first_name": "John" } } 

$ curl -XDELETE localhost:9200/my-app/users/AUxOBeJbue-jeR3jNcHj { "found": true, "_index": "my-app", "_type": "users", "_id": "AUxOBeJbue-jeR3jNcHj", "_version": 3 } 

searching 

$ curl -XPOST localhost:9200/my-app/users/_search -d ‘{ "query": { "match": { "first_name": "John" } } }’ 

{ ..., "hits": { "total": 4, "max_score": 0.30685282, "hits": [ { "_index": "my-app", "_type": "users", "_id": "AUxOJWlCue-jeR3jNcHn", "_score": 0.30685282, "_source": { "email": "[email protected]", "first_name": "John", "last_name": "Doe", "info": { "bio": "Like rock music", "age": 35, "interests": [ "dolphins", "whales" ] }, "join_date": "2014/05/01" } },... 

$ curl -XPOST localhost:9200/my-app/users/_search -d ‘{ "query": { "filtered": { "query": { "match": { "first_name": "John" } }, "filter": { "range": { "info.age": { "gt": 34 } } } } } }’ 

{ "took": 6, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 1, "max_score": 0.30685282, "hits": [ { "_index": "my-app", "_type": "users", "_id": "AUxOJWlCue-jeR3jNcHn", "_score": 0.30685282, "_source": { "email": "[email protected]", "first_name": "John", "last_name": "Doe", "info": { "bio": "Like rock music", "age": 35, "interests": ["dolphins","whales"] }, "join_date": "2014/05/01" } } ] } } 

queries vs. filters 

queries vs. filters 

full-text search 

$ curl -XPOST localhost:9200/my-app/users/_search -d ‘{ "query": { "match": { "info.bio": "rock climbing" } } }’ 

{ "took": 9, "timed_out": false, "_shards": {...}, "hits": { "total": 2, "max_score": 0.2169777, "hits": [ { "_index": "my-app", "_type": "users", "_id": "AUxOJe-4ue-jeR3jNcHo", "_score": 0.2169777, "_source": { "email": "[email protected]", "first_name": "John", "last_name": "Kennedy", "info": { "bio": "I love rock climbing!", "age": 29, "interests": ["dolphins","whales"] }, "join_date": "2014/05/01" } }, { "_index": "my-app", "_type": "users", "_id": "AUxOJWlCue-jeR3jNcHn", "_score": 0.02250402, "_source": { "email": "[email protected]", "first_name": "John", "last_name": "Doe", "info": { "bio": "Like rock music", "age": 35, "interests": ["dolphins","whales"] }, "join_date": "2014/05/01" } } ] } } 

$ curl -XPOST localhost:9200/my-app/users/_search -d ‘{ "query": { "match_phrase": { "info.bio": "rock climbing" } } }’ 

{…, "hits": { "total": 1, "max_score": 0.30685282, "hits": [ { "_index": "my-app", "_type": "users", "_id": "AUxOJe-4ue-jeR3jNcHo", "_score": 0.30685282, "_source": { "email": "[email protected]", "first_name": "John", "last_name": "Kennedy", "info": { "bio": "I love rock climbing!", "age": 29, "interests": ["dolphins","whales"] }, "join_date": "2014/05/01" } } ] } } 

$ curl -XPOST localhost:9200/my-app/users/_search -d ‘{ "query": { "match_phrase": { "info.bio": "rock climbing" } }, "highlight": { "fields": { "info.bio": {} } } }’ 

... { "_index": "my-app", "_type": "users", "_id": "AUxOJe-4ue-jeR3jNcHo", "_score": 0.30685282, "_source": { "email": "[email protected]", "first_name": "John", "last_name": "Kennedy", "info": { "bio": "I love rock climbing!", "age": 29, "interests": ["dolphins","whales"] }, "join_date": "2014/05/01" }, "highlight": { "info.bio": ["I love rock climbing!"] } } ] } } 

is this it? 

No content

No content

analytics 

aggregations 

buckets metrics 

$ curl -XPOST localhost:9200/my-app/users/_search?search_type=count -d'{ "aggs": { "all_interests": { "terms": { "field": "info.interests" } } } }' 

{ ..., "hits": {"total": 4,"max_score": 0,"hits": []}, "aggregations": { "all_interests": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "whales", "doc_count": 4 }, { "key": "dolphins", "doc_count": 3 } ] } }} 

$ curl -XPOST localhost:9200/my-app/users/_search?search_type=count -d'{ "aggs": { "all_interests": { "terms": { "field": "info.interests" } } } }' 

$ curl -XPOST localhost:9200/my-app/users/_search?search_type=count -d'{ "aggs": { "all_interests": { "terms": { "field": "info.interests" }, "aggs": { "avg_age": { "avg": { "field": "info.age" } } } } } }' 

{ …, "aggregations": { "all_interests": { …, "buckets": [ { "key": "whales", "doc_count": 4, "avg_age": { "value": 31.5 } }, { "key": "dolphins", "doc_count": 3, "avg_age": { "value": 32.333333333333336 } } ] }}} 

$ curl -XPOST localhost:9200/my-app/users/_search?search_type=count -d'{ "query": { "match_phrase": { "info.bio": "rock climbing" } }, "aggs": { "all_interests": { "terms": { "field": "info.interests" }, "aggs": { "avg_age": { "avg": { "field": "info.age" } } } } }}' 

{ …, "aggregations": { "all_interests": { …, "buckets": [ { "key": "whales", "doc_count": 1, "avg_age": { "value": 29 } } ] }}} 

No content

questions? 

Resources ➔ http://www.elastic. co/guide/en/elasticsearch/guide/current/index. html ➔ https://leanpub.com/elasticsearch-quick-start ➔ https://www.elastic.co/use-cases ➔ https://speakerdeck.com/asm89/elasticsearch 

Resources ➔ https://speakerdeck. com/johnbeynon/elasticsearch-101 ➔ http://blog.madewithlove.be/post/integrating- elasticsearch-with-your-laravel-app/ ➔ http://blog.madewithlove.be/post/elasticsearch- aggregations/