ELASTICSEARCH - FIRST STEPS

XX Betabeers at Granada

Transcript

2. ElasticSearch – First Steps: About me • Jesús María Villar Vázquez • 10+ years of experience. • Backend Engineer at BestSecret. • Java enthusiastic. • Follow me in twitter! @geeksusma
3. ElasticSearch – First Steps Summary: • What is ES? • Installing and running ElasticSearch • Indexing. Creating, updating and deleting data. • Some query examples. • Usage in BestSecret • What else? • Work in BestSecret
4. ElasticSearch – First Steps: What is ES? • Is a distributed, RESTful search and analytic engine based on Lucene. • Uses standard RESTful APIs and JSON. • Built in Java, but with support to use clients in Java, Python, .net and Groovy. • Has Hadoop and Spark connectors. • Based in index, can be used as a NoSql solution, but without distribuited transactions. • Used by BestSecret, Wikimedia, Foursquare, GitHub, CERN and many more…
5. ElasticSearch – First Steps: Installing and running ElasticSearch • Prerequisites à Ensure you have Java 8 installed and running in your WorkStation • Go to https://www.elastic.co/downloads/elasticsearch, download the latest distribution, place in a location with permissions of writing, reading and execution and unzip it. • Run bin/elasticsearch (or bin\elasticsearch.bat on Windows) • http://localhost:9200/
6. ElasticSearch – First Steps: Installing and running ElasticSearch • And that’s all! Easy, isn’t?
7. ElasticSearch – First Steps: Indexing • ElasticSearch works fetching the data through indexes. • So that, first thing to start is populating some indexes in our ES server. • As easy as creating/updating the indexes through a PUT request. Take care if one index exists, will be overwritten. • Syntax: http://localhost:9200/<index>/<type>/[<documentId>] • <index>: Required. • <type>: Required. • <documentId>: Optional. Can be generated by ES if don’t provide any.
8. ElasticSearch – First Steps: Indexing • We can put requests in the server using curl or ”sense” • Sense is a browser add-on for Chrome, you can do requests to our local server in easyly.
9. ElasticSearch – First Steps: Indexing – Create Data
10. ElasticSearch – First Steps: Indexing – Create Data
11. ElasticSearch – First Steps: Indexing – Update Data
12. ElasticSearch – First Steps: Indexing – Delete Data
13. ElasticSearch – QuickStart: Indexing – Create some movies curl -XPUT "http://localhost:9200/movies/movie/1" -d' { "title": "The Godfather", "director": "Francis Ford Coppola", "year": 1972, "genres": ["Crime", "Drama"] }' curl -XPUT "http://localhost:9200/movies/movie/2" -d' { "title": "Lawrence of Arabia", "director": "David Lean", "year": 1962, "genres": ["Adventure", "Biography", "Drama"] } ' curl -XPUT "http://localhost:9200/movies/movie/3" -d' { "title": "To Kill a Mockingbird", "director": "Robert Mulligan", "year": 1962, "genres": ["Crime", "Drama", "Mystery"] } ' curl -XPUT "http://localhost:9200/movies/movie/4" -d' { "title": "Apocalypse Now", "director": "Francis Ford Coppola", "year": 1979, "genres": ["Drama", "War"] } ' curl -XPUT "http://localhost:9200/movies/movie/5" -d' { "title": "Kill Bill: Vol. 1", "director": "Quentin Tarantino", "year": 2003, "genres": ["Action", "Crime", "Thriller"] } ' curl -XPUT "http://localhost:9200/movies/movie/6" -d' { "title": "The Assassination of Jesse James by the Coward Robert Ford", "director": "Andrew Dominik", "year": 2007, "genres": ["Biography", "Crime", "Drama"] }'
14. ElasticSearch – First Steps: Some query examples • The _search endpoint. • optional with an index and type. • In order to search for our movies we can make POST requests to either of the following URLs: • http://localhost:9200/_search - Search across all indexes and all types. • http://localhost:9200/movies/_search - Search across all types in the movies index. • http://localhost:9200/movies/movie/_search - Search explicitly for documents of type movie within the movies index. • Query body: used to execute more accurated queries. Is just a Json based in the ”ES query DSL” • { "query": { //Query DSL here } }
15. ElasticSearch – First Steps: Some query examples • Example #1 - Basic free text search. • "query": { "query_string": { "query": "Kill" } } • Example #2 - Basic search specifying fields. • "query": { "query_string": { "query": "ford", "fields": ["title"] } } • Example #3 - Filtering. • { "query": { "bool": { "must": { "match": { "genres": "drama" } }, "filter": { "term": { "year": "1962" } } } }} • { "query": { "bool": { "must": { "match_all":{ } }, "filter": { "term": { "year": "1962" } } } }} • { "query": { "constant_score": { "filter": { "term": { "year": 1962 } } } }}
16. ElasticSearch – First Steps: Some query examples • Example #4 - Mapping. • { "query": { "constant_score": { "filter": { "term": { "director": "Francis Ford Coppola" } } } } } à Does not retrieve data! What is happening then? • "query": { "constant_score": { "filter": { "term": { "director": "francis" } } } } à Is retrieving data? • Why? • When we index a document with ES, it does two things: Stores the original data and index each JSON property into one or more fields in a Lucene index. If we don’t provide a custom mapping, the default mapping will be applied depending of the type (string, number…)
17. ElasticSearch – First Steps: Some query examples • Example #4 - Mapping. • Because we didn’t provide a custom mapping to the director field, the index in the director fields does not contain “Francis Ford Coppola”. Instead it’s something more like [“francis”, “ford”, “coppola”] • If we want to filter by the exact value of a field, we have to modify how it’s mapped. Easy doing the next PUT request to the _mapping endpoint (PUT movies/movie/_mapping). • "movie": { "properties": { "director": { "type": "string", "index": "not_analyzed" } } }
18. ElasticSearch – First Steps: Some query examples • Example #4 - Mapping. • Sometimes we can’t modify already existing indexes. So that we have to go to another solution, in this case we can upgrade our field to a “multifield”. That’s means is that we’ll map the field multiple times for indexing. • This solution always will be better than the most common workaround (creating a new index) • PUT movies/movie/_mapping{ "movie": { "properties": { "director": { "type": "string", "fields": { "original": {"type" : "string", "index" : "not_analyzed"} } } } }} • After a reindexing, executing the next query will shown the data as we expected J • { "query": { "constant_score": { "filter": { "term": { "director.original": "Francis Ford Coppola" } } } } }
19. ElasticSearch – First Steps: Usage in BestSecret • To improve the search mechanism and autocomplete for the free text feature.
20. ElasticSearch – First Steps: Usage in BestSecret • To maintain the Category Tree and the Search Facets as fast as possible. Also to enable dynamic loading.
21. ElasticSearch – First Steps: Usage in BestSecret • To have the feature “You might also like…” and collect “Recently viewed items”
22. ElasticSearch – First Steps: Usage in BestSecret • To provide exclusive mobile features: Wishlist.
23. ElasticSearch – First Steps: What else? • http://joelabrahamsson.com/ • https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html • https://www.elastic.co/community
24. ElasticSearch – First Steps: What else? • http://joelabrahamsson.com/ • https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html • https://www.elastic.co/community • … or you could join into https://www.bestsecret.com/home.htm J
25. ElasticSearch – First Steps: Work in BestSecret • What is BestSecret? (http://www.bestsecret.com) • Founded in 2007. • Fashion company focused in e-commerce. 15M items, 35K ships per day to all Europe • Headquarter in Munich. • +350M€ Yearly Gross Revenue. Goal in 2020: 700M€. • Local Shops in DE, AT, CH, SE, FR & UK, soon NL, DK & NO. • BestSecret Granada. • Technical Office opened in Mid 2015. • 23 Engineers full time dedication in Agile teams. Soon 45. • Main Technologies: Java, Mobile and .NET. • https://www.youtube.com/watch?v=ZHVdrCXnJZQ
26. ElasticSearch – First Steps: Work in BestSecret • What are we looking for? • Full Stacks developers/engineers. • Fluent English. • 3+ years of proven experience with Java. • Knowledge about testing (Junit, Itest, End-to-end) automation and continous integration.
27. ElasticSearch – First Steps: Work in BestSecret • What we offer? • Permanent role inmediately. • Salary over sector average. • Training. Conferences, travels and many more. • Brand new equipment, you’ll never be out of fashion. • StartUp atmosphere • More info: • https://tinyurl.com/linkedin-job-position • https://tinyurl.com/linkedin-bestsecret-page
28. ElasticSearch – First Steps: Questions? • Questions?