Elasticsearch Meetup

Transcript

1. Imteyaz Ahmad @ahmad_imt
2.  Twitter https://twitter.com/ahmad_imt  LinkedIn https://in.linkedin.com/in/imte yaz-ahmad-09b66b28
3.  Overview  Index API  Document API  Search API  Analysis  Analyzers  Tokenizer Agenda
4.  Open source search and analytics engine  Built on top on Apache Lucene What is it?? Near Real Time Distributed { “name”: “Imteyaz”, “eyes” : “Black” } Document Oriented
5. …a bit of history 2004 2015 2010
6. Use case
7.  Index  Type  Document  Fields  Shards and Replicas  Node  Cluster Fundamentals
8. Fundamentals Cluster Node 1 Node 2 Node 3 elk meetup Indices Shards Documents { “id” : “123” } { “id” : “456” }
9. Database Analogy Relational DB Databases Tables Row Columns Elasticsearch Indices Types Document Fields
10.  Create Index  Delete Index  Get Index  Indices Exists  Open / Close Index API Indices API
11. Indices API  Creating an index PUT http://<server:port>/<index_name>  Example  Terminal: curl -XPUT 'http://localhost:9200/meetup/‘ –d ‘{ "settings": { "number_of_shards": 5, "number_of_replicas": 1 }’  Sense UI: PUT /meetup { "settings": { "number_of_shards": 5, "number_of_replicas": 1 }
12. Indices API  Deleting an index DELETE http://<server:port>/<index_name>  Example  Terminal: curl -XDELETE 'http://localhost:9200/meetup/‘  Sense UI: DELETE /meetup
13. Indices API  Closing an index POST http://<server:port>/<index_name>/_close  Example  Terminal: curl –XPOST 'http://localhost:9200/meetup/_close’  Sense UI: POST /meetup/_close
14. Indices API  Indices Exists HEAD http://<server:port>/<index_name>/ HEAD http://<server:port>/<index_name>/<type_name>  Example  Terminal: curl -i –XHEAD 'http://localhost:9200/meetup/‘ curl -i –XHEAD 'http://localhost:9200/meetup/imti/'  Sense UI: HEAD /meetup/ HEAD /meetup/imti/
15.  Single document APIs  Index API  Get API  Delete API  Update API  Multi-document APIs  Multi Get API  Bulk API Document API { “name” : “meetup”, “post_date”: “2016-01-14” }
16.  Index API  Create/put a document of specific type curl -XPUT 'http://localhost:9200/meetup/elk/1’ –d ‘{ “name” : “logstash”, “date” : “2016-01-1T14:12:12”, “speakers” : [“Rohan”] }’ Document API
17.  Get API  Getting a typed document based on its id curl -XGET 'http://localhost:9200/meetup/elk/1’  Checking if a document exists curl –i –XHEAD ‘http://localhost:9200/meetup/elk/1’ Document API
18.  Delete API  Delete a typed document based on id curl -XDELETE 'http://localhost:9200/meetup/elk/1’ Document API
19.  Update API  Update with partial doc curl -XPOST 'http://localhost:9200/meetup/elk/1/_update’ –d ‘{ “doc”: { “name” : “elasticsearch” } }’  Update with script (please enable inline scripting before running this) curl -XPOST 'http://localhost:9200/meetup/elk/1/_update’ –d ‘{ "script" : { "inline": "ctx._source.speakers += speaker", "params" : { "speaker" : “Imteyaz“ } } }’ Document API
20. Analysis  Tokenize the block of text  Normalize the terms  Inverted Index
21. Inverted Index  A list of all unique words in a document  A list of all document containing a particular word  Example:  Elasticsearch meetup was organized on 14 February.  On 14 February we have arranged an elasticsearch meetup.
22. Inverted Index Term Doc_1 Doc_2 Elasticsearch X - meetup X X was X organized X - on X - 14 X X February X X On - X we - X had - X arranged - X an - X Elasticsearch - X Example: 1. Elasticsearch meetup was organized on 14 February. 2. On 14 February we had arranged an elasticsearch meetup.
23. Inverted Index Term Doc_1 Doc_2 Elasticsearch X - meetup X X was X organized X - on X - 14 X X February X X On - X we - X had - X arranged - X an - X elasticsearch - X Term Doc_1 Doc_2 14 X X an - X arranged - X elasticsearch X X february X X had - X meetup X X on X X organized X was X we - X
24. Inverted Index Term Doc_1 Doc_2 Elasticsearch X - meetup X X was X organized X - on X - 14 X X February X X On - X we - X had - X arranged - X an - X elasticsearch - X Term Doc_1 Doc_2 14 X X an - X arranged - X elasticsearch X X february X X had - X meetup X X on X X organized X was X we - X
25. Analyzer  A wrapper that combine followings:  Character filters  Tokenizer  Token filters  Built-in Analyzers  Standard analyzer  Simple analyzer  Whitespace analyzer  Language analyzer Standard analyzer
26. Analyzer  Examples "Set the shape to semi-transparent by calling set_trans(5)" Standard analyzer Analyzer Property Resluts Standard Default, split according to the word boundary set, the, shape, to, semi, transparent, by, calling, set_trans, 5 Simple Splits the text on anything that isn’t a letter set, the, shape, to, semi, transparent, by, calling, set, trans Whitespace Splits the texts on whitespace and does not lowercase Set, the, shape, to, semi-transparent, by, calling, set_trans(5) Language Take the peculiarity and stop words of a language set, shape, semi, transpar, call, set_tran, 5# #- this is the example of english analyzer
27. Standard analyzer
28. References 1. https://www.elastic.co/guide/en/elasticsearch/referenc e/current/index.html 2. https://www.elastic.co/guide/en/elasticsearch/guide/c urrent/index.html
29.  Who am I: http://www.onechallenge.org/2014/01/01/dianes-journey-tcks-who- am-i/  Distributed System: http://www.kgionline.com/distributed-computing.jpg  The time is now: http://www.mimer.com/Portals/0/Bilder/realtime_img.gif  http://www.stephen.fm/images/elastic-architecture-example.jpeg  https://s3.amazonaws.com/lowres.cartoonstock.com/hobbies-leisure- wordplay-word_play-word_game-word_game-wordgame-rnin1300_low.jpg  http://powerupsuccess.com/wp-content/uploads/2015/07/Home-Market- Analysis.jpg  http://cdn2.hubspot.net/hub/120635/file-2113812581- jpg/blog_images/Questions.jpg?t=1454619342665 Copyright and Photo Credits