Elasticsearch

Transcript

1. Buscas poderosas em BILHÕES de documentos? Seu sistema pode prover isso de forma escalável e resiliente com o Elasticsearch Matheus de Faria Moraes
2. Agenda ▰ What is Elasticsearch; ▰ Use Cases; ▰ Basic Concepts; ▰ Document and Index; ▰ Cluster and Nodes; ▰ Primary Shards and Replica Shards; ▰ Near Real Time (NRT); ▰ Demo.
3. whoami I am Matheus Moraes Developer and Speaker @Sensedia Java, NoSQL and Microservices enthusiast
4. What is Elasticsearch?
5. What is Elasticsearch? ▰ Full-text search and analytics engine; ▰ Highly scalable; ▰ Open-source; ▰ Store, search, and analyze big volumes of data in near real time; ▰ REST APIs; ▰ Good documentation; ▰ Apache Lucene.
6. Use Cases
7. 2 billion documents 8 million code repositories 4 million active users ~ 300 search / minute
9. Basic Concepts
10. Document and Index curl -X PUT localhost:9200/cities/_doc/1 \ -H 'Content-Type: application/json' \ -d '{ "city": "Tanabi", "state": "SP", "country": "BR", "population": 25000 }'
11. CLUSTER NODE 1 ★ NODE 2 NODE 3 Cluster and Nodes
12. CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 P1 Primary Shards
13. Primary Shard Benefits ▰ Elasticity ▰ Horizontal Scaling
14. CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0 R1 R1 R0 P1 Replica Shards 2/2
15. CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0 R1 R1 R0 P1 A A A B B B Cluster, Nodes and Shards 2/2
16. CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0 R1 R1 R0 P1 A A A B B B Cluster, Nodes and Shards 2/2
17. Replica Shard Benefits ▰ H.A. ▰ Resilience ▰ Search Throughput
18. Topologies ▰ Default 7.0 1 / 1 ▰ Old Default 5 / 1 ▰ Search performance 1 / 10 ▰ Index performance 20 / 1
19. Index creation with shards curl -X PUT localhost:9200/cities \ -H 'Content-Type: application/json' \ -d '{ "settings": { "number_of_shards": 2, "number_of_replicas": 1 } }'
20. Searchable and Persistent Documents
21. curl -X PUT localhost:9200/cities/_doc/1 \ -H 'Content-Type: application/json' \ -d '{ "city": "Tanabi", "state": "SP", "country": "BR", "population": 25000 }' && \ curl -X GET localhost:9200/cities/_search?pretty&q=name:Tanabi N R T
22. Search by segment (Lucene) Searchable Commit Point
23. Lucene commits are expensive ▰ fsync ▰ Disk Searchable Commit Point
24. In-memory buffer and Translog
25. 1. Documents are indexed In-memory buffer Searchable Commit Point { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } Translog
26. 2. Refresh In-memory buffer Searchable Commit Point { } { } { } { } { } { } { } { } Translog
27. 3. The translog keeps accumulating documents In-memory buffer Searchable Commit Point { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } Translog { } { } { } { } { } { } { } { } { }
28. 4. Flush (Lucene commit) In-memory buffer Searchable Commit Point Translog
29. Big picture CLUSTER NODE 1 ★ P0 R1 NODE 3 R0 R1 NODE 2 P1 R0 Searchable Commit Point { }{ }{ } In-memory buffer Translog { }{ }{ }{ }{ }{ }
30. Refresh interval curl -X PUT localhost:9200/cities/_settings \ -H 'Content-Type: application/json' \ -d '{ "index" : { "refresh_interval" : "3s" } }'
31. ?refresh (Index, Update, Delete, and Bulk) ▰ Empty or true ▰ wait_for ▰ false (default) POST cities/_refresh Refresh
32. Demo matheusfm/elasticsearch-demo
33. Thank you! matheusfm matheusfm mfariam