Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Elasticsearch
Search
Matheus Moraes
June 14, 2019
Technology
0
140
Elasticsearch
Matheus Moraes
June 14, 2019
Tweet
Share
More Decks by Matheus Moraes
See All by Matheus Moraes
Nova Era de Políticas no Kubernetes: Explorando Abordagens Nativas de Policy Enforcement
matheusfm
0
36
Kubernetes Operator with Go
matheusfm
0
140
Observability before and after Service Mesh
matheusfm
0
100
Incorpore a Supernanny para seus microservices e entenda o que andam aprontando em produção
matheusfm
0
310
Other Decks in Technology
See All in Technology
Introduction to Sansan, inc / Sansan Global Development Center, Inc.
sansan33
PRO
0
2.6k
技術職じゃない私がVibe Codingで感じた、AGIが身近になる未来
blueb
0
120
Roo CodeとClaude Code比較してみた
pharma_x_tech
1
270
Snowflake Intelligenceで実現できるノーコードAI活用
takumimukaiyama
1
170
Kotlinで学ぶ 代数的データ型
ysknsid25
5
1k
開発フェーズだけではない AI導入はどのように進めていくべきか / How should we proceed with AI adoption beyond the development stage?
i35_267
2
140
フルカイテン株式会社 エンジニア向け採用資料
fullkaiten
0
7.2k
What's new in OpenShift 4.19
redhatlivestreaming
1
170
脅威をモデリングしてMCPのセキュリティ対策を考えよう
flatt_security
4
1.4k
ハッカー視点で学ぶサイバー攻撃と防御の基本
nomizone
3
1.7k
AWS全冠したので振りかえってみる
tajimon
0
120
Introduction to Sansan for Engineers / エンジニア向け会社紹介
sansan33
PRO
5
38k
Featured
See All Featured
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
48
5.4k
Raft: Consensus for Rubyists
vanstee
139
7k
Side Projects
sachag
454
42k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
Speed Design
sergeychernyshev
30
990
Designing for Performance
lara
609
69k
Building an army of robots
kneath
306
45k
Music & Morning Musume
bryan
46
6.6k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
42
2.4k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
The Art of Programming - Codeland 2020
erikaheidi
54
13k
Transcript
Buscas poderosas em BILHÕES de documentos? Seu sistema pode prover
isso de forma escalável e resiliente com o Elasticsearch Matheus de Faria Moraes
Agenda ▰ What is Elasticsearch; ▰ Use Cases; ▰ Basic
Concepts; ▰ Document and Index; ▰ Cluster and Nodes; ▰ Primary Shards and Replica Shards; ▰ Near Real Time (NRT); ▰ Demo.
whoami I am Matheus Moraes Developer and Speaker @Sensedia Java,
NoSQL and Microservices enthusiast
What is Elasticsearch?
What is Elasticsearch? ▰ Full-text search and analytics engine; ▰
Highly scalable; ▰ Open-source; ▰ Store, search, and analyze big volumes of data in near real time; ▰ REST APIs; ▰ Good documentation; ▰ Apache Lucene.
Use Cases
2 billion documents 8 million code repositories 4 million active
users ~ 300 search / minute
None
Basic Concepts
Document and Index curl -X PUT localhost:9200/cities/_doc/1 \ -H 'Content-Type:
application/json' \ -d '{ "city": "Tanabi", "state": "SP", "country": "BR", "population": 25000 }'
CLUSTER NODE 1 ★ NODE 2 NODE 3 Cluster and
Nodes
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 P1
Primary Shards
Primary Shard Benefits ▰ Elasticity ▰ Horizontal Scaling
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 Replica Shards 2/2
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 A A A B B B Cluster, Nodes and Shards 2/2
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 A A A B B B Cluster, Nodes and Shards 2/2
Replica Shard Benefits ▰ H.A. ▰ Resilience ▰ Search Throughput
Topologies ▰ Default 7.0 1 / 1 ▰ Old Default
5 / 1 ▰ Search performance 1 / 10 ▰ Index performance 20 / 1
Index creation with shards curl -X PUT localhost:9200/cities \ -H
'Content-Type: application/json' \ -d '{ "settings": { "number_of_shards": 2, "number_of_replicas": 1 } }'
Searchable and Persistent Documents
curl -X PUT localhost:9200/cities/_doc/1 \ -H 'Content-Type: application/json' \ -d
'{ "city": "Tanabi", "state": "SP", "country": "BR", "population": 25000 }' && \ curl -X GET localhost:9200/cities/_search?pretty&q=name:Tanabi N R T
Search by segment (Lucene) Searchable Commit Point
Lucene commits are expensive ▰ fsync ▰ Disk Searchable Commit
Point
In-memory buffer and Translog
1. Documents are indexed In-memory buffer Searchable Commit Point {
} { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } Translog
2. Refresh In-memory buffer Searchable Commit Point { } {
} { } { } { } { } { } { } Translog
3. The translog keeps accumulating documents In-memory buffer Searchable Commit
Point { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } Translog { } { } { } { } { } { } { } { } { }
4. Flush (Lucene commit) In-memory buffer Searchable Commit Point Translog
Big picture CLUSTER NODE 1 ★ P0 R1 NODE 3
R0 R1 NODE 2 P1 R0 Searchable Commit Point { }{ }{ } In-memory buffer Translog { }{ }{ }{ }{ }{ }
Refresh interval curl -X PUT localhost:9200/cities/_settings \ -H 'Content-Type: application/json'
\ -d '{ "index" : { "refresh_interval" : "3s" } }'
?refresh (Index, Update, Delete, and Bulk) ▰ Empty or true
▰ wait_for ▰ false (default) POST cities/_refresh Refresh
Demo matheusfm/elasticsearch-demo
Thank you! matheusfm matheusfm mfariam