Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Elasticsearch
Search
Matheus Moraes
June 14, 2019
Technology
0
110
Elasticsearch
Matheus Moraes
June 14, 2019
Tweet
Share
More Decks by Matheus Moraes
See All by Matheus Moraes
Nova Era de Políticas no Kubernetes: Explorando Abordagens Nativas de Policy Enforcement
matheusfm
0
14
Kubernetes Operator with Go
matheusfm
0
80
Observability before and after Service Mesh
matheusfm
0
66
Incorpore a Supernanny para seus microservices e entenda o que andam aprontando em produção
matheusfm
0
220
Other Decks in Technology
See All in Technology
スタートアップが AWS FTR を取得するべき理由
izzii
0
170
SONiCスイッチを商用サービスに入れてみた(三井情報株式会社)
sonic
0
190
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
5
39k
データベース06: SQL (3/3) 副問い合わせ
trycycle
0
120
パソコン音痴な私がモバイル開発界隈でぬくぬく成長している理由
mitchan
0
200
RAGのサービスをリリースして1年が経ちました
segavvy
4
1.5k
左手は添えるだけ!?AWS Well-Architected Frameworkが教えてくれる大事なデータの守り方
ohtk79
0
360
現場訪問で変わる毎日の開発
andpad
1
210
意義から考えるObservability入門 #srenext
stefafafan
2
820
iOS'te Tipografi
seyfoyun
0
120
From here to resilience - a travel guide
ufried
1
230
Oracle Database 23ai Overview
oracle4engineer
PRO
0
310
Featured
See All Featured
Product Roadmaps are Hard
iamctodd
45
9.8k
How to Ace a Technical Interview
jacobian
273
22k
Bootstrapping a Software Product
garrettdimon
PRO
302
110k
Become a Pro
speakerdeck
PRO
13
4.6k
Happy Clients
brianwarren
92
6.4k
For a Future-Friendly Web
brad_frost
172
9k
In The Pink: A Labor of Love
frogandcode
138
21k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
226
51k
Keith and Marios Guide to Fast Websites
keithpitt
408
22k
The World Runs on Bad Software
bkeepers
PRO
61
6.8k
Fontdeck: Realign not Redesign
paulrobertlloyd
77
5k
4 Signs Your Business is Dying
shpigford
176
21k
Transcript
Buscas poderosas em BILHÕES de documentos? Seu sistema pode prover
isso de forma escalável e resiliente com o Elasticsearch Matheus de Faria Moraes
Agenda ▰ What is Elasticsearch; ▰ Use Cases; ▰ Basic
Concepts; ▰ Document and Index; ▰ Cluster and Nodes; ▰ Primary Shards and Replica Shards; ▰ Near Real Time (NRT); ▰ Demo.
whoami I am Matheus Moraes Developer and Speaker @Sensedia Java,
NoSQL and Microservices enthusiast
What is Elasticsearch?
What is Elasticsearch? ▰ Full-text search and analytics engine; ▰
Highly scalable; ▰ Open-source; ▰ Store, search, and analyze big volumes of data in near real time; ▰ REST APIs; ▰ Good documentation; ▰ Apache Lucene.
Use Cases
2 billion documents 8 million code repositories 4 million active
users ~ 300 search / minute
None
Basic Concepts
Document and Index curl -X PUT localhost:9200/cities/_doc/1 \ -H 'Content-Type:
application/json' \ -d '{ "city": "Tanabi", "state": "SP", "country": "BR", "population": 25000 }'
CLUSTER NODE 1 ★ NODE 2 NODE 3 Cluster and
Nodes
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 P1
Primary Shards
Primary Shard Benefits ▰ Elasticity ▰ Horizontal Scaling
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 Replica Shards 2/2
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 A A A B B B Cluster, Nodes and Shards 2/2
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 A A A B B B Cluster, Nodes and Shards 2/2
Replica Shard Benefits ▰ H.A. ▰ Resilience ▰ Search Throughput
Topologies ▰ Default 7.0 1 / 1 ▰ Old Default
5 / 1 ▰ Search performance 1 / 10 ▰ Index performance 20 / 1
Index creation with shards curl -X PUT localhost:9200/cities \ -H
'Content-Type: application/json' \ -d '{ "settings": { "number_of_shards": 2, "number_of_replicas": 1 } }'
Searchable and Persistent Documents
curl -X PUT localhost:9200/cities/_doc/1 \ -H 'Content-Type: application/json' \ -d
'{ "city": "Tanabi", "state": "SP", "country": "BR", "population": 25000 }' && \ curl -X GET localhost:9200/cities/_search?pretty&q=name:Tanabi N R T
Search by segment (Lucene) Searchable Commit Point
Lucene commits are expensive ▰ fsync ▰ Disk Searchable Commit
Point
In-memory buffer and Translog
1. Documents are indexed In-memory buffer Searchable Commit Point {
} { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } Translog
2. Refresh In-memory buffer Searchable Commit Point { } {
} { } { } { } { } { } { } Translog
3. The translog keeps accumulating documents In-memory buffer Searchable Commit
Point { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } Translog { } { } { } { } { } { } { } { } { }
4. Flush (Lucene commit) In-memory buffer Searchable Commit Point Translog
Big picture CLUSTER NODE 1 ★ P0 R1 NODE 3
R0 R1 NODE 2 P1 R0 Searchable Commit Point { }{ }{ } In-memory buffer Translog { }{ }{ }{ }{ }{ }
Refresh interval curl -X PUT localhost:9200/cities/_settings \ -H 'Content-Type: application/json'
\ -d '{ "index" : { "refresh_interval" : "3s" } }'
?refresh (Index, Update, Delete, and Bulk) ▰ Empty or true
▰ wait_for ▰ false (default) POST cities/_refresh Refresh
Demo matheusfm/elasticsearch-demo
Thank you! matheusfm matheusfm mfariam