Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Elasticsearch
Search
Matheus Moraes
June 14, 2019
Technology
0
110
Elasticsearch
Matheus Moraes
June 14, 2019
Tweet
Share
More Decks by Matheus Moraes
See All by Matheus Moraes
Nova Era de Políticas no Kubernetes: Explorando Abordagens Nativas de Policy Enforcement
matheusfm
0
16
Kubernetes Operator with Go
matheusfm
0
89
Observability before and after Service Mesh
matheusfm
0
76
Incorpore a Supernanny para seus microservices e entenda o que andam aprontando em produção
matheusfm
0
240
Other Decks in Technology
See All in Technology
Scaling Technical Excellence at 104: Evolution in AWS and Developer Empowerment
scotthsieh825
1
160
Flutter研修【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
160
Luupの開発組織におけるインシデントマネジメントの変遷 ver.RoadtoSRENEXT2024
grimoh
1
270
AWSサービスメニュー開発をしていてAWSを好きだ!と感じた瞬間
toru_kubota
0
130
What if...? 처음부터 다시 LLM 어플리케이션을 개발한다면
huffon
0
1k
GoとアクターモデルでES+CQRSを実践! / proto_actor_es_cqrs
ytake
1
160
ギークの理想が7つ集まるエムスリーで夢を叶えよう - エムスリー株式会社
m3_engineering
1
260
AWSで”最小権限の原則”を実現するための考え方 /20240722-ssmjp-aws-least-privilege
opelab
10
4.4k
データベース研修 分析向けSQL入門【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
110
dxd2024-生成AIに振り回された3か月間の成功と失敗/dxd2024-link-and-motivation
lmi
2
260
AWSでRAGを作る法方
sonoda_mj
1
140
Git 研修 Basic【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
310
Featured
See All Featured
Building Adaptive Systems
keathley
34
2k
How GitHub Uses GitHub to Build GitHub
holman
471
290k
Fontdeck: Realign not Redesign
paulrobertlloyd
79
5.1k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
13
430
How STYLIGHT went responsive
nonsquared
93
5k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
18
1.2k
How to Ace a Technical Interview
jacobian
274
23k
Bash Introduction
62gerente
607
210k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
35
6.3k
What's in a price? How to price your products and services
michaelherold
239
11k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
26
1.8k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
129
32k
Transcript
Buscas poderosas em BILHÕES de documentos? Seu sistema pode prover
isso de forma escalável e resiliente com o Elasticsearch Matheus de Faria Moraes
Agenda ▰ What is Elasticsearch; ▰ Use Cases; ▰ Basic
Concepts; ▰ Document and Index; ▰ Cluster and Nodes; ▰ Primary Shards and Replica Shards; ▰ Near Real Time (NRT); ▰ Demo.
whoami I am Matheus Moraes Developer and Speaker @Sensedia Java,
NoSQL and Microservices enthusiast
What is Elasticsearch?
What is Elasticsearch? ▰ Full-text search and analytics engine; ▰
Highly scalable; ▰ Open-source; ▰ Store, search, and analyze big volumes of data in near real time; ▰ REST APIs; ▰ Good documentation; ▰ Apache Lucene.
Use Cases
2 billion documents 8 million code repositories 4 million active
users ~ 300 search / minute
None
Basic Concepts
Document and Index curl -X PUT localhost:9200/cities/_doc/1 \ -H 'Content-Type:
application/json' \ -d '{ "city": "Tanabi", "state": "SP", "country": "BR", "population": 25000 }'
CLUSTER NODE 1 ★ NODE 2 NODE 3 Cluster and
Nodes
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 P1
Primary Shards
Primary Shard Benefits ▰ Elasticity ▰ Horizontal Scaling
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 Replica Shards 2/2
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 A A A B B B Cluster, Nodes and Shards 2/2
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 A A A B B B Cluster, Nodes and Shards 2/2
Replica Shard Benefits ▰ H.A. ▰ Resilience ▰ Search Throughput
Topologies ▰ Default 7.0 1 / 1 ▰ Old Default
5 / 1 ▰ Search performance 1 / 10 ▰ Index performance 20 / 1
Index creation with shards curl -X PUT localhost:9200/cities \ -H
'Content-Type: application/json' \ -d '{ "settings": { "number_of_shards": 2, "number_of_replicas": 1 } }'
Searchable and Persistent Documents
curl -X PUT localhost:9200/cities/_doc/1 \ -H 'Content-Type: application/json' \ -d
'{ "city": "Tanabi", "state": "SP", "country": "BR", "population": 25000 }' && \ curl -X GET localhost:9200/cities/_search?pretty&q=name:Tanabi N R T
Search by segment (Lucene) Searchable Commit Point
Lucene commits are expensive ▰ fsync ▰ Disk Searchable Commit
Point
In-memory buffer and Translog
1. Documents are indexed In-memory buffer Searchable Commit Point {
} { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } Translog
2. Refresh In-memory buffer Searchable Commit Point { } {
} { } { } { } { } { } { } Translog
3. The translog keeps accumulating documents In-memory buffer Searchable Commit
Point { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } Translog { } { } { } { } { } { } { } { } { }
4. Flush (Lucene commit) In-memory buffer Searchable Commit Point Translog
Big picture CLUSTER NODE 1 ★ P0 R1 NODE 3
R0 R1 NODE 2 P1 R0 Searchable Commit Point { }{ }{ } In-memory buffer Translog { }{ }{ }{ }{ }{ }
Refresh interval curl -X PUT localhost:9200/cities/_settings \ -H 'Content-Type: application/json'
\ -d '{ "index" : { "refresh_interval" : "3s" } }'
?refresh (Index, Update, Delete, and Bulk) ▰ Empty or true
▰ wait_for ▰ false (default) POST cities/_refresh Refresh
Demo matheusfm/elasticsearch-demo
Thank you! matheusfm matheusfm mfariam