Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Elasticsearch
Search
Matheus Moraes
June 14, 2019
Technology
0
130
Elasticsearch
Matheus Moraes
June 14, 2019
Tweet
Share
More Decks by Matheus Moraes
See All by Matheus Moraes
Nova Era de Políticas no Kubernetes: Explorando Abordagens Nativas de Policy Enforcement
matheusfm
0
28
Kubernetes Operator with Go
matheusfm
0
110
Observability before and after Service Mesh
matheusfm
0
94
Incorpore a Supernanny para seus microservices e entenda o que andam aprontando em produção
matheusfm
0
290
Other Decks in Technology
See All in Technology
インフラをつくるとはどういうことなのか、 あるいはPlatform Engineeringについて
nwiizo
4
500
サーバーレスアーキテクチャと生成AIの融合 / Serverless Meets Generative AI
_kensh
12
2.7k
日経電子版 x AIエージェントの可能性とAgentic RAGによって提案書生成を行う技術
masahiro_nishimi
1
250
SCSAから学ぶセキュリティ管理
masakamayama
0
140
Kubernetesでメールの大量配信をしている話/k8sjp-20250205
hfukamachi
0
350
パブリッククラウドのプロダクトマネジメントとアーキテクト
tagomoris
4
970
AIをプロダクトに実装するならAPIで分離しよう 〜タクシーアプリ『GO』のアーキテクチャ実例紹介〜
74th
2
140
開発スピードは上がっている…品質はどうする? スピードと品質を両立させるためのプロダクト開発の進め方とは #DevSumi #DevSumiB / Agile And Quality
nihonbuson
0
130
Googleマップ/Earthが一般化した 地図タイルのイマ
mapconcierge4agu
1
180
FastConnect の冗長性
ocise
1
9.5k
テストアーキテクチャ設計で実現する高品質で高スピードな開発の実践 / Test Architecture Design in Practice
ropqa
3
470
ビジネスと現場活動をつなぐソフトウェアエンジニアリング~とあるスタートアッププロダクトの成長記録より~
mizunori
0
130
Featured
See All Featured
What’s in a name? Adding method to the madness
productmarketing
PRO
22
3.3k
Bootstrapping a Software Product
garrettdimon
PRO
305
110k
Docker and Python
trallard
44
3.2k
VelocityConf: Rendering Performance Case Studies
addyosmani
328
24k
Music & Morning Musume
bryan
46
6.3k
Typedesign – Prime Four
hannesfritz
40
2.5k
A Tale of Four Properties
chriscoyier
158
23k
BBQ
matthewcrist
86
9.4k
Gamification - CAS2011
davidbonilla
80
5.1k
The Art of Programming - Codeland 2020
erikaheidi
53
13k
Rebuilding a faster, lazier Slack
samanthasiow
79
8.8k
Side Projects
sachag
452
42k
Transcript
Buscas poderosas em BILHÕES de documentos? Seu sistema pode prover
isso de forma escalável e resiliente com o Elasticsearch Matheus de Faria Moraes
Agenda ▰ What is Elasticsearch; ▰ Use Cases; ▰ Basic
Concepts; ▰ Document and Index; ▰ Cluster and Nodes; ▰ Primary Shards and Replica Shards; ▰ Near Real Time (NRT); ▰ Demo.
whoami I am Matheus Moraes Developer and Speaker @Sensedia Java,
NoSQL and Microservices enthusiast
What is Elasticsearch?
What is Elasticsearch? ▰ Full-text search and analytics engine; ▰
Highly scalable; ▰ Open-source; ▰ Store, search, and analyze big volumes of data in near real time; ▰ REST APIs; ▰ Good documentation; ▰ Apache Lucene.
Use Cases
2 billion documents 8 million code repositories 4 million active
users ~ 300 search / minute
None
Basic Concepts
Document and Index curl -X PUT localhost:9200/cities/_doc/1 \ -H 'Content-Type:
application/json' \ -d '{ "city": "Tanabi", "state": "SP", "country": "BR", "population": 25000 }'
CLUSTER NODE 1 ★ NODE 2 NODE 3 Cluster and
Nodes
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 P1
Primary Shards
Primary Shard Benefits ▰ Elasticity ▰ Horizontal Scaling
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 Replica Shards 2/2
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 A A A B B B Cluster, Nodes and Shards 2/2
CLUSTER NODE 1 ★ NODE 2 NODE 3 P0 R0
R1 R1 R0 P1 A A A B B B Cluster, Nodes and Shards 2/2
Replica Shard Benefits ▰ H.A. ▰ Resilience ▰ Search Throughput
Topologies ▰ Default 7.0 1 / 1 ▰ Old Default
5 / 1 ▰ Search performance 1 / 10 ▰ Index performance 20 / 1
Index creation with shards curl -X PUT localhost:9200/cities \ -H
'Content-Type: application/json' \ -d '{ "settings": { "number_of_shards": 2, "number_of_replicas": 1 } }'
Searchable and Persistent Documents
curl -X PUT localhost:9200/cities/_doc/1 \ -H 'Content-Type: application/json' \ -d
'{ "city": "Tanabi", "state": "SP", "country": "BR", "population": 25000 }' && \ curl -X GET localhost:9200/cities/_search?pretty&q=name:Tanabi N R T
Search by segment (Lucene) Searchable Commit Point
Lucene commits are expensive ▰ fsync ▰ Disk Searchable Commit
Point
In-memory buffer and Translog
1. Documents are indexed In-memory buffer Searchable Commit Point {
} { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } Translog
2. Refresh In-memory buffer Searchable Commit Point { } {
} { } { } { } { } { } { } Translog
3. The translog keeps accumulating documents In-memory buffer Searchable Commit
Point { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } { } Translog { } { } { } { } { } { } { } { } { }
4. Flush (Lucene commit) In-memory buffer Searchable Commit Point Translog
Big picture CLUSTER NODE 1 ★ P0 R1 NODE 3
R0 R1 NODE 2 P1 R0 Searchable Commit Point { }{ }{ } In-memory buffer Translog { }{ }{ }{ }{ }{ }
Refresh interval curl -X PUT localhost:9200/cities/_settings \ -H 'Content-Type: application/json'
\ -d '{ "index" : { "refresh_interval" : "3s" } }'
?refresh (Index, Update, Delete, and Bulk) ▰ Empty or true
▰ wait_for ▰ false (default) POST cities/_refresh Refresh
Demo matheusfm/elasticsearch-demo
Thank you! matheusfm matheusfm mfariam