Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Elasticsearch for SQL Users
Search
Elastic Co
October 26, 2016
Technology
0
140
Elasticsearch for SQL Users
As given at the All Things Open 2016 conference.
Elastic Co
October 26, 2016
Tweet
Share
More Decks by Elastic Co
See All by Elastic Co
Les Vendredis noirs : même pas peur ! - Breizhcamp
elastic
15
780
Confoo Montreal: Ingest node: enriching documents within Elasticsearch
elastic
16
810
Elastic{ON} 2018 - Sipping from the Firehose: Scalable Endpoint Data for Incident Response
elastic
6
4.2k
Elastic{ON} 2018 - A Security Analytics Platform for Today
elastic
3
11k
Elastic{ON} 2018 - The State of Geo in Elasticsearch
elastic
7
12k
Elastic{ON} 2018 - Reliable by design - Applying formal methods to distributed systems
elastic
5
4.7k
Elastic{ON} 2018 - Bigger, Faster, Stronger - Leveling Up Enterprise Logging
elastic
1
4.9k
Elastic{ON} 2018: Latest in Logstash
elastic
1
4.5k
Elastic{ON} 2018 - Lessons Learned from Workday's Search Application Journey from POC to Production
elastic
2
2.4k
Other Decks in Technology
See All in Technology
Google Cloud Next '24でブログを10本書いた方法と勉強会を沸かせた方法
yasumuusan
0
310
Building a RAG-poweredAI chat appwith Python and VS Code
pamelafox
0
100
推しは推せるときに推せ! プロダクトにフィードバックしていこう
nakasho
0
320
Além do else! Categorizando Pokemóns com Pattern Matching no JavaScript
wmsbill
0
640
オーナーシップを持つ領域を明確にする
konifar
13
3.2k
Java EE/Jakarta EEの現状と将来―クラウドネイティブ時代にJava EEは対応できるのか?―
takakiyo
1
170
JSON攻略法.pdf
miyakemito
8
5.1k
家族アルバム みてねにおけるGrafana活用術 / Grafana Meetup Japan Vol.1 LT
isaoshimizu
1
780
20分で完全に理解するGrafanaダッシュボード
hamadakoji
3
680
GrafanaMeetup_AmazonManagedGrafanaのアクセス制御機能とマルチテナント環境下でのアクセス制御について
daitak
0
250
障害対応をちょっとずつよくしていくための 演習の作りかた
heleeen
0
240
ExaDB-D dbaascli で出来ること
oracle4engineer
PRO
0
2.1k
Featured
See All Featured
What the flash - Photography Introduction
edds
64
11k
The Pragmatic Product Professional
lauravandoore
25
5.8k
Why You Should Never Use an ORM
jnunemaker
PRO
51
8.6k
Large-scale JavaScript Application Architecture
addyosmani
504
110k
What’s in a name? Adding method to the madness
productmarketing
PRO
16
2.6k
Thoughts on Productivity
jonyablonski
58
3.8k
Principles of Awesome APIs and How to Build Them.
keavy
121
16k
Fireside Chat
paigeccino
21
2.6k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
501
140k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
The Cult of Friendly URLs
andyhume
74
5.7k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
244
20k
Transcript
1 Shaunak Kashyap Developer at Elastic @shaunak Elasticsearch for SQL
users
The Elastic Stack 2 Store, Index & Analyze Ingest User
Interface Plugins Hosted Service
3 Agenda Search queries Data modeling Architecture 1 2 3
2 4 Agenda Search queries Data modeling Architecture 1 3
5 Agenda Search queries Data modeling 1 2 3 Architecture
6 Search Queries https://www.flickr.com/photos/samhames/4422128094
7 CREATE TABLE IF NOT EXISTS emails ( sender VARCHAR(255)
NOT NULL, recipients TEXT, cc TEXT, bcc TEXT, subject VARCHAR(1024), body MEDIUMTEXT, datetime DATETIME ); CREATE INDEX emails_sender ON emails(sender); CREATE FULLTEXT INDEX emails_subject ON emails(subject); CREATE FULLTEXT INDEX emails_body ON emails(body); curl -XPUT 'http://localhost:9200/enron' -d' { "mappings": { "email": { "properties": { "sender": { "type": "keyword" }, "recipients": { "type": "keyword" }, "cc": { "type": "keyword" }, "bcc": { "type": "keyword" }, "subject": { "type": "text", "analyzer": "english" }, "datetime": { "type": "date" } } } } Schemas
8 Loading the data
9 [LIVE DEMO] • Search for text in a single
field • Search for text in multiple fields • Search for a phrase https://github.com/ycombinator/es-enron
10 Other Search Features Stemming Synonyms Did you mean? •
Jump, jumped, jumping • Queen, monarch • Monetery => Monetary
11 Data Modeling https://www.flickr.com/photos/samhames/4422128094 https://www.flickr.com/photos/ericparker/7854157310
12 To analyze (text) or not to analyze (keyword)? PUT
cities/city/1 { "city": "Raleigh", "population": 431746 } PUT cities/city/2 { "city": "New Albany", "population": 8829 } PUT cities/city/3 { "city": "New York", "population": 8406000 } POST cities/_search { "query": { "match": { "city": "New Albany" } } } QUERY + = ?
PUT cities/city/1 { "city": "Raleigh", "population": 431746 } 13 To
analyze (text) or not to analyze (keyword)? PUT cities/city/2 { "city": "New Albany", "population": 8829 } PUT cities/city/3 { "city": "New York", "population": 8406000 } Term Document IDs albany 2 new 2,3 raleigh 1 york 3
14 To analyze (text) or not to analyze (keyword)? PUT
cities { "mappings": { "city": { "properties": { "city": { "type": "keyword" } } } } } MAPPING Term Document IDs New Albany 2 New York 3 Raleigh 1
PUT blog/post/1 { "author_id": 1, "title": "...", "body": "..." }
PUT blog/post/2 { "author_id": 1, "title": "...", "body": "..." } PUT blog/post/3 { "author_id": 1, "title": "...", "body": "..." } 15 Relationships: Application-side joins PUT blog/author/1 { "name": "John Doe", "bio": "..." } POST blog/author/_search { "query": { "match": { "name": "John" } } } QUERY 1 POST blog/post/_search { "query": { "match": { "author_id": <each id from query 1 result> } } } QUERY 2
PUT blog/post/1 { "author_name": "John Doe", "title": "...", "body": "..."
} PUT blog/post/2 { "author_name": "John Doe", "title": "...", "body": "..." } 16 Relationships: Data denormalization POST blog/post/_search { "query": { "match": { "author_name": "John" } } } QUERY PUT blog/post/3 { "author_name": "John Doe", "title": "...", "body": "..." }
17 Relationships: Nested objects PUT blog/author/1 { "name": "John Doe",
"bio": "...", "blog_posts": [ { "title": "...", "body": "..." }, { "title": "...", "body": "..." }, { "title": "...", "body": "..." } ] } POST blog/author/_search { "query": { "match": { "name": "John" } } } QUERY
18 Relationships: Parent-child documents PUT blog/author/1 { "name": "John Doe",
"bio": "..." } POST blog/post/_search { "query": { "has_parent": { "type": "author", "query": { "match": { "name": "John" } } } QUERY PUT blog { "mappings": { "author": {}, "post": { "_parent": { "type": "author" } } } } PUT blog/post/1?parent=1 { "title": "...", "body": "..." } PUT blog/post/2?parent=1 { "title": "...", "body": "..." } PUT blog/post/3?parent=1 { "title": "...", "body": "..." }
19 Architecture https://www.flickr.com/photos/samhames/4422128094 https://www.flickr.com/photos/haribote/4871284379/
20 RDBMS Triggers database by Creative Stall from the Noun
Project 1 2
21 Async replication to Elasticsearch 1 2 3 ESSynchronizer flow
by Yamini Ahluwalia from the Noun Project
22 Async replication to Elasticsearch with Logstash 1 2 3
23 Forked writes from application 1 2
24 Forked writes from application (more robust) 1 2 queue
by Huu Nguyen from the Noun Project ESSynchronizer 3 4
25 Forked writes from application (more robust with Logstash) 1
2 3 4
26 Questions? @shaunak https://www.flickr.com/photos/nicknormal/2245559230/