Slide 1
Slide 1 text
elasticsearch
Slide 2
Slide 2 text
asm89
/
@iam_asm89
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
Why
do I need
a search engine?
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
Elasticsearch
Slide 9
Slide 9 text
Elasticsearch
● Schemaless document store
● Distributed and horizontally scalable
● Zero configuration setup
● REST
Slide 10
Slide 10 text
Unstructured search
Slide 11
Slide 11 text
Aggregations
Slide 12
Slide 12 text
Structured search
Slide 13
Slide 13 text
Enrichment
Slide 14
Slide 14 text
Sorting
Slide 15
Slide 15 text
Pagination
Slide 16
Slide 16 text
Suggestions
Slide 17
Slide 17 text
Installation
Slide 18
Slide 18 text
$ wget https://download../.../elasticsearch-1.2.1.tar.gz
$ tar xf elasticsearch-1.2.1.tar.gz
$ cd elasticsearch-1.2.1/
$ bin/elasticsearch
https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.2.1.tar.gz
Slide 19
Slide 19 text
Terminology
Slide 20
Slide 20 text
Relational database Elasticsearch
database ⇒ index
table ⇒ type
row ⇒ document
column ⇒ field
schema ⇒ mapping
index ⇒ (everything is indexed)
SQL ⇒ query DSL
Slide 21
Slide 21 text
APIs
Slide 22
Slide 22 text
$ curl -XPUT localhost:9200/social/tweet/42 -d '{
"text": "#dpc14 is awesome!"
}';
Slide 23
Slide 23 text
$ curl -XPUT localhost:9200/social/tweet/42 -d '{
"text": "#dpc14 is awesome!"
}';
$ curl -XGET localhost:9200/social/tweet/42
$ curl -XDELETE localhost:9200/social/tweet/42
Slide 24
Slide 24 text
$ curl localhost:9200/_search?query=dpc
Slide 25
Slide 25 text
$ curl localhost:9200/_search -d '{
"query": {
"query_string": {
"query": "dpc awesome"
}
}
}'
Slide 26
Slide 26 text
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"max_score" : 1,
"hits" : [
{
"_score" : 1,
"_id" : "42",
"_source" : {
"text" : "#dpc14 is awesome!"
},
"_type" : "tweet",
"_index" : "social"
}
],
"total" : 1
}
}
Slide 27
Slide 27 text
How does
a search engine
work?
Slide 28
Slide 28 text
cakedc/search
liip/search-bundle
symfony-cmf/search-bundle
elasticsearch/elasticsearch
doctrine/orm
doctrine/dbal
Take some text
Slide 29
Slide 29 text
cakedc search
liip search bundle
symfony cmf search bundle
elasticsearch elasticsearch
doctrine orm
doctrine dbal
Tokenize it
Slide 30
Slide 30 text
bundle liip
cakedc orm
cmf search
dbal elasticsearch
doctrine symfony
Find unique tokens
Slide 31
Slide 31 text
search bundle doctrine
cakedc/search
liip/search-bundle
symfony-cmf/search-bundle
elasticsearch/elasticsearch
doctrine/orm
doctrine/dbal
Link the terms and documents
Slide 32
Slide 32 text
cakedc search
liip search bundle
symfony cmf search bundle
elasticsearch elasticsearch
doctrine orm
doctrine dbal
Slide 33
Slide 33 text
search bundle doctrine
cakedc/search
liip/search-bundle
symfony-cmf/search-bundle
elasticsearch/elasticsearch
doctrine/orm
doctrine/dbal
Search for “bundle”
Slide 34
Slide 34 text
search bundle doctrine
cakedc/search
liip/search-bundle
symfony-cmf/search-bundle
elasticsearch/elasticsearch
doctrine/orm
doctrine/dbal
Search for “doctrine”
Slide 35
Slide 35 text
What is
stored in
Elasticsearch?
Slide 36
Slide 36 text
Document
{
"tweet": "PHP is GREAT",
"posted": "2014-06-28",
"user": {
"name": "Alexander",
"nick": "iam_asm89"
},
"tags": ["php", "opinion"],
"retweets": 42
}
Slide 37
Slide 37 text
Fields
{
"tweet": "PHP is GREAT",
"posted": "2014-06-28",
"user": {
"name": "Alexander",
"nick": "iam_asm89"
},
"tags": ["php", "opinion"],
"retweets": 42
}
Slide 38
Slide 38 text
Values
{
"tweet": "PHP is GREAT",
"posted": "2014-06-28",
"user": {
"name": "Alexander",
"nick": "iam_asm89"
},
"tags": ["php", "opinion"],
"retweets": 42
}
Slide 39
Slide 39 text
Field types
{ # object
"tweet": "PHP is GREAT", # string
"posted": "2014-06-28", # string
"user": { # nested object
"name": "Alexander", # string
"nick": "iam_asm89" # string
},
"tags": ["php", "opinion"], # array
"retweets": 42 # integer
}
Slide 40
Slide 40 text
Nested objects are flattened
{
"tweet": "PHP is GREAT",
"posted": "2014-06-28",
"user": {
"name": "Alexander",
"nick": "iam_asm89"
},
"tags": ["php", "opinion"],
"retweets": 42
}
Slide 41
Slide 41 text
Nested objects are flattened
{
"tweet": "PHP is GREAT",
"posted": "2014-06-28",
"user.name": "Alexander",
"user.nick": "iam_asm89",
"tags": ["php", "opinion"],
"retweets": 42
}
Slide 42
Slide 42 text
Values are analyzed into terms
{
"tweet": "PHP is GREAT",
"posted": "2014-06-28",
"user.name": "Alexander",
"user.nick": "iam_asm89",
"tags": ["php", "opinion"],
"retweets": 42
}
Slide 43
Slide 43 text
Values are analyzed into terms
{
"tweet": ['php', 'great'],
"posted": [Date(2014-06-28)],
"user.name": ['alexander'],
"user.nick": ['iam', 'asm89'],
"tags": ['php', 'opinion'],
"retweets": [42]
}
Slide 44
Slide 44 text
$ curl -XPUT localhost:9200/social/tweet/_mapping -d
'{
"tweet" : {
"properties" : {
"tweet" : {"type" : "string" },
"created" : {"type" : "date" }
}
}
}'
Slide 45
Slide 45 text
Mapping
● CAN add to an existing mapping
● CAN NOT change the mapping for a field
Slide 46
Slide 46 text
Field types
● String / integer / long / float / double / boolean / null
● Date
● Arrays
● IP
● Geo point
● Geo shape
Slide 47
Slide 47 text
Index settings
{
"type": "string",
"index": "analyzed"
}
# "Foo Bar" => ['foo', 'bar']
Slide 48
Slide 48 text
Index settings
{
"type": "string",
"index": "not_analyzed"
}
# "Foo Bar" => ['Foo Bar']
Slide 49
Slide 49 text
Index settings
{
"type": "string",
"index": "no"
}
# "Foo Bar" => []
Slide 50
Slide 50 text
Index settings
{
"type": "string",
"index": "no",
"analyzer": "default"
}
Slide 51
Slide 51 text
Analyzers
● Standard
● Simple
● Whitespace
● Stop
● Keyword
● Pattern
● Language
● Snowball
● Custom
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-analyzers.html
Slide 52
Slide 52 text
Token filters
● standard
● ascii folding
● length
● lowercase
● uppercase
● ngram
● edge ngram
● porter stem
● shingle
● stop
● word delimiter
● stemmer
● keyword marker
● snowball
● phonetic
● synonym
● compound word
● reverse
● truncate
● unique
● pattern replace
● trim
● hunspell
● normalization
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-tokenfilters.html
Slide 53
Slide 53 text
Aggregations
Slide 54
Slide 54 text
Aggregations
● Analytics
histograms, distributions, statistics
● Over a partition of your data
● Can be composed
Slide 55
Slide 55 text
No content
Slide 56
Slide 56 text
No content
Slide 57
Slide 57 text
No content
Slide 58
Slide 58 text
No content
Slide 59
Slide 59 text
No content
Slide 60
Slide 60 text
Percolator
Slide 61
Slide 61 text
Percolator
● Search backwards
● Register a query
● Get notified when a new document comes in that matches
the query
Slide 62
Slide 62 text
Percolator
● Real-time search result updates
● News alerts
● Price monitoring
● Logs monitoring
Slide 63
Slide 63 text
Shards
& clusters
Slide 64
Slide 64 text
No content
Slide 65
Slide 65 text
● More primary shards
– faster indexing
– scalability
● More replicas
– faster searching
– more failover
Slide 66
Slide 66 text
● Auto-discovery
● Single master
● Immediate failover with master re-election
Clustering
Slide 67
Slide 67 text
Tools
Slide 68
Slide 68 text
https://github.com/elasticsearch/elasticsearch-php
https://github.com/ruflin/Elastica
Slide 69
Slide 69 text
https://github.com/mobz/elasticsearch-head
Slide 70
Slide 70 text
Logstash
& Kibana
Slide 71
Slide 71 text
No content
Slide 72
Slide 72 text
Try it!
Slide 73
Slide 73 text
https://github.com/elasticsearch/elasticsearch
http://elasticsearch.org
#elasticsearch @ freenode IRC
Slide 74
Slide 74 text
@iam_asm89
https://joind.in/10882