Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What's New in Percolator

Luca Cavanna
October 02, 2013
14k

What's New in Percolator

Presentation given at the BeLux elasticsearch meetup on October 2nd 2013 in Brussels

Luca Cavanna

October 02, 2013
Tweet

Transcript

  1. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    What’s New in
    Percolator?
    @lucacavanna
    Elasticsearch User Group BeLux

    View full-size slide

  2. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Agenda

    View full-size slide

  3. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Agenda
    What is the percolator?
    What do people use it for?
    How does it work?
    Why did it need a redesign?
    What was changed exactly?

    View full-size slide

  4. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Agenda
    What is the percolator?
    What do people use it for?
    How does it work?
    Why did it need a redesign?
    What was changed exactly?

    View full-size slide

  5. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Agenda
    What is the percolator?
    What do people use it for?
    How does it work?
    Why did it need a redesign?
    What was changed exactly?

    View full-size slide

  6. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Agenda
    What is the percolator?
    What do people use it for?
    How does it work?
    Why did it need a redesign?
    What was changed exactly?

    View full-size slide

  7. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Agenda
    What is the percolator?
    What do people use it for?
    How does it work?
    Why did it need a redesign?
    What has changed?

    View full-size slide

  8. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Perco ...what?

    View full-size slide

  9. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    We are used to docs...

    View full-size slide

  10. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    PUT /twitter/tweet/1 -d '
    {
    "tweet": "#elasticsearch is AWESOME",
    "nick": "@lucacavanna",
    "name": "Luca Cavanna",
    "date": "2013-10-02",
    "rt": 5,
    "loc": {
    ! "lat": 13.4,
    ! "lon": 52.5
    }
    }
    '
    We are used to docs...

    View full-size slide

  11. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    ...and queries

    View full-size slide

  12. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    ...and queries
    GET /_search -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    }
    }
    '

    View full-size slide

  13. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Let’s do the opposite

    View full-size slide

  14. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Let’s index a query...

    View full-size slide

  15. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    PUT /_percolator/twitter/es-tweets
    Let’s index a query...
    verb (index request)

    View full-size slide

  16. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    PUT /_percolator/twitter/es-tweets
    Let’s index a query...
    reserved index

    View full-size slide

  17. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    PUT /_percolator/twitter/es-tweets
    Let’s index a query...
    index to register query against

    View full-size slide

  18. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    PUT /_percolator/twitter/es-tweets
    Let’s index a query...
    query identifier

    View full-size slide

  19. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Let’s index a query...
    PUT /_percolator/twitter/es-tweets -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    }
    }
    '
    the query

    View full-size slide

  20. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/_percolate
    ...and percolate a doc
    verb (percolate request)

    View full-size slide

  21. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/_percolate
    ...and percolate a doc
    index to percolate against
    (needs to exist although empty)

    View full-size slide

  22. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/_percolate
    ...and percolate a doc
    percolate api endpoint

    View full-size slide

  23. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/_percolate -d '
    {
    “doc”: {
    "tweet": "#elasticsearch is AWESOME",
    "nick": "@lucacavanna",
    "name": "Luca Cavanna",
    "date": "2013-10-02",
    "rt": 5,
    "loc": {
    ! "lat": 13.4,
    ! "lon": 52.5
    }
    }
    }
    '
    ...and percolate a doc
    document

    View full-size slide

  24. {
    "ok": "true",
    "matches": ["es-tweets"]
    }
    Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    ...and percolate a doc
    # 200 OK
    matching queries

    View full-size slide

  25. {
    "ok": "true",
    "matches": ["es-tweets"]
    }
    Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    ...and percolate a doc
    # 200 OK
    PUT /_percolator/twitter/es-tweets -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    }
    }
    '

    View full-size slide

  26. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Why would you do that?

    View full-size slide

  27. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited

    View full-size slide

  28. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Let me know when it’s coming!

    View full-size slide

  29. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    weather forecast alerts
    PUT /_percolator/forecast/summer -d '
    {
    "query": {
    “range”: {
    “max” : {
    “gte” : 30
    }
    }
    }
    }
    '

    View full-size slide

  30. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    weather forecast alerts
    GET /forecast/type/_percolate -d '
    {
    “doc” : {
    "date": "2013-07-25",
    "wind": 1,
    "min": "22",
    "max": "32",
    "rain": 0
    }
    }
    '

    View full-size slide

  31. {
    "ok": "true",
    "matches": ["summer"]
    }
    Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    # 200 OK
    weather forecast alerts

    View full-size slide

  32. {
    "ok": "true",
    "matches": ["summer"]
    }
    Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    # 200 OK
    the summer is coming!
    weather forecast alerts

    View full-size slide

  33. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Alerting applies also to:

    View full-size slide

  34. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Alerting applies also to:
    Price monitoring
    News alerts
    Stock alerts
    Logs monitoring

    View full-size slide

  35. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Alerting applies also to:
    Price monitoring
    News alerts
    Stock alerts
    Logs monitoring

    View full-size slide

  36. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Alerting applies also to:
    Price monitoring
    News alerts
    Stock alerts
    Logs monitoring

    View full-size slide

  37. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Alerting applies also to:
    Price monitoring
    News alerts
    Stock alerts
    Logs monitoring

    View full-size slide

  38. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Alerting applies also to:
    Price monitoring
    News alerts
    Stock alerts
    Logs monitoring
    ...and many more

    View full-size slide

  39. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited

    View full-size slide

  40. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited

    View full-size slide

  41. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited

    View full-size slide

  42. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    How do I make my ads better?

    View full-size slide

  43. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Store users queries
    PUT /_percolator/ads/user1 -d '
    {
    "query": {
    “match”: {
    “text” : “iphone subscription”,
    “operator” : “and”
    }
    }
    }
    '

    View full-size slide

  44. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Let’s percolate the ad...
    GET /ads/apple/_percolate -d '
    {
    “doc” : {
    "text": "iPhone from 1 euro",
    "url": “http://www.myads.com/myads”
    }
    }
    '

    View full-size slide

  45. {
    "ok": "true",
    "matches": []
    }
    Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    # 200 OK
    No query matches
    ...and get feedback back

    View full-size slide

  46. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Let’s modify the ad...
    GET /ads/apple/_percolate -d '
    {
    “doc” : {
    "text": “iPhone subscription from 1 euro",
    "url": “http://www.myads.com/myads”
    }
    }
    '

    View full-size slide

  47. {
    "ok": "true",
    "matches": ["user1"]
    }
    Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    # 200 OK
    How many people would
    the updated ad reach?
    ...and get feedback back

    View full-size slide

  48. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Enhance documents

    View full-size slide

  49. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Store articles patterns
    PUT /_percolator/article/apple -d '
    {
    "query": {
    “multi_match”: {
    “fields” : [“title”,”content”],
    “query” : “ios apple iphone ipad”
    }
    }
    }
    '

    View full-size slide

  50. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    percolate a new article
    GET /article/tech/_percolate -d '
    {
    “doc” : {
    "title": "iOS 7 released",
    "content": “iOS 7, the seventh major
    iteration of the mobile platform, includes more
    than 200 new features, including a brand new
    bright and colorful user interface.”
    }
    }
    '

    View full-size slide

  51. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    percolate a new article
    {
    "ok": "true",
    "matches": ["apple"]
    }
    GET /article/tech/_percolate -d '
    {
    “doc” : {
    "title": "iOS 7 released",
    "content": “iOS 7, the seventh major
    iteration of the mobile platform, includes more
    than 200 new features, including a brand new
    bright and colorful user interface.”
    }
    }
    '

    View full-size slide

  52. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    index the enriched article
    PUT /article/tech/1 -d '
    {
    "title": "iOS 7 released",
    "content": “iOS 7, the seventh major iteration
    of the mobile platform, includes more than 200
    new features, including a brand new bright and
    colorful user interface.”,
    “tag” : “apple”
    }
    '

    View full-size slide

  53. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    How does it work?

    View full-size slide

  54. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /_percolator/_mapping
    special mapping

    View full-size slide

  55. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /_percolator/_mapping
    {
    "_default_" : {
    "properties" : {
    "query" : {
    "type" : "object",
    "enabled" : false
    }
    }
    }
    }
    special mapping

    View full-size slide

  56. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /_percolator/_mapping
    {
    "_default_" : {
    "properties" : {
    "query" : {
    "type" : "object",
    "enabled" : false
    }
    }
    }
    }
    special mapping
    queries are only stored

    View full-size slide

  57. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    How does it work?

    View full-size slide

  58. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    How does it work?
    All queries are loaded in memory
    Each doc is indexed in memory
    All queries get executed against it
    Execution time linear (# of queries)
    The memory index gets cleaned up

    View full-size slide

  59. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    How does it work?
    All queries are loaded in memory
    Each doc is indexed in memory
    All queries get executed against it
    Execution time linear (# of queries)
    The memory index gets cleaned up

    View full-size slide

  60. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    How does it work?
    All queries are loaded in memory
    Each doc is indexed in memory
    All queries get executed against it
    Execution time linear (# of queries)
    The memory index gets cleaned up

    View full-size slide

  61. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    How does it work?
    All queries are loaded in memory
    Each doc is indexed in memory
    All queries get executed against it
    Execution time linear to # of queries
    The memory index gets cleaned up

    View full-size slide

  62. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    How does it work?
    All queries are loaded in memory
    Each doc is indexed in memory
    All queries get executed against it
    Execution time linear to # of queries
    The memory index gets cleaned up

    View full-size slide

  63. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Query metadata
    PUT /_percolator/twitter/es-tweets -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    }
    }
    '

    View full-size slide

  64. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Query metadata
    PUT /_percolator/twitter/es-tweets -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    },
    “user_id”: “lucacavanna”
    }
    '

    View full-size slide

  65. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Query metadata
    PUT /_percolator/twitter/es-tweets -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    },
    “user”: {
    “name” : “Luca Cavanna”,
    “nick” : “lucacavanna”
    }
    }
    '

    View full-size slide

  66. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Retrieve query by id
    GET /_percolator/twitter/es-tweets

    View full-size slide

  67. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /_percolator/twitter/es-tweets
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    },
    “user”: {
    “name” : “Luca Cavanna”,
    “nick” : “lucacavanna”
    }
    }
    Retrieve query by id

    View full-size slide

  68. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/_percolate -d '
    {
    “doc” : {
    "tweet": "#elasticsearch is AWESOME",
    "nick": "@lucacavanna"
    },
    “query” : {
    “term” : {“user.nick” : “lucacavanna”}
    }
    }
    '
    Drill down

    View full-size slide

  69. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/_percolate -d '
    {
    “doc” : {
    "tweet": "#elasticsearch is AWESOME",
    "nick": "@lucacavanna"
    },
    “query” : {
    “term” : {“user.nick” : “lucacavanna”}
    }
    }
    '
    Drill down
    restricts the # of queries to run

    View full-size slide

  70. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /_percolator/_settings
    special settings

    View full-size slide

  71. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /_percolator/_settings
    {
    "_percolator" : {
    "settings" : {
    "index.version.created" : "900599",
    "index.number_of_replicas" : "0",
    "index.number_of_shards" : "1",
    "index.auto_expand_replicas" : "0-all"
    }
    }
    }
    special settings

    View full-size slide

  72. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Node 1
    _percolator
    0P

    View full-size slide

  73. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Node 1
    _percolator
    0P
    Node 2
    _percolator
    0R

    View full-size slide

  74. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Node 1
    _percolator
    0P
    Node 2
    _percolator
    0R
    Node 3
    _percolator
    0R

    View full-size slide

  75. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Looks pretty cool,
    doesn’t it?

    View full-size slide

  76. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    What’s wrong with it
    then?

    View full-size slide

  77. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    A single shard doesn’t
    scale!

    View full-size slide

  78. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    What has changed in
    master (1.0)?

    View full-size slide

  79. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Let’s index a query...
    PUT /_percolator/twitter/es-tweets -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    }
    }
    '

    View full-size slide

  80. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Let’s index a query...
    PUT /anyindex/twitter/es-tweets -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    }
    }
    '
    any index can contain queries to
    percolate against

    View full-size slide

  81. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Let’s index a query...
    PUT /anyindex/_percolator/es-tweets -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    }
    }
    '
    reserved type for queries to
    percolate against

    View full-size slide

  82. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    How many shards now?

    View full-size slide

  83. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    As many as you like!

    View full-size slide

  84. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    This can actually scale!

    View full-size slide

  85. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /index1,index2/tweet/_percolate -d '
    {
    “doc”: {
    "tweet": "#elasticsearch is AWESOME",
    "nick": "@lucacavanna",
    "name": "Luca Cavanna",
    "date": "2013-10-02",
    "rt": 5,
    "loc": {
    ! "lat": 13.4,
    ! "lon": 52.5
    }
    }
    }
    '
    Multi-index support

    View full-size slide

  86. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /alias1/tweet/_percolate -d '
    {
    “doc”: {
    "tweet": "#elasticsearch is AWESOME",
    "nick": "@lucacavanna",
    "name": "Luca Cavanna",
    "date": "2013-10-02",
    "rt": 5,
    "loc": {
    ! "lat": 13.4,
    ! "lon": 52.5
    }
    }
    }
    '
    Full alias support

    View full-size slide

  87. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Routing support
    PUT /anyindex/_percolator/id?routing=user1 -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    }
    }
    '

    View full-size slide

  88. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/_percolate?routing=user1 -d '
    {
    “doc”: {
    "tweet": "#elasticsearch is AWESOME",
    "nick": "@lucacavanna",
    "name": "Luca Cavanna",
    "date": "2013-10-02",
    "rt": 5,
    "loc": {
    ! "lat": 13.4,
    ! "lon": 52.5
    }
    }
    }
    Routing support

    View full-size slide

  89. {
    "took": 9,
    “_shards” : {
    “total” : 5,
    “successful” : 5,
    “failed” : 0
    },
    “count” : 1,
    "matches": [
    {
    “_index” : “index1”,
    “_id” : “es-tweets”
    }
    ]
    }
    Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    New percolate response

    View full-size slide

  90. {
    "took": 9,
    “_shards” : {
    “total” : 5,
    “successful” : 5,
    “failed” : 0
    },
    “count” : 1,
    "matches": [
    {
    “_index” : “index1”,
    “_id” : “es-tweets”
    }
    ]
    }
    Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    New percolate response

    View full-size slide

  91. {
    "took": 9,
    “_shards” : {
    “total” : 5,
    “successful” : 5,
    “failed” : 0
    },
    “count” : 1,
    "matches": [
    {
    “_index” : “index1”,
    “_id” : “es-tweets”
    }
    ]
    }
    Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    New percolate response

    View full-size slide

  92. {
    "took": 9,
    “_shards” : {
    “total” : 5,
    “successful” : 5,
    “failed” : 0
    },
    “count” : 1,
    "matches": [
    {
    “_index” : “index1”,
    “_id” : “es-tweets”
    }
    ]
    }
    Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    New percolate response

    View full-size slide

  93. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Some brand new
    features

    View full-size slide

  94. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/1/_percolate
    Percolate existing doc

    View full-size slide

  95. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/1/_percolate
    Percolate existing doc
    1) index that contains the document

    View full-size slide

  96. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/1/_percolate
    Percolate existing doc
    1) index that contains the document
    2) and the queries (_percolator type)

    View full-size slide

  97. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/1/_percolate
    Percolate existing doc
    type & id of the existing doc

    View full-size slide

  98. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/1/_percolate?
    percolate_index=my-percolator
    Percolate existing doc
    separate index where the queries are

    View full-size slide

  99. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/_percolate/count -d '
    {
    “doc”: {
    "tweet": "#elasticsearch is AWESOME",
    "nick": "@lucacavanna",
    "name": "Luca Cavanna",
    "date": "2013-10-02",
    "rt": 5,
    "loc": {
    ! "lat": 13.4,
    ! "lon": 52.5
    }
    }
    }
    '
    Percolate count api

    View full-size slide

  100. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Percolate count api
    {
    "took": 9,
    “_shards” : {
    “total” : 5,
    “successful” : 5,
    “failed” : 0
    },
    “count” : 1
    }

    View full-size slide

  101. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/_percolate -d '
    {
    “doc” : {
    "tweet": "#elasticsearch is AWESOME",
    "nick": "@lucacavanna"
    },
    “filter” : {
    “term” : {“user.nick” : “lucacavanna”}
    }
    }
    '
    Support for filters

    View full-size slide

  102. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Highlighting
    PUT /twitter/_percolator/1 -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    }
    }
    '

    View full-size slide

  103. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Highlighting
    PUT /twitter/_percolator/1 -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    }
    }
    '
    PUT /twitter/_percolator/2 -d '
    {
    "query": {
    "match": { "tweet": "awesome" }
    }
    }
    '

    View full-size slide

  104. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    GET /twitter/tweet/_percolate -d '
    {
    “doc”: {
    "tweet": "#elasticsearch is AWESOME"
    },
    “highlight”: {
    “fields” : { “tweet”:{} }
    },
    “size” : 5
    }
    '
    Highlighting

    View full-size slide

  105. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    {
    .........
    “matches”: [
    {
    “_index” : “twitter”,
    “_id” : “1”,
    "tweet": "#elasticsearch is AWESOME"
    },
    {
    “_index” : “twitter”,
    “_id” : “2”,
    "tweet": "#elasticsearch is AWESOME"
    }
    ]
    }
    '
    Highlighting

    View full-size slide

  106. PUT /_percolator/twitter/es-tweets -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    },
    “user”: {
    “name” : “Luca Cavanna”,
    “nick” : “lucacavanna”
    }
    }
    '
    Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Basic scoring

    View full-size slide

  107. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Basic scoring
    PUT /_percolator/twitter/es-tweets -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    },
    “user”: {
    “name” : “Luca Cavanna”,
    “nick” : “lucacavanna”
    }
    }
    '
    GET /twitter/tweet/_percolate -d '
    {
    “doc” : {
    "tweet": "#elasticsearch is AWESOME",
    "nick": "@lucacavanna"
    },
    “query” : {
    “match” : {“user.name” : “Luca”}
    }
    }
    '

    View full-size slide

  108. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Basic scoring
    PUT /_percolator/twitter/es-tweets -d '
    {
    "query": {
    "match": { "tweet": "elasticsearch" }
    },
    “user”: {
    “name” : “Luca Cavanna”,
    “nick” : “lucacavanna”
    }
    }
    '
    GET /twitter/tweet/_percolate -d '
    {
    “doc” : {
    "tweet": "#elasticsearch is AWESOME",
    "nick": "@lucacavanna"
    },
    “query” : {
    “match” : {“user.name” : “Luca”}
    }
    }
    '
    How well does
    it match?

    View full-size slide

  109. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Sorting
    Sort by score
    Only return the score
    Limit number of results
    Further improvements

    View full-size slide

  110. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Sorting
    Sort by score
    Only return the score
    Limit number of results
    Further improvements

    View full-size slide

  111. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Sorting
    Sort by score
    Only return the score
    Limit number of matches
    Further improvements

    View full-size slide

  112. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Sorting
    Sort by score
    Only return the score
    Limit number of matches
    Boosting based on metadata

    View full-size slide

  113. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Sorting
    Sort by score
    Only return the score
    Limit number of matches
    Boosting based on metadata
    Further improvements

    View full-size slide

  114. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Multi percolate
    GET /_mpercolate

    View full-size slide

  115. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    Multi percolate
    GET /_mpercolate
    {“percolate” : {“index”:”twitter”, “type”:”tweet”} }
    {“doc”: {“text”:“elasticsearch is AWESOME!”} }
    {“percolate” : {“index”:”twitter”, “type”:”tweet”, “id”:”id1”} }
    {}
    {“count” : {“index”:”twitter”, “type”:”tweet”} }
    {“doc”: {“text”:“elasticsearch rocks!”} }
    {“count” : {“index”:”twitter”, “type”:”tweet”, “id”:”id2”} }
    {}

    View full-size slide

  116. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
    www.elasticsearch.org

    View full-size slide