Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elasticsearch 💙 Neos

Elasticsearch 💙 Neos

The presentation starts with some basics about the elasticsearch integration in Neos CMS. Shows new features and changes in the elasticsearch contentrepository adapter version 7 and introduces some handy tools like the queue indexer and the elastic sync package.

Daniel Lienert

April 08, 2020
Tweet

More Decks by Daniel Lienert

Other Decks in Technology

Transcript

  1. Neos Elasticsearch

    View Slide

  2. Daniel Lienert
    • Software Architect

    @ punkt.de / Karlsruhe
    • Neos Core Team Member
    • @dlienert

    View Slide

  3. Flowpack.
    ElasticSearch.
    ContentRepositoryAdaptor

    View Slide

  4. 1. Why you want to use it
    2. What it provides
    3. Whats new in version 7
    4. Packages around the CR Adapter

    View Slide

  5. View Slide

  6. Search

    View Slide

  7. View Slide

  8. View Slide

  9. Flowpack.SearchPlugin

    View Slide

  10. Amazon S3
    H1
    Amazon S3
    H3
    Amazon S3
    H…
    Amazon S3
    text
    Amazon S3
    H2

    View Slide

  11. Fulltext Indexing

    View Slide

  12. Custom result rendering

    View Slide

  13. Query Data

    View Slide

  14. FlowQuery

    View Slide

  15. Find And Filter

    View Slide

  16. Sort And Limit

    View Slide

  17. Elasticsearch

    View Slide

  18. Find And Filter

    View Slide

  19. Sort And Limit

    View Slide

  20. View Slide

  21. Aggregations

    View Slide

  22. Aggregations

    View Slide

  23. Elasticsearch
    Contentrepository
    Adapter v7

    View Slide

  24. Compatible to
    Elasticsearch 6.x and 7.x

    View Slide

  25. Version 5.6 not longer supported
    Version EOL Date
    5.6.x 2019-03-11
    6.8.x 2020-11-20
    7.6.x 2021-08-11

    View Slide

  26. Removal of mapping types
    Flow Classname
    Neos NodeType
    _type

    View Slide

  27. Elasticsearch Data Storage - History
    Elasticsearch v1
    Index 1 Index 2
    Type A Type B Type A Type …
    field_a: integer field_a: string

    View Slide

  28. Elasticsearch Data Storage - History
    Elasticsearch > v2
    Index 1 Index 2
    Type A Type B Type A Type …
    field_a: integer field_a: integer

    View Slide

  29. Removal of mapping types
    Elasticsearch > v6
    Index 1 Index 2
    field_a: integer field_a: integer field_a: string field_a: string

    View Slide

  30. Flow Classname
    Neos NodeType
    neos_type
    Removal of mapping types

    View Slide

  31. One Index per
    Dimension Combination

    View Slide

  32. 1. Language Specific Analyzer and
    Filter
    2. Language Specific StopWords
    3. Indexes are easier to maintain

    View Slide

  33. As before.
    Same configuration
    apply to all indices

    View Slide

  34. Configure every
    dimension index
    separately

    View Slide

  35. Caution:
    The configurations are
    not merged

    View Slide

  36. Index Names until v6
    Pattern:
    -
    Example
    neoscr-1586069025
    Alias
    neoscr -> neoscr-1586069025

    View Slide

  37. Index Names v7
    Pattern:
    --
    Example
    •neoscr-4f534b1eb0c1a785da31e681fb5e91ff-1586069025
    •neoscr-0359ed5c416567b8bc2e5ade0f277b36-1586069025
    Alias
    neoscr-0359ed5c416567b8bc2e5ade0f277b36 ->
    neoscr-0359ed5c416567b8bc2e5ade0f277b36-1586069025

    View Slide

  38. Show Dimension to Index Mapping
    ./flow nodeindexmaping:indices

    View Slide

  39. Exclude NodeTypes
    from Indexing

    View Slide

  40. Configuration
    per NodeType

    View Slide

  41. Show Indexable Configuration
    ./flow nodetype:showIndexableConfiguration

    View Slide

  42. Index build steps /
    parts are executed in
    isolated subcommands

    View Slide

  43. Isolated index build steps

    View Slide

  44. New commands help
    debugging

    View Slide

  45. Test fulltext search
    ./flow search:fulltext

    View Slide

  46. Show what has been indexed
    ./flow search:viewnode [] [<field>]

    View Slide

  47. Show what has been indexed
    ./flow search:viewnode [] [<field>]

    View Slide

  48. Elasticsearch
    Packages

    View Slide

  49. Queue Indexer

    View Slide

  50. Slicer
    Worker
    Worker
    Worker
    Worker
    Work

    View Slide

  51. View Slide

  52. Flowpack.ElasticSearch.ContentRepositoryQueueIndexer

    View Slide

  53. Elastic Sync

    View Slide

  54. Elasticsearch
    Neos
    Neos
    Local
    Elasticsearch
    Staging

    View Slide

  55. View Slide

  56. PunktDe.Elastic.Sync

    View Slide

  57. Asset Usage

    View Slide

  58. View Slide

  59. PunktDe.Elastic.AssetUsageInNodes

    View Slide

  60. @dlienert daniellienert

    View Slide