Elasticsearch 💙 Neos

The presentation starts with some basics about the elasticsearch integration in Neos CMS. Shows new features and changes in the elasticsearch contentrepository adapter version 7 and introduces some handy tools like the queue indexer and the elastic sync package.

Transcript

1. Neos Elasticsearch
   

   View Slide

2. Daniel Lienert
   • Software Architect

   @ punkt.de / Karlsruhe
   • Neos Core Team Member
   • @dlienert
   

   View Slide

3. Flowpack.
   ElasticSearch.
   ContentRepositoryAdaptor
   

   View Slide

4. 1. Why you want to use it
   2. What it provides
   3. Whats new in version 7
   4. Packages around the CR Adapter
   

   View Slide

5. View Slide

6. Search
   

   View Slide

7. View Slide

8. View Slide

9. Flowpack.SearchPlugin
   

   View Slide

10. Amazon S3
    H1
    Amazon S3
    H3
    Amazon S3
    H…
    Amazon S3
    text
    Amazon S3
    H2
    

    View Slide

11. Fulltext Indexing
    

    View Slide

12. Custom result rendering
    

    View Slide

13. Query Data
    

    View Slide

14. FlowQuery
    

    View Slide

15. Find And Filter
    

    View Slide

16. Sort And Limit
    

    View Slide

17. Elasticsearch
    

    View Slide

18. Find And Filter
    

    View Slide

19. Sort And Limit
    

    View Slide

20. View Slide

21. Aggregations
    

    View Slide

22. Aggregations
    

    View Slide

23. Elasticsearch
    Contentrepository
    Adapter v7
    

    View Slide

24. Compatible to
    Elasticsearch 6.x and 7.x
    

    View Slide

25. Version 5.6 not longer supported
    Version EOL Date
    5.6.x 2019-03-11
    6.8.x 2020-11-20
    7.6.x 2021-08-11
    

    View Slide

26. Removal of mapping types
    Flow Classname
    Neos NodeType
    _type
    

    View Slide

27. Elasticsearch Data Storage - History
    Elasticsearch v1
    Index 1 Index 2
    Type A Type B Type A Type …
    field_a: integer field_a: string
    

    View Slide

28. Elasticsearch Data Storage - History
    Elasticsearch > v2
    Index 1 Index 2
    Type A Type B Type A Type …
    field_a: integer field_a: integer
    

    View Slide

29. Removal of mapping types
    Elasticsearch > v6
    Index 1 Index 2
    field_a: integer field_a: integer field_a: string field_a: string
    

    View Slide

30. Flow Classname
    Neos NodeType
    neos_type
    Removal of mapping types
    

    View Slide

31. One Index per
    Dimension Combination
    

    View Slide

32. 1. Language Specific Analyzer and
    Filter
    2. Language Specific StopWords
    3. Indexes are easier to maintain
    

    View Slide

33. As before.
    Same configuration
    apply to all indices
    

    View Slide

34. Configure every
    dimension index
    separately
    

    View Slide

35. Caution:
    The configurations are
    not merged
    

    View Slide

36. Index Names until v6
    Pattern:
    -
    Example
    neoscr-1586069025
    Alias
    neoscr -> neoscr-1586069025
    

    View Slide

37. Index Names v7
    Pattern:
    --
    Example
    •neoscr-4f534b1eb0c1a785da31e681fb5e91ff-1586069025
    •neoscr-0359ed5c416567b8bc2e5ade0f277b36-1586069025
    Alias
    neoscr-0359ed5c416567b8bc2e5ade0f277b36 ->
    neoscr-0359ed5c416567b8bc2e5ade0f277b36-1586069025
    

    View Slide

38. Show Dimension to Index Mapping
    ./flow nodeindexmaping:indices
    

    View Slide

39. Exclude NodeTypes
    from Indexing
    

    View Slide

40. Configuration
    per NodeType
    

    View Slide

41. Show Indexable Configuration
    ./flow nodetype:showIndexableConfiguration
    

    View Slide

42. Index build steps /
    parts are executed in
    isolated subcommands
    

    View Slide

43. Isolated index build steps
    

    View Slide

44. New commands help
    debugging
    

    View Slide

45. Test fulltext search
    ./flow search:fulltext
    

    View Slide

46. Show what has been indexed
    ./flow search:viewnode [] [<field>]
    

    View Slide

47. Show what has been indexed
    ./flow search:viewnode [] [<field>]
    

    View Slide

48. Elasticsearch
    Packages
    

    View Slide

49. Queue Indexer
    

    View Slide

50. Slicer
    Worker
    Worker
    Worker
    Worker
    Work
    

    View Slide

51. View Slide

52. Flowpack.ElasticSearch.ContentRepositoryQueueIndexer
    

    View Slide

53. Elastic Sync
    

    View Slide

54. Elasticsearch
    Neos
    Neos
    Local
    Elasticsearch
    Staging
    

    View Slide

55. View Slide

56. PunktDe.Elastic.Sync
    

    View Slide

57. Asset Usage
    

    View Slide

58. View Slide

59. PunktDe.Elastic.AssetUsageInNodes
    

    View Slide

60. @dlienert daniellienert
    

    View Slide