Evolving Search at an ecommerce marketplace

Kaufland owns and runs an online marketplace, spanning several countries and languages, searching tens of millions of products, millions of times per day. Elasticsearch has been the product search engine at Kaufland e-commerce for more than a decade, surviving the move from data centers into the cloud and the internal shift from a Monolith to Microservices.

Our team owns search from the microservices, maintaining the whole cluster up to improving relevance. We'll cover the journey of continuous improvement over the last years, finding surprising performance improvements and how we plan to evolve the search in the next years.

Transcript

1. Evolving Search
2. Agenda
3. Introduction
4. Kau and e-commerce
5. About me
6. Questions?
7. Questions?
8. Architecture
9. Architecture BigQuery Kafka Indexer Elasticsearch Searcher Monolith reads reads reads writes reads
10. Architecture
11. Architecture
12. Searches
13. Phase: Analysis { "title": "Modernes Wandbild ... Kunstdruck New York", "search_stats": [ { "id_item": 306593629, "rank": 0.00012203718164007487, "term": "modernes wandbild new york" } ] }
14. Phase: Analysis - Category Guessing
15. Phase: Analysis - Shorter queries
16. Phase: Search
17. Search: FTS vs. signals
18. Quiz: ← ←
19. Running in production
20. Cost reduction
21. Journey to Elasticsearch 8
22. Node caches lling up
23. Node caches lling up
24. Upgrade: Unreleased shard locks
25. Performance
26. Performance
27. Less scripting esb .scriptScoreFunction( esb.script('source', "Math.sqrt(1+doc['rank'].value/10)"), ) esb .function(esb.fieldValueFactorFunction('rank').missing(1).weight(
28. Unexpected refreshes
29. Set
30. Set
31. Range query vs. term query { "term" : { "term_exact.token_count" : 2 } } { "range" : { "term_exact.token_count" : { "gte" : 2, "lte" : 2 } } }
32. Range query vs. term query
33. has no e ect
34. Tracking total hits: weird behaviour
35. Switching o a microservice
36. Improving relevance
37. Keyword search
38. ELSER
39. Query Processing Pipeline
40. Query expansion
41. Summary
42. Upgrade!
43. Summary
44. Thank you
45. One last thing
46. Thank you