The Trix to Creating a {Total}ly New Customer Experience: Elastic @ General Mills

As our marketing efforts become more precise, our search capabilities needed to change to keep up. Migrating from Endeca to Elastic has been an interesting experience. As Endeca is a configuration intensive application and the search engine is a bit of a black box, Elastic has allowed us to reduce the configuration effort to almost nothing. Untested script code script code is a thing of past. Our searching has become very precise and efficient. This talk will be about our journey.
Doug Nelson | Tech Lead | General Mills

Transcript

1. Doug Nelson The Trix to Creating a {Total}ly New Consumer Search Experience: Elastic @ General Mills Application Architect | General Mills October 6, 2016
2. General Mills in 50 Seconds 2
3. Open source at General Mills •  We are committed to using open source offerings •  Elastic •  Redis •  Kafka 3
4. How we use Search •  Indexed Content Delivery – ( started 4 - 5 years ago) §  Search Objects o Summary o Detail §  Classfications – Metadata system •  Sites are driven by Content Lists •  Site Search 4
5. Lets see it in action 5 BettyCrocker.Com Pillsbury.com
6. Endeca •  Single Tenant – Multiple Tenant •  Cluster of Endeca Servers •  One index per port •  Each Request is filled by a single server 6
7. Elastic •  Requests are distributed •  Indices are 3 shards with 2 replicas §  Single request only hits 3/5 of the data nodes •  Data/Master nodes – tcp only •  Client nodes – http 7
8. Index configuration •  Endeca §  Single Tenant §  Multi Tenant §  Multi Lingual •  Elastic §  Index by Tenant §  Index by Language §  Index by Version 8
9. Analysis and search relevancy •  Endeca §  Black Box Analyzer §  Single Metric per search interface for relevancy ranking •  Elastic §  Developed custom index analyzers §  Custom Scoring – Function Score Query §  Multiple Simultaneous Boosting Metrics 9
10. Processing search results •  Endeca §  Search result collection of name value pairs §  Servers had to hydrate a rich object §  For performance these had to be cached •  Elastic §  Index the rich object §  Servers only incur JSON deserialization hit §  We still cache these 10
11. Index configurations •  Endeca §  Single Tenant Indexes – repeated search configuration per index §  Custom Index Scripts – PERL (untested scripts) o A lot of these scripts were custom to each individual Endeca index •  Elastic §  Index configuration – Index Templates ( Single Configuration ) §  Index templates contain mappings and custom analyzer definitions §  Script code is now .Net code covered with unit tests. •  Single index configuration has been a huge timesaver as only one configuration needs to be maintained. 11
12. Additional Elastic gains •  Search Templates §  Parameterized Queries §  Specialized queries for each type of search §  Multiple query statements organized by search interface §  Sorting and Filtering •  Monitoring – Marvel •  Security – Shield •  Index Aliases §  Indices are versioned by timestamp §  Alias cover the index names §  Allows us to fall back to previous index 12
13. Questions? 13
14. www.elastic.co 14