UJUG Solr At Experticity

Using Solr for Store Catalog

Transcript

1. Experticity, Inc Product seeding to industry professionals and influencers. Des Lownds • 15 years of Software Development • 12 at startups • Older than Dirt • Not a Designer “When more of the right people are excited about your brand and knowledgeable about what you sell, more customers will buy more of the right products.” E-Learning for retail sales professionals.
2. Our eCommerce Challenges • Multi-tenant > 3K active catalogs > 1M skus > 100 fullfilment outlets • Full featured Deep Categorization Color Swatches Dynamic pricing model Product Bundles
3. The Epic Our OLTP database is struggling under the load of store catalog browsing. • Scalability is primary concern, Plan for: Dataset growth Query load increase • Short timeline • Support facetted browsing ( group and count queries )
4. The Plan • Use Solr as backed for catalog browsing • Use replication to scale query load • OLTP remains as system of record • Use DIH to synchronize data • Denormalize from dozens of tables to 3 document types • Continue to support OLTP as a fallback
5. Unforseen Problems Memory Constraints • Each product can exist in many categories with a different sort order in each • Solving this with Dynamic Fields in solr resulted in very high max heap requirements Lesson Learned: • If theres a natural way to shard your data up front, Do It. • Free schema brings its own challenges.
6. Unforseen problems DIH • Granular data synchronization with DIH is complex. • DIH was too heavy handed for inventory feeds. Lesson Learned: • Don't underestimate Data Ingestion • Consider document centric event driven rather than an ETL derived approach
7. What went well • All initial goals were accomplished, except dataset growth goals, and we have a plan for that. • Laid a groundwork for transition from monolithic application to componentized architecture • Solr is now key part of our architecture, supporting store catalog, geolocation queries, and other searches