Query Understanding and Ecommerce

This presentation, delivered as a "last lecture" at eBay, discusses query understanding in ecommerce search. It explains that query understanding is critical for relevance and desirability, and search applications should guide users to more specific queries they can better understand. It also discusses modeling queries as "bags of documents" and vice versa to understand similarity between queries and documents. The goal is to recognize queries with the same intent to improve recall, and model documents based on the queries intended to find them.

Transcript

1. Query Understanding and Ecommerce Daniel Tunkelang
2. Introduction • Many moving parts need to align to succeed in ecommerce search. • Query understanding is perhaps the most critical of those parts. • Query understanding offers extraordinary potential for improvement.
3. Overview • Relevance and Desirability • Guiding Searchers to Better Queries • Search Queries vs. Search Intents • Bags of Documents, Bags of Queries
4. Relevance and Desirability
5. Retrieval + ranking focus on relevance + desirability. • Relevance = how much a result responds to the query. • Desirability = the query-independent utility of a result. • This is a simplified model that ignores personalization and other factors. • But what does all this have to do with query understanding?
6. What matters most to the buyer is query-dependent. • A relevance-only model would depend only on query-dependent signals. • A smarter approach filters on relevance but then focuses on desirability. • Queries expressing more specific intent set a higher bar for relevance. christmas ornaments gaggia brera water tank vs.
7. Guiding Searchers to Better Queries
8. Application should guide searchers to better queries. • No query understanding model or system is ever going to be perfect. • A search application will understand some queries but not others. • A failure to understand the query undermines retrieval and ranking. • Hence, autocomplete, related searches, and all other query suggestions should promote queries that the search application can understand!
9. Suggest unambiguous, high-specificity queries. • A search application should never suggest queries it cannot understand. • In particular, that means not suggesting ambiguous queries, e.g., “mixer”. • All else equal, it should favor more specific over less specific queries. • Queries with higher specificity tend to have higher conversion rates. • That is why it is important to model and measure query specificity!
10. Search Queries vs. Search Intents
11. • Information retrieval researchers worry about queries with multiple intents. jaguar or ? • A more practical concern is multiple queries that map to the same intent. lightning to 3.5mm iphone to aux Search Query != Search Intent
12. Search experience should be based on the intent. Recall? CTR? Conversion Rate? ... ? or
13. Use queries with similar intent to increase recall. Similar but not equivalent intent.
14. Bags of Documents, Bags of Queries
15. Bag of documents: query as mean of product vectors. ► ► [0.13, 0.81, … ] [0.09, 0.75, … ] … ► [0.11, 0.79, … ] [0.13, 0.81, … ] [0.09, 0.77, … ] … ► [0.12, 0.78, … ] ► cos > 0.98 black tshirts for men mens black t-shirt
16. ML generalizes the bag-of-documents model to tail queries. • Train using (query1, query2, similarity) triples from offline model. • Oversample similar query pairs to increase sensitivity where it matters. • Fine-tune a pre-trained micro-BERT sentence transformer model. • Concatenate the output of a query classifier to the query keywords.
17. Duality: we can model a document as a bag of queries. • A document can be modeled based on the queries intended to find it. = mens t shirt black tshirts for men … • We can use this model to measure retrievability, which is recall in practice. • Useful as feedback for indexing, both in general and for the particular item.
18. Summary • The tradeoff between relevance and desirability depends on query specificity. • All query suggestions should be unambiguous, preferably high-specificity. • Measure query similarity to recognize queries with same or similar intent. • Model queries as bags of documents, and documents as bags of queries.
19. One more thing…
20. Thank you! dtunkelang@gmail.com https://www.linkedin.com/in/dtunkelang/ https://dtunkelang.medium.com/ https://queryunderstanding.com/ http://contentunderstanding.com/