Conducting Independent Surveys with AI Search

Transcript

2. 2024/12/3 Masanori Okazaki Internal AI Utilization: Conducting Independent Surveys with AI Search - Search with Vectors Refine with LLM
3. Self-introduction CTO Office, Infrastructure Division Leader Masanori Okazaki Career After graduating from a technical college, I worked for 10 years at a control system SIer in my hometown of Ibaraki. Later, I was invited to Tokyo by someone I met through work and moved there almost unintentionally. I gained experience at a startup venture and as the sole IT department of a small research company before joining GMO Research & AI. Hobbies: My dog, F1, and road biking Favorite thing: My dog
4. Independent Survey Questionnaire? The focus this time was on a response rate survey questionnaire. Specifically, examples of such requests include: 'I want to survey 100 people who use the YYY lotion from XXX cosmetics company.’ First, an actual survey is conducted to check the response rate, which indicates the percentage of respondents meeting the target criteria. Based on these results, the number of people to whom the survey should be distributed is calculated to achieve the goal of 100 respondents, and the survey is then distributed.
5. What about so far? The survey results were stored in Google Drive and were being searched using Google Drive's search functionality.Challenges:The large volume of data made searching difficult.Search was limited to exact string matches, making it hard to pinpoint the desired information.Consequences:Surveys that had already been conducted were sometimes repeated.It took a long time to locate the desired information.
6. Then let's try it With semantic search, using natural language,Wouldn't it become easier to find what you need?Even with vague memory of phrases, you might find something close to your goal.
7. Semantic Search? 0 0.5 1 1.5 2 2.5 3 3.5 0 0.5 1 1.5 2 2.5 3 Semantic Search is a search technology that understands the meaning of queries and documents, providing more relevant results. In this case, words and sentences were converted into a vector space,making their meanings comparable as numerical values. Unlike keyword-based searches, it enables matching based on meaning and context
8. 第1弾 Vectorize questions and choices to try searching with natural language! First, vectorize and save questions and choices Survey Results Response Rate Vectorization of Questions + Choices Survey about XXX: Visited YYY: 1 year ago Vectorization BigQuery
9. 第1弾 Try searching with vectors. BigQuery OpenAI Embeding People who have visited YYYY within the past two years Vectorization VECTOR_SEARCH
10. Phase 1 Results Results ・How close (in terms of distance in vector space) should the results be output? ・There is noise. The search results are unexpectedly hard to interpret... So, what should we do?
11. Phase 2 Let’s refine the search results Using LLM! (Similar to RAG)
12. Phase 2 BigQuery OpenAI Embeding OpenAI Text Generation People who have visited YYYY within the past two years Vectorization VECTOR_SEARCH Text Generation
13. Phase 2 Key Points for Refinement １．Convert vector search results to CSV and include them in the prompt. ２．Since I wanted to use the refined results for additional database searches,I specified the output format.
14. Phase 2 Results Results 1. Noise was removed, and only necessary items are displayed! ２.Even with output formatting, additional database searches functioned smoothly and reliably. 3. The idea of 'semantically refining data' felt refreshing. Surprisingly, it turned out quite well!
15. Thank you for your attention!