Food, Wine and Machine Learning: Teaching a Bot to Taste

I presented this at RubyKaigi 2017 in Hiroshima. It explores some ways to use Machine Learning and AI in Ruby.

Transcript

1. Food, Wine and Machine Learning: Teaching a Bot to Taste Mai Nguyen Twitter - @happywinebot / @containsulfite www.happywinebot.com happywinebot@gmail.com
2. About me Ruby/Rails development 2006 - 2010 in Washington DC 2016 - Moved to Wellington, Senior Developer at Loyalty NZ Took a break to travel, study, and work in Winemaking Worked in Australia, France, California, New Zealand Missed learning new things everyday
3. Why I started learning machine learning People always ask me wine questions Many kinds of wine, and no one knows what they should drink There must be some way to automate this
4. What is Machine Learning? Artificial Intelligence (AI) Machine Learning (ML) Deep Learning Natural Language Processing (NLP)
5. What is Machine Learning? Does not rely on coded rules Creates its own model based on training data Can be supervised or unsupervised • Supervised - where data examples have known outputs to train upon • Unsupervised - no outputs defined, finds hidden structure in unlabeled data Many types of algorithms for different problems
6. What can machine learning do for me? Infinite Combination of Inputs ∞ Finite Output (Boolean, number, etc) class Machine def learn(data_set) do_something(data_set) end def predict(input) # kanpai!
7. Usecases for machine learning Classification - spam filtering, sentiment, fraud detection, ad targeting & personalisation, medical diagnosis Imputation - infer missing values of input to make complete datasets Recommendations - products, job recruiting, dating, content Predictions - stock-market, demand forecasting, weather, sports results, asset management
8. Features of Machine Learning Accuracy improves as you collect more data Scalable Can be automated - learn automatically as answers are validated Can be fast Customisable - built from your own data
9. Why aren’t you using it? I like Ruby, I don’t want to write Python. I don’t have time to learn these algorithms. I am not a data scientist.
10. Ruby resources Algorithms and tools ported to Ruby http://sciruby.com/ Natural Language Processing gems: https://github.com/arbox/nlp-with-ruby Machine Learning gems: https://github.com/arbox/machine-learning-with-ruby PyCall - call Python from Ruby
11. Popular ML APIs and services BigML Amazon Machine Learning APIs (only in N. Va and Ireland) Microsoft Azure Machine Learning APIs (not all regions) Google Cloud Machine Learning Engine - https://github.com/somaticio/tensorflow.rb
12. Popular NLP APIs and services Wit.ai Microsoft Language Understanding Intelligent Service (LUIS) API MonkeyLearn Google Cloud Speech and Natural Language API, api.ai Amazon Alexa (N. Va and Oregon) and Lex (N. Va only) IBM Watson API
13. First steps What is the question you want to answer? What data do you have access to? Custom data Free public data - UC Irvine Machine Learning Repository, Kaggle.com, etc
14. What makes good data? Representative of future data Complete Relevant features - minimal noise Lots of it - the more the better!
15. Making your data better Feature engineering: transforming inputs into predictive features Fill in missing data Impute with adjacent data Mean/median Machine learning to make an educated guess Use only the inputs that are relevant*
16. The Machine Learning Modeling Process Optimisation Prediction & Evaluation Training Data Preparation Test Dataset Training Dataset Model Building
17. The Machine Learning Prediction Process Data Preparation New Data for Prediction Predicted Results! Prediction
18. When you wouldn’t use machine learning Rules are known, well defined and finite High accuracy Data is unavailable / difficult to obtain
19. Machine learning challenges Mistakes in training data can be hard to spot Biases in your training data can be magnified 100% accuracy is near impossible Testing is difficult - edge cases Future data may not resemble past data Determining successful outcome
20. A practical example: HappyWineBot Problem: Average consumers find wine intimidating and don’t know how to select wines for their meals Solution: Build a chatbot on the internet that will educate, entertain, and match wine and food
21. HappyWineBot will need to: Converse with users to answer questions Understand tastes and flavours of food Understand tastes and flavours of wine Evaluate food and wine combinations for good matches
22. Teach the bot to converse First explored AIML -- not ideal for large vocabulary NLP API services Twitter integration (pull) Facebook integration (push) Chat content to answer questions
23. HappyWineBot will need to: ✔ Converse with users to answer questions Understand tastes and flavours of food Understand tastes and flavours of wine Evaluate food and wine combinations for good matches
24. Tasting fundamentals Sweet Bitter Physical sensations* 10,000 to 1 trillion smells Umami Salty Sour Fatty
25. Features of food that affect taste Ingredients Herbs Spices Protein Cooking Method Tastes Flavours Texture Weight Intensity
26. Teach the bot about food Use personal cooking experience to generate supervised training data Different flavour and taste attributes --> features to predict Ingredients and cooking methods == Recipe Recipes need to be complete and detailed Online recipe sites - Scrape only relevant parts Food Training Data
27. Teach the bot about food Models 50+ features per recipe Features chosen specifically for interactions with wine One classification model per flavour / feature
28. Salty - ketchup, cheese, pickles, mustard Sweet - ketchup Sour - ketchup, pickles, tomato, mustard Bitter - grilling flavours Umami - beef, tomato, ketchup Soft - bun ... Recipe Classification Example
29. HappyWineBot will need to: ✔ Converse with users to answer questions ✔ Understand tastes and flavours of food Understand tastes and flavours of wine Evaluate food and wine combinations for good matches
30. Features of wine that affect taste Climate Grape Varieties Viticultural Practices Winemaking Decisions Oak Style Age Viscosity / Weight Flavour Intensity Tannins Acidity Sweetness Bitterness Aromas and Flavours
31. Teach the bot about wine Wine Data 100+ wine categories codified with 40+ attributes Rely on previous knowledge, experience, wine literature Focus on general categories of wine for now Wine information for individual wines is inconsistent
32. HappyWineBot will need to: ✔ Converse with users to answer questions ✔ Understand tastes and flavours of food ✔ Understand tastes and flavours of wine Evaluate food and wine combinations for good matches
33. What is a good food and wine pairing? Wine and food enhance each other Wine enhances the food Food enhances the wine
34. Strategies for pairing wine and food Origin of the wine and cuisine Match flavour intensity and weight Complement the basic tastes for harmony Match and contrast flavours, textures Avoid problematic combinations
35. Teach the bot to evaluate pairings Weighted scores for ranking Wine and food matching - inherently rules-based Well-defined rules, so ML is not necessary Rely on previous knowledge and experience, literature
36. Demo
38. Lessons learned Time consuming to generate data manually Chat content Recipe features for training Wine types and attributes Can never have too much data People say many unexpected things
39. Takeaways Machine Learning can answer so many questions! Many options to use Machine Learning in your Ruby stack Quality and quantity of data is very important What are you waiting for, give machine learning a try!
40. Cheers! Twitter - @happywinebot / @containsulfite www.happywinebot.com happywinebot@gmail.com