Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Food, Wine and Machine Learning: Teaching a Bot to Taste

A9ae99f612eaf56b16cdf12d35ba74e1?s=47 Mai Nguyen
September 19, 2017

Food, Wine and Machine Learning: Teaching a Bot to Taste

I presented this at RubyKaigi 2017 in Hiroshima. It explores some ways to use Machine Learning and AI in Ruby.

A9ae99f612eaf56b16cdf12d35ba74e1?s=128

Mai Nguyen

September 19, 2017
Tweet

Transcript

  1. Food, Wine and Machine Learning: Teaching a Bot to Taste

    Mai Nguyen Twitter - @happywinebot / @containsulfite www.happywinebot.com happywinebot@gmail.com
  2. About me Ruby/Rails development 2006 - 2010 in Washington DC

    2016 - Moved to Wellington, Senior Developer at Loyalty NZ Took a break to travel, study, and work in Winemaking Worked in Australia, France, California, New Zealand Missed learning new things everyday
  3. Why I started learning machine learning People always ask me

    wine questions Many kinds of wine, and no one knows what they should drink There must be some way to automate this
  4. What is Machine Learning? Artificial Intelligence (AI) Machine Learning (ML)

    Deep Learning Natural Language Processing (NLP)
  5. What is Machine Learning? Does not rely on coded rules

    Creates its own model based on training data Can be supervised or unsupervised • Supervised - where data examples have known outputs to train upon • Unsupervised - no outputs defined, finds hidden structure in unlabeled data Many types of algorithms for different problems
  6. What can machine learning do for me? Infinite Combination of

    Inputs ∞ Finite Output (Boolean, number, etc) class Machine def learn(data_set) do_something(data_set) end def predict(input) # kanpai!
  7. Usecases for machine learning Classification - spam filtering, sentiment, fraud

    detection, ad targeting & personalisation, medical diagnosis Imputation - infer missing values of input to make complete datasets Recommendations - products, job recruiting, dating, content Predictions - stock-market, demand forecasting, weather, sports results, asset management
  8. Features of Machine Learning Accuracy improves as you collect more

    data Scalable Can be automated - learn automatically as answers are validated Can be fast Customisable - built from your own data
  9. Why aren’t you using it? I like Ruby, I don’t

    want to write Python. I don’t have time to learn these algorithms. I am not a data scientist.
  10. Ruby resources Algorithms and tools ported to Ruby http://sciruby.com/ Natural

    Language Processing gems: https://github.com/arbox/nlp-with-ruby Machine Learning gems: https://github.com/arbox/machine-learning-with-ruby PyCall - call Python from Ruby
  11. Popular ML APIs and services BigML Amazon Machine Learning APIs

    (only in N. Va and Ireland) Microsoft Azure Machine Learning APIs (not all regions) Google Cloud Machine Learning Engine - https://github.com/somaticio/tensorflow.rb
  12. Popular NLP APIs and services Wit.ai Microsoft Language Understanding Intelligent

    Service (LUIS) API MonkeyLearn Google Cloud Speech and Natural Language API, api.ai Amazon Alexa (N. Va and Oregon) and Lex (N. Va only) IBM Watson API
  13. First steps What is the question you want to answer?

    What data do you have access to? Custom data Free public data - UC Irvine Machine Learning Repository, Kaggle.com, etc
  14. What makes good data? Representative of future data Complete Relevant

    features - minimal noise Lots of it - the more the better!
  15. Making your data better Feature engineering: transforming inputs into predictive

    features Fill in missing data Impute with adjacent data Mean/median Machine learning to make an educated guess Use only the inputs that are relevant*
  16. The Machine Learning Modeling Process Optimisation Prediction & Evaluation Training

    Data Preparation Test Dataset Training Dataset Model Building
  17. The Machine Learning Prediction Process Data Preparation New Data for

    Prediction Predicted Results! Prediction
  18. When you wouldn’t use machine learning Rules are known, well

    defined and finite High accuracy Data is unavailable / difficult to obtain
  19. Machine learning challenges Mistakes in training data can be hard

    to spot Biases in your training data can be magnified 100% accuracy is near impossible Testing is difficult - edge cases Future data may not resemble past data Determining successful outcome
  20. A practical example: HappyWineBot Problem: Average consumers find wine intimidating

    and don’t know how to select wines for their meals Solution: Build a chatbot on the internet that will educate, entertain, and match wine and food
  21. HappyWineBot will need to: Converse with users to answer questions

    Understand tastes and flavours of food Understand tastes and flavours of wine Evaluate food and wine combinations for good matches
  22. Teach the bot to converse First explored AIML -- not

    ideal for large vocabulary NLP API services Twitter integration (pull) Facebook integration (push) Chat content to answer questions
  23. HappyWineBot will need to: ✔ Converse with users to answer

    questions Understand tastes and flavours of food Understand tastes and flavours of wine Evaluate food and wine combinations for good matches
  24. Tasting fundamentals Sweet Bitter Physical sensations* 10,000 to 1 trillion

    smells Umami Salty Sour Fatty
  25. Features of food that affect taste Ingredients Herbs Spices Protein

    Cooking Method Tastes Flavours Texture Weight Intensity
  26. Teach the bot about food Use personal cooking experience to

    generate supervised training data Different flavour and taste attributes --> features to predict Ingredients and cooking methods == Recipe Recipes need to be complete and detailed Online recipe sites - Scrape only relevant parts Food Training Data
  27. Teach the bot about food Models 50+ features per recipe

    Features chosen specifically for interactions with wine One classification model per flavour / feature
  28. Salty - ketchup, cheese, pickles, mustard Sweet - ketchup Sour

    - ketchup, pickles, tomato, mustard Bitter - grilling flavours Umami - beef, tomato, ketchup Soft - bun ... Recipe Classification Example
  29. HappyWineBot will need to: ✔ Converse with users to answer

    questions ✔ Understand tastes and flavours of food Understand tastes and flavours of wine Evaluate food and wine combinations for good matches
  30. Features of wine that affect taste Climate Grape Varieties Viticultural

    Practices Winemaking Decisions Oak Style Age Viscosity / Weight Flavour Intensity Tannins Acidity Sweetness Bitterness Aromas and Flavours
  31. Teach the bot about wine Wine Data 100+ wine categories

    codified with 40+ attributes Rely on previous knowledge, experience, wine literature Focus on general categories of wine for now Wine information for individual wines is inconsistent
  32. HappyWineBot will need to: ✔ Converse with users to answer

    questions ✔ Understand tastes and flavours of food ✔ Understand tastes and flavours of wine Evaluate food and wine combinations for good matches
  33. What is a good food and wine pairing? Wine and

    food enhance each other Wine enhances the food Food enhances the wine
  34. Strategies for pairing wine and food Origin of the wine

    and cuisine Match flavour intensity and weight Complement the basic tastes for harmony Match and contrast flavours, textures Avoid problematic combinations
  35. Teach the bot to evaluate pairings Weighted scores for ranking

    Wine and food matching - inherently rules-based Well-defined rules, so ML is not necessary Rely on previous knowledge and experience, literature
  36. Demo

  37. None
  38. Lessons learned Time consuming to generate data manually Chat content

    Recipe features for training Wine types and attributes Can never have too much data People say many unexpected things
  39. Takeaways Machine Learning can answer so many questions! Many options

    to use Machine Learning in your Ruby stack Quality and quantity of data is very important What are you waiting for, give machine learning a try!
  40. Cheers! Twitter - @happywinebot / @containsulfite www.happywinebot.com happywinebot@gmail.com