Machine Learning for Designers

Artificial intelligence is more and more becoming the core of digital products. Designing for Products based on AI requires Designers to know about Machine Learning.

This talk is an easy walk through the most important elements of Machine Learning. It looks at the fundamental principles of using practical examples. It showcases applications of the different types of Machine Learning. The use-cases range from text categorization to image recognition, on to speech analysis. The goal is to show what is important for designers and why.

Transcript

1. Machine Learning for Designers Memi Beltrame
2. Machine Learning for people with a fuzzy idea about it Or rather Memi Beltrame
3. Design is becoming physical, automated and connected
4. The convergence of disciplines Service Design Future design teams will unite these disciplines Interaction Design Industrial Design Katerina Kamprani - The Uncomfortable
5. https://pxhere.com/en/photo/1006116 An example
6. https://pxhere.com/en/photo/1006116 Her pain is your pain
7. https://pxhere.com/en/photo/1006116 Her pain is your pain Some of you probably are familiar with the situation: Whenever your baby is sick and crying there is a chance that there is something wrong with their ear. An infected eardrum has to be urgently taken care of to avoid lasting damage. This might be dfficult if you don't have immediate access to medical services, if you live in isolated places or do not have personal transportation
8. otoscope This is an
9. otoscope This is an It can be used to look at the eardrum to see if the ear is inflamed. Because the otoscope is connected to an iPhone, an image can be taken of the eardrum. You don't need medical staff which is present to examine the patient.
11. The image is sent to a service that tells me if I should go to a doctor or not.
12. AI: Image recognition, Data analysis Industrial Design Interaction Design Service Design
13. Machine Learning is the main driver
14. What is Machine Learning?
15. This service uses Machine Learning to make a prediction of the diagnosis
16. machine learning: training machines to receive input data and predict an output value
17. 3 methods how machines learn Supervised learning You train the machine with data The machine learns to make predictions ✔ ❌
18. 3 methods how machines learn Supervised learning You train the machine with data The machine learns to make predictions #1 method used in machine learning
19. 3 methods how machines learn Supervised learning You train the machine with data The machine learns to make predictions Unsupervised learning The machine is given a lot of data and it uses algorithms to find out interesting patterns.
20. Let's get some pizza data 1 2 3 4 5 6 1 2 3 4 5 6 7 Average # of pizzas per week Average # of toppings per pizza
21. Find patterns 1 2 3 4 5 6 1 2 3 4 5 6 7 Average # of toppings per pizza Average # of pizzas per week
22. Find patterns 1 2 3 4 5 6 1 2 3 4 5 6 7 Average # of toppings per pizza Average # of pizzas per week You can run this data through an algorithm and it finds groups of items that are close together.
23. Take Action 1 2 3 4 5 6 1 2 3 4 5 6 7 Average # of toppings per pizza Average # of pizzas per week
24. Take Action 1 2 3 4 5 6 1 2 3 4 5 6 7 Average # of toppings per pizza Average # of pizzas per week With these groups you now can directly address the different groups. The group on the top right probably are big households you can target specifically. The group on the left are those that order less frequently so you could address this and offer a super Tuesday for those that don't order on that day. The last one is for the people that love boring pizza: give them what they want, but larger! The applications of this clustering by unsupervised learning are market segmentation or fraud detection in banking.
25. 3 methods how machines learn Supervised learning You train the machine with data The machine learns to make predictions Unsupervised learning The machine is given a lot of data and it uses algorithms to find out interesting patterns. Reinforcement learning The machine continuously learns from the environment in an iterative fashion. 
 It starts dumb and gets smarter.
26. Reinforcement Learning The machine is given a set of rules and a goal
 • Physics: Gravity etc • Wheels turn • Goal get farther than previous cars
 It trains itself by keeping the features that helped it reach the goal. BoxCar 2D: Computation Intelligence Car Evolution (Needs Flash) http://boxcar2d.com/
27. Reinforcement Learning After a few dozen generations the machine has succeded in creating a vehicle that looks like a car and can reliably drive
28. #1 method: supervised learning Bedrooms m2 Neighbourhood Floors Sale Price 4 96 Hipsterton 2 1’500’000 2 89 Snoringham 3 750’000 3 75 Hipsterton 1 1’200’000 3 79 Snoringham 2 820’000 • Give the machine a training set with features • Give it the target values • It figures out how important each feature is • The machine can make predictions of target values Features Target
29. #1 method: supervised learning Bedrooms m2 Neighbourhood Floors 4 96 Hipsterton 2 2 89 Snoringham 3 3 75 Hipsterton 1 3 79 Snoringham 2 Predictions improve with • more features • a larger learning sample Features
30. #1 method: supervised learning Features Target 8 This also works for character recognition
31. #1 method: supervised learning Features Target owl Or image classification
32. Train the machine with edge cases • Train the machine to learn what matches and what does not • Train with edge cases Owl or Apple?
33. machines make predictions using algorithms
34. how machines use algorithms 500g white flour, 2 tsp salt 7g fast-action yeast
 3 tbsp olive oil 300ml water 475g plain flour, 1 tsp salt 10g dried yeast
 1 tbsp olive oil 400ml water The algorithm finds the valid weights of the individual features of a data-set to make the right prediction 2 cups flour, 1 cup salt 1 tsp olive oil 1 cup water Bread Bread Salty play dough
35. how machines use algorithms 1. Take a lot of training data 
 2. Pass it through a generic algorithm 
 (some mathematical formula)
 3. Let the machine figure out its own logic based on the data. Emails Generic Machine Learning Algorithm Spam Not Spam
36. generic algorithms There are many generic algorithms that already exist.
 
 
 The same generic algorithm can be used to solve problems in completely different areas. Emails Algorithm Spam Not Spam Articles Algorithm Finance Politics Sports
37. 2 types of algorithms Classification algorithms Emails Algorithm Spam Not Spam The goal is to predict discrete values, e.g. {1,0}, {True, False}, {spam, not spam}. Regression algorithms House- Details Algorithm Price of House The goal is to predict continuous values, e.g. home prices, weather temperatures A big part of ML is about classification
38. image recognition Chihuahua or Muffin? Most image recognition is about classification
39. image classification: pose estimation Performed in real time At multiple scales For a varying number of recognizable elements Realtime Multi-Person 2D Human Pose Estimation
40. What about language?
41. images as numbers Images can be recognized because their data can be encoded In this case all pixels are encoded as a grey-tone from 0 to 255 Can we do the same with language?
42. translation versus conversation Do you have the time? Translation goal: Produce an equivalent Conversation goal: Understand the meaning Avez-vous l’heure? It’s 7pm. Yes There are two main applications of Machine Lerarning in connection to language: Translation and conversation Translation goal: Produce an equivalent and grammatically correct sentence in a different language. Conversation goal: Understand the meaning of the sentence and follow up with the appropriate response.
43. translation versus conversation Do you have the time? Translation goal: Produce an equivalent Conversation goal: Understand the meaning Avez-vous l’heure? It’s 7pm. Yes
44. statistical translation
45. statistical translation Each word of the sentence can have several meanings.
46. statistical translation I try | to run | at | the prettiest | open space. I want | to run | per | the more tidy | open space. I mean | to forget | at | the tidiest | beach. I try | to go | per | the more tidy | seaside. I want | to go | to | the prettiest | beach. The algorithm compares the possible translations against existing ones. The algorithm picks the translation with the highest probability.
47. statistical translation Input Measurements of input sentence Output I want to go to the prettiest beach.
48. statistical translation The translation service DeepL is based on this type of statistical translation https://deepl.com
49. Audio Text conversational interfaces Machine learning is a crucial part of these interfaces.
50. new challenges and disciplines • recognizing intent • understanding context • voice and tone • shaping conversations in a humane way }Linguistics Ethics
51. intent - what does it all mean? types of meaning understand the words literal: understand the actual meaning implied: Do you have the time? metaphors & metonymies referenced: Wall Street is in crisis How long was Tony Blair Prime Minister In this case the algorithm didn't even understand the literal meaning of the question Machines have a hard time figuring out what we want to know. In a natural conversion the meaning of the things we say is not always straight forward.
52. intent - what does it all mean? types of meaning understand the words literal: understand the actual meaning implied: Do you have the time? metaphors & metonymies referenced: Wall Street is in crisis How long was Tony Blair Prime Minister In this case the algorithm didn't even understand the literal meaning of the question
53. This is from a conversation I had with a banking chatbot
54. Elements that make 
 this artificial: • Not picking up intent 
 „give me a spot on saturday“ • Literal repetition
55. context context is even harder than intent • the sequence in time • understanding the surroundings • semantic context 
 homonymy: " is not a #
56. voice and tone: change registers we adapt the way we speak to the situation we’re in Depending on: • how serious the situation is • how formal it is • how we are connected to the person Conversational interfaces need to take this into account. 
 This is a design task Yes Sporty Neutral Date Night Ready for your style? How would you describe your style? I'd totally raid your closet... 
 Sporty is my style! Do you wear colors or nah? Fab, I bet you look great in everything! Where are you going in your hot new outfit?
57. The designers’ role
58. Designers are content experts Icons by Sarah Rudkin Developers Build the machine Domain experts
 Have the domain specific knowledge Designers • Content oversight for training: 
 What makes good training data? • Mediator between engineering and domain experts • Ethical considerations
59. ethics matter Machines learn from us
 We choose what to teach We need to • challenge and stress test from a diverse point of view • put humans before technology
 (once again) • bring our principles of what good design is to the AI world This is a design task The images show a soap dispenser which does not work for dark skintones. Algorithms inherit all biases and blind spots of those who make them
60. Machine Learning is 
 everywhere Learn to see its opportunities Get a seat at the table now Understand the implications of using machine learning Bring Design principles into the mix to make empowering and ethical products
61. Thanks! I’m @bratwurstkomet 
 I like kitten and ice cream
62. Resources A visual introduction to machine learning 
 http://www.r2d3.us Machine Learning is Fun! 
 (the perfect series of articles to get you started)
 https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec3c471 30 Free Courses: Neural Networks, Machine Learning, AI
 https://www.datasciencecentral.com/profiles/blogs/neural-networks-for-machine-learning Watson Knowledge Studio 
 https://www.ibm.com/watson/developercloud/doc/wks/wks_overview_full.shtml 2 Minutes Papers: a youtube channel dedicated to condensing the results of scientific papers on artificial intelligence.
 https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg Realtime Multi-Person 2D Human Pose Estimation 
 https://www.youtube.com/watch?v=pW6nZXeWlGM BoxCar 2D: Computation Intelligence Car Evolution (Needs Flash)
 http://boxcar2d.com/ Google AI Experiments
 https://experiments.withgoogle.com/collection/ai Differences Between AI and Machine Learning, and Why it Matters
 https://medium.com/datadriveninvestor/differences-between-ai-and-machine-learning-and-why-it-matters-1255b182fc6