2017 How Deep Learning Changes the Design Process (1)

In this talk, presented at the UX Forum in Munich, I had a look at the Artificial Intelligence landscape, and the differences between Machine Learning and Deep Learning. The session walks through a big range of use cases in fields such as object and face recognition and showcases how the design industry is charging up with new powers.

Transcript

1. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S © Lawrence Lek Part 1 1 / 2
2. What is Deep Learning? © Lawrence Lek
3. Alan Turing, 1950
4. Dartmouth Summer Research Project, 1956
5. „ H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S EVERY ASPECT OF LEARNING OR ANY OTHER FEATURE OF INTELLIGENCE CAN BE SO PRECISELY DESCRIBED THAT A MACHINE CAN BE MADE TO SIMULATE IT J. McCarthy, Dartmouth College, 1956
6. Artificial Intelligence AI landscape © Michael Copeland
7. Artificial Intelligence AI landscape © Michael Copeland
8. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S AI landscape THE TERMS ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING ARE OFTEN USED INTERCHANGEABLY – YET THEY AREN’T THE SAME
9. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Machine Learning Machine learning is the ability to learn without being explicitly programmed. Programs that can learn from their own mistakes and improve their performance over time.
10. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Machine Learning Data mining Image recognition Facebook newsfeed Netflix suggestions Big Data analysis Amazon suggestions Predictive analytics
11. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Machine Learning Notifications Predictive Keyboards Spam/Quality Control Systems Copywriting Design Patterns Gesture Recognition Content Actions/Feedback Magic Numbers Search Related Content Character Recognition
12. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Machine Learning Notifications Predictive Keyboards Ranked Feeds Spam/Quality Control Systems Copywriting Design Patterns Gesture Recognition Content Actions/Feedback Magic Numbers Search Related Content Character Recognition © Jan Korsanke
13. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S AI landscape Artificial intelligence is becoming increasingly ubiquitous within the technology industry, with capabilities that are much more practical than most consumers may think.
14. Artificial Intelligence AI landscape AI landscape © Narrative Science
15. E P I C O R C H A P T E R AI landscape © Narrative Science
16. Artificial Intelligence AI landscape © cloud-nqb.com
17. Artificial Intelligence AI landscape © Lux Research
18. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S AI vs Big Data © Matthew Mayo Machine Learning Artifical Intelligence Data Mining Data Science Deep Learning Big Data
19. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Deep Learning Deep Learning is based on Artificial Neural Networks. These are systems that are designed to process information in ways that are similar to the ways biological brains work. It uses a certain set of Machine Learning algorithms that run in multiple layers.
20. Artificial Intelligence Machine Learning vs Deep Learning © Rasmus Rothe
21. „ H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S AI HAS BY NOW SUCCEEDED IN DOING ESSENTIALLY EVERYTHING THAT REQUIRES ‘THINKING’ BUT HAS FAILED TO DO MOST OF WHAT PEOPLE DO ‘WITHOUT THINKING.’ Donald Knuth, Stanford University
22. Why it is important? © Lawrence Lek
23. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Why it is important? Possible Desirable Profitable A user experience powered by a system that learns © Fabien Girardin
24. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Why it is important? Google has decided to pivot its business model from that simple mobile-first toward machine learning.
25. Why it is important?
26. Why it is important?
27. Mobile Frameworks & libraries TensorFlow / TensorFlow Lite Caffe2Go ML Kit
28. © Frank Chen
29. © Frank Chen
30. © Frank Chen
31. © Frank Chen
32. © Frank Chen
33. What can be done today? © Lawrence Lek
34. Image recognition error rate
35. OBJECT RECOGNITION USE CASES
36. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google Lens Can an algorithm see what you see and help you take action based on this information?
37. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google Lens Google Lens turns the camera from a passive tool that’s capturing the world around you to one that’s allowing you to interact with what’s in your camera’s viewfinder.
38. Google Lens
39. Google Lens
41. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Photoeditor SDK Can an algorithm extract foreground content from its background?
42. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Photoeditor SDK Professional image editing tools help you to select distinctive objects, but, they aren’t available on your mobile device, where you take and publish the images, and they usually require some hands-on time, before you can produce anything usable.
43. Photoeditor SDK
44. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Photoeditor SDK
45. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Adobe - Deep Image Matting Can an algorithm extract foreground content from its background – in movies too?
46. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Adobe - Deep Image Matting Previous algorithms have poor performance when an image has similar foreground and background colors or complicated textures. With neural networks you can extract foreground content from its background intelligently and accurately - in movies. This might kill the green screen.
47. Adobe - Deep Image Matting
48. Adobe - Deep Image Matting
49. Adobe - Deep Image Matting
50. Adobe - Deep Image Matting
51. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Entrupy Can an algorithm help to spot a real Chanel bag from a fake?
52. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Entrupy Counterfeits are a painful thorn in the side of luxury fashion brands but they can be even more of a headache for digital re-sellers. Trying to spot a real Chanel from a fake? Deep Learning tech can help.
53. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Entrupy Entrupy is a portable scanning device that instantly detects imitation designer bags by taking microscopic pictures that take into account details of the material, processing, workmanship, serial number, and wear/tear. It then employs the technique of deep learning to compare the images against a vast database that includes top luxury brands and if the bag is deemed authentic, users immediately get a Certificate of Authenticity.
54. Entrupy
55. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S EyeEm - Vision Can an algorithm learn and suggest aesthetics in a way an art director would?
56. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S EyeEm - Vision What are aesthetics anyway? Is it just what you “like”? How does it all work? When you as a human find it hard to express what you like do you think a machine going to find it easy?
57. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S EyeEm - Vision EyeEm - Vision
58. „ H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S ANY SUFFICIENTLY ADVANCED TECHNOLOGY IS INDISTINGUISHABLE FROM MAGIC. Arthur C. Clarke, British science fiction writer
59. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Neural Storyteller Can an algorithm make up stories from any given image?
60. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Neural Storyteller Neural Storyteller is an artificial intelligence model that when given an image, can generate a story about the image using features in the image.
61. Neural Storyteller H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S
62. Neural Storyteller H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S
63. Neural Storyteller H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S
64. Neural Storyteller H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S
65. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Rapto Can an algorithm turn your normal day into a rap song?
66. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Rapto Rapto lets you create music using artificial intelligence & your camera. Simply point the camera towards any object around you and Rapto will use it's inbuilt neural network to understand the object & create rap music!
67. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Rapto
68. FACE RECOGNITION USE CASES
69. E P I C O R C H A P T E R Subtopic or example New versions of facial recognition technologies use Deep Learning, as it is especially effective for image recognition because it makes a computer zero in on the facial features that will most reliably identify a person.
70. Apple - Machine Leraning Kit
71. Apple - Live Photo Loops
72. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google Allo Can an algorithm create a Bitmoji for you on the fly?
73. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google Allo With Google Allo simply snap a selfie, and it’ll return an automatically generated illustrated version of you, on the fly.
74. Google Allo
75. Google Allo
76. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Nest Cam IQ Can an algorithm detect strangers?
77. E P I C O R C H A P T E R Nest Cam IQ The Nest Cam IQ can differentiate between friends and family members, or a stranger. Insights can range from telling you the kids are home from school to sending an alert if an unfamiliar person is in the living room.
78. Nest Cam IQ
79. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S China & Russia Can an algorithm register me at events?
80. E P I C O R C H A P T E R China & Russia In China, facial recognition technology is already finding consumer applications. In Russia, FindFace is an controversial an app used to identify members of a crowd to match them with social network accounts.
81. China & Russia
82. DRAWING RECOGNITION USE CASES
83. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Apple Notes Can an algorithm understand what you are writing?
84. Apple Notes
85. Apple - Natural Language API
86. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google Quick Draw Can an algorithm understand what you are drawing?
87. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google Quick Draw
88. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google Quick Draw
89. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google AutoDraw Can an algorithm help to teach you drawing?
90. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google AutoDraw AutoDraw’s suggestion tool uses the same technology used in QuickDraw, to guess what you’re trying to draw. Right now, it can guess hundreds of drawings and we look forward to adding more over time.
91. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google AutoDraw
92. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google AutoDraw
93. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S pix2pix Can an algorithm take your drawings to generate images?
94. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S pix2pix Image-to-image is a Tensorflow port of pix2pix. The pix2pix model works by training on pairs of images such as building facade labels to building facades, and then attempts to generate the corresponding output image from any input image you give it.
95. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S pix2pix
96. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S pix2pix
97. pix2pix
98. pix2pix
99. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Drawing music Can an algorithm take your drawings to generate music?
100. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Drawing music Kene Cogan has built an experiment where a classified musical object triggers an according sound track in Ableton Live.
101. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Drawing music
102. OTHER USE CASES
103. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Style Transfer Can an algorithm re-create complex aesthetics and even fine art?
104. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Style Transfer In fine art humans have mastered the skill to create unique visual experiences through composing a complex interplay between content and style. Style Transfer is an artificial system that creates artistic images of high perceptual quality.
105. Style Transfer
106. Style Transfer
107. Style Transfer
108. Style Transfer
109. Style Transfer
110. Style Transfer
111. Adobe Research
112. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Logo Experiment Can an algorithm generate logo artworks?
113. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Logo Experiment Rob Peart tried to generate Death Metal logos from Neural Networks ;)
114. Darkthrone Logo Experiment
115. E P I C O R C H A P T E R Waking The Cadaver Logo Experiment
116. E P I C O R C H A P T E R From the sunset, forest and grief Logo Experiment
117. E P I C O R C H A P T E R Logo Experiment
118. E P I C O R C H A P T E R Logo Experiment
119. E P I C O R C H A P T E R Logo Experiment
120. E P I C O R C H A P T E R Logo Experiment
121. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Logo Experiment Can an algorithm generate logo artworks?
122. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S NVIDIA - Iray Can an algorithm squeeze render times?
123. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S NVIDIA - Iray NVIDIA brings AI to ray tracing to speed graphics workloads. Upload your partially rendered image and AI Renderer renders the rest for you. By predicting final images from only partly finished results, Iray AI produces accurate, photorealistic models without having to wait for the final image to be rendered.
124. NVIDIA - Iray
125. NVIDIA - Iray
126. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Autodesk - DreamCatcher Can an algorithm generate product design that solves complex problems?
127. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Autodesk - DreamCatcher Dreamcatcher is a generative CAD system that enables designers to craft a definition of their design problem through goals and constraints.
128. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Autodesk - DreamCatcher The three structural elements shown are all designed to carry the same structural loads and forces.
129. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Autodesk - DreamCatcher A Lightweight bike stem generated by an algorithm
130. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Autodesk - DreamCatcher This is a car frame that is designed by a generative algorithm
131. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Autodesk - DreamCatcher A a lightweight load-bearing engine block
132. Autodesk - DreamCatcher
133. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Objectifier Can an algorithm help objects to respond to human gestures?
134. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Objectifier Objectifier empowers people to train objects in their daily environment to respond to their unique behaviors. It gives an experience of training an artificial intelligence; much like training a dog - you teach it only what you want it to care about. Just like a dog, it sees and understands its environment.
135. Objectifier H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S
136. Objectifier H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S
137. Objectifier H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S
138. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google Inbox - Smart Reply Can an algorithm suggest email replies?
139. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Google Inbox - Smart Reply When you're checking email on the go, it can be cumbersome and time-consuming to reply to all or even some of them. What if we can save mobile users the hassle of tapping out answers on those tiny keyboards?
140. Google Inbox - Smart Reply
141. Google Inbox - Smart Reply
142. „ H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S I DIDN’T LEARN TO TALK FROM A LINGUIST, I LEARNED TO TALK FROM HEARING OTHER PEOPLE TALK Greg Corrado, Co-founder of Google Brain
143. What is next? © Lawrence Lek
144. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S What’s next? Computers can now see, hear, and translate languages with unprecedented accuracies. They are also learning to generate images, sound, and text.
145. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S What’s next? However the systems we’re building still fall into the category of “Narrow AI” — they can achieve super- human performance in a specific domain, but lack the ability to do anything sensible outside of it.
146. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S What’s next? However the systems we’re building still fall into the category of “Narrow AI” — they can achieve super-human performance in a specific domain, but lack the ability to do anything sensible outside of it. © Jan Korsanke
147. H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S Open AI - UNIVERSE
148. „ H O W D E E P L E A R N I N G C H A N G E S T H E D E S I G N P R O C E S S MACHINE LEARNING IS GOING TO UPEND YOUR INDUSTRY AND YOUR PRODUCT. Ken Norton, Google Ventures
149. Alexander Meinhardt krunchtime.org KRUN C H T I M E I NT E R ACT I V E