SCARY STORIES ABOUT AI GONE WRONG

As Artificial Intelligence becomes a bigger part of our lives, it’s time to examine the dangers. It is easy to see the negative impacts of failing to evaluate ethical consequences in retrospect, but often much harder when initially developing intelligent systems. In this talk we will examine three separate examples of the application of AI to various systems. We will look at humorous example(s) where the lack of effective human supervision prevented the system from doing its job altogether, examples where failing to correct early problems led to unintended (and icky) consequences, and effectively human-led AI process(es) where expertise enabled the engineers involved to correct the system before it got off track.

It can be easy to blame artificial intelligence when outcomes are bad, but ultimately we are responsible for guiding these systems according to what is valued. I will provide key questions that we can ask as engineers to avoid nasty surprises: what are the situations where this system will be forced to make hard decisions? What should its priorities be? How can we best validate these results along the way? Attendees will leave with a better understanding of the limitations and strengths of machine learning and how when left unattended intelligent systems can reach unforeseen conclusions.

Transcript

1. @amandasopkin
2. SCARY STORIES OF AI GONE WRONG @amandasopkin
3. Agenda Introduction to AI Funny examples of AI gone wrong Scary examples of AI gone wrong How to NOT go wrong
4. “Artificial intelligence”: is the ability of a computer program or a machine to think and learn
5. “Machine learning”: the use of algorithms to parse data, learn from that data, and make informed decisions based on what it has learned
6. “Deep learning”: the use of an artificial neural network to learn and make decisions
7. AI: why it scares us
8. AI: why it scares us Pop culture
9. AI: why it scares us We worry it’ll replace us
10. AI: why it scares us It’s impersonal
11. AI: Why it’s important...
12. AI: why it’s important It’s where the money is
13. Out of 2,830 startups in Europe that were classified as being AI companies, only 1,580 accurately fit that description, according to the eye-opening stat on page 99 of a new report from MMC, a London-based venture capital firm. In many cases the label, which refers to computer systems that can perform tasks normally requiring human intelligence, was simply wrong.
14. AI: why it’s important It’s where the money is
15. AI: why it’s important It can improve our jobs
16. AI: why it’s important It can improve our lives
17. AI: cardinal rules @amandasopkin
18. Do not set it and forget it!
19. Supervise (heavily)!
20. If it’s not just for fun, don’t play around.
21. AI: surefire recipes for disaster
22. Be in a bubble
23. Fail to realize mistakes
24. Go big right away
25. Content warning: Some of these AI stories will contain content that is racist or sexist
26. Gather round the campfire...
27. AI Flops: tay.ai
28. AI Flops: tay.ai built by "mining relevant public data" and combining that with input from editorial staff, "including improvisational comedians."
30. coolio sup that’s lit
31. coolio sup that’s lit + + =
33. Content warning
35. AI Flops: tay.ai Training data included “relevant public data”
36. Cause of failure: internet trolls
37. coolio sup that’s lit I HATE WOMEN U ARE GARBAGE $@#%&!
38. AI Flops: tay.ai lesson: consider adverse effects of data source
39. Data cleaning: art and science Balance the data Eliminate outliers/irrelevant points General formatting
40. Next...some funny videos
41. AI Flops: GES 2017
42. lesson: Test with real users AI Flops: GES 2017
43. AI Flops: CES 2018
44. Some computer vision flops
45. AI Flops: Google Photos Reddit user MalletsDarker posted three photos taken at a ski resort: two were landscapes, the other shot of his friend
48. In reality...
50. More visual AI issues
51. AI Flops: Wolves. Vs. Dogs Professor Domingos’ wolves vs. dogs system appeared 100% accurate
52. AI Flops: Wolves. Vs. Dogs
53. In reality: System was identifying animals based on snow backgrounds
55. AI Scary Stories: Wolves vs. Dogs lesson: Make sure you are solving the intended problem!
56. Now let’s look at something more serious...
57. AI Flops: Passport analysis
59. In reality...
60. AI Flops: Passport analysis In practice: Nearly 20% failure rate
62. "No hard feelings on my part. I've always had very small eyes and facial recognition technology is relatively new and unsophisticated" - Lee
63. AI Scary Stories: Passport analysis lesson: count on failure!
64. Scary story time
65. AI Scary Stories: Amazon’s resume ingestor
66. AI Scary Stories: Amazon’s resume ingestor Aim: to get top candidates from a pool of resumes
67. Reality: severe gender bias
68. penalized resumes that included the word “women’s,” as in “women’s chess club captain” “downgraded graduates of two all-women’s colleges”
71. AI Scary Stories: Amazon’s resume ingestor lesson: Beware of unintended patterns!
72. Next up...Dr. Robot
73. AI Scary Stories: Watson’s Oncology Advisor $62 million
74. AI Scary Stories: Watson’s Oncology Advisor “Physicians like it. Physicians have said to me, if I took it away now, I’d have a revolt,” Deborah DiSanzo, general manager of IBM Watson Health, June 2017
75. IBM published multiple studies demonstrating that Watson would achieve a high level of “concordance” with the treatment recommendations of oncologists
77. “This product is a piece of s---. We bought it for marketing and with hopes that you would achieve the vision. We can’t use it for most cases.” - Doctor at Jupiter Hospital
78. Watson suggested that doctors give a cancer patient with severe bleeding a drug that could worsen the bleeding.
79. What went wrong?
80. “The cases [were] synthetic.” - IBM Internal, June 2017
82. Concordance studies were “designed in a way that makes negative findings unlikely”
83. AI Scary Stories: Watson’s Oncology Advisor lesson: use real data!
84. Self driving cars
85. AI Scary Stories: Self driving car
86. The Mill Avenue collision, which killed 49-year-old Elaine Herzberg as she walked across the street midblock, was the first fatal crash with a pedestrian and a self-driving car.
87. What went wrong? Mainly human error
88. The future of driverless cars
91. Musts for driverless cars - Plan for failure - Correct early and often - Supervise heavily - Consider the potential for harm @amandasopkin
92. What about when AI is TOO good?
93. AI Scary Stories: Identification
94. AI Scary Stories: Identification
95. Chairwoman Dong Mingzhu identified by jaywalker detection after being detected on a bus ad
100. lesson: Consider the harm
101. Next: racist robots
102. AI Scary Stories: Faceception
103. AI Scary Stories: Faceception “Gilboa envisions governments considering his findings along with other sources to better identify terrorists.”
105. Via computer != Necessarily true
108. AI Scary Stories: Faceception Lesson: Consider the harm
109. Ask yourself: How would this technology be used in a dystopia?
110. Next: more racist robots
111. AI Scary Stories: Beauty.AI
116. AI Scary Stories: Beauty.AI
117. AI Scary Stories: Beauty.AI The machine’s algorithm was supposed to examine facial symmetry and identify wrinkles and blemishes in order to find the contestants who most embodied “human beauty.”
118. In reality...
119. The algorithm didn’t favor women with dark skin. Six thousand people from 100 countries around the world submitted their photos, and 44 winners were later announced -- only one of whom had dark skin.
121. x y
122. AI Scary Stories: Beauty.AI lesson: Use lots of data!
123. IBM’s Million Faces
124. IBM’s Million Faces The first of its kind available to the global research community, DiF provides a dataset of annotations of 1 million human facial images. Using publicly available images from the YFCC-100M Creative Commons data set, we annotated the faces using 10 well-established and independent coding schemes from the scientific literature. The coding schemes principally include objective measures of human faces, such as craniofacial features, as well as more subjective annotations, such as human-labeled predictions of age and gender.
125. Next, racist robots get into government
126. AI Scary Stories: Criminal Justice
127. That’s my kids stuff!
129. 911... I have a crime to report
130. = $80
132. = $86
134. AI Scary Stories: Criminal Justice
135. What really happened 2 years later...
137. “The Justice Department’s National Institute of Corrections now encourages the use of such combined assessments at every stage of the criminal justice process.”
138. AI Scary Stories: Criminal Justice lesson: correct early and often!
139. Let’s talk about social media
140. AI Scary Stories: Social Media Engagement
141. AI Scary Stories: Social Media Engagement “when left unchecked, people will engage disproportionately with more sensationalist and provocative content”
144. A little more insight from youtube:
145. A little more control from youtube:
146. A little more transparency from youtube:
147. A long road to go..
148. So...how do we make it work?
149. Success story: Banking assistant
150. Success story: Banking assistant
153. I need some help supreme overlord
154. Thank you : - ) Now I get it.
155. “she held over 4,000 conversations with 700 users and was able to resolve the majority of those queries independently, allowing employees to get consistent support without delay”, Järborg adds.
156. What went well? - Supervised heavily - Corrected early and often - Real data @amandasopkin
157. Success Story: Collab bots
158. Success story: Collaboration
159. Cobots and humans collaborate to handle different tasks based on workflow.
161. “Everything is just where I need it. I don’t have to lift up the heavy parts,” says Jürgen Heidemann, who has worked at SEW for 40 years, since he was 18. “This is more satisfying because I am making the whole system. I only did one part of the process in the old line.”
162. What went well? - Potential for harm considered - Corrected early and often - Supervised heavily @amandasopkin
163. Success Story: AI in Food
170. “Using AI, the team will be able to optimize its climate recipes for multiple factors, including taste, cost, and sustainability, and create recipes for growing a myriad of crops.”
171. “All of the data is available open-source, along with instructions to build a food computer yourself.”
172. 895% increase in a particular flavor molecule
173. Tradeoff between taste and size: Bigger plant = less delicious
174. What went well? - Open source (easy corrections) - Lots of (real) data - Increasees “experimental thoroughput” @amandasopkin
175. So, what did we learn?
176. Correct early and often
177. Use real* data *as much as possible
178. Use lots of data
179. Count on failure
180. Consider the worst case scenario
181. Our ethical responsibility: Responsible technology
183. ACM Code of Ethics: “Avoid harm to others”
184. Wrapping up...
185. The choice is ours
186. Wrapping up... - Take AI seriously - Be a hover parent to your AI - Humans are often not the best source of truth @amandasopkin
187. @amandasopkin
188. Sources • Flaticon.com • Washington Post • Google images • STAT News