So you can sleep at night… ethics in software development (May 2019)

Given at Agile Manchester on 8 May 2019.

As software teams in a post-truth world, what are the moral implications of what we do when software doesn’t have to be malicious to be dangerous? None of what we build exists in a vacuum. To make ethical decisions, we must understand the implications of what we’re building. We must be well-informed, aware of the risks, brave, humble, willing to speak up and to listen. Come along and think about the ethical implications of the software we build.

Transcript

1. So you can sleep at night… ethics in software development Jonathan Rothwell @jorodev (he/him) © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight
2. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com
3. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com
4. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com
5. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com
6. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com
7. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com
8. #SleepAtNight @jorodev © Zuhlke 2019 Photo © European Parliament
9. #SleepAtNight @jorodev © Zuhlke 2019 –Greta Thunberg Adults keep saying: “We owe it to the young people to give them hope.” But I don’t want your hope. I don’t want you to be hopeful. I want you to panic. I want you to feel the fear I feel every day. I want you to act. I want you to act as you would in a crisis. I want you to act as if the house is on fire, because it is. Photo © European Parliament
10. #SleepAtNight @jorodev © Zuhlke 2019
11. CONTAINS: SCARY STORIES, SPECULATIONS FREE FROM: CONCLUSIONS PRODUCED IN A FACILITY CONTAINING NUTS ⚠
12. #SleepAtNight @jorodev © Zuhlke 2019 –Maciej Cegłowski, Build a Better Monster April 2017 http://idlewords.com/talks/build_a_better_monster.htm This year especially there’s an uncomfortable feeling in the tech industry that we did something wrong, that in following our credo of “move fast and break things”, some of what we knocked down were the load-bearing walls of our democracy.
13. #SleepAtNight @jorodev © Zuhlke 2019 “I’m just an engineer,” he said.
14. #SleepAtNight @jorodev © Zuhlke 2019 “I’m just an engineer,” he said.
15. #SleepAtNight @jorodev © Zuhlke 2019 No software exists in a vacuum. Software has context.
16. #SleepAtNight @jorodev © Zuhlke 2019
17. #SleepAtNight @jorodev © Zuhlke 2019 ACM Code of Ethics and Professional Conduct Affi rming our obligation to use our skills to benefi t society Code Code Code The Code • General Ethical Principles • Professional Responsibilities • Professional Leadership Principles • Compliance with the Code • Case Studies • Using the Code c1_ACM_Code0830.indd 1 8/30/18 10:44 AM We do have codes of conduct…
18. #SleepAtNight @jorodev © Zuhlke 2019 It’s impossible to get struck off 
 as a software developer.
19. #SleepAtNight @jorodev © Zuhlke 2019 Buddhism’s “Three Poisons” Ignorance Attachment Hatred CC https://en.wikipedia.org/wiki/File:Bhavacakra_Thikse.jpg
20. #SleepAtNight @jorodev © Zuhlke 2019 གཏི་muག Ignorance
21. #SleepAtNight @jorodev © Zuhlke 2019
22. #SleepAtNight @jorodev © Zuhlke 2019
23. #SleepAtNight @jorodev © Zuhlke 2019
24. #SleepAtNight @jorodev © Zuhlke 2019
25. #SleepAtNight @jorodev © Zuhlke 2019
26. #SleepAtNight @jorodev © Zuhlke 2019 “computer algorithm failure”
27. #SleepAtNight @jorodev © Zuhlke 2019 Bad software ruins people’s lives.
28. #SleepAtNight @jorodev © Zuhlke 2019 འདོད་ཆགས་ Attachment
29. #SleepAtNight @jorodev © Zuhlke 2019 Wouldn’t it be cool if…
30. #SleepAtNight @jorodev © Zuhlke 2019
31. #SleepAtNight @jorodev © Zuhlke 2019
32. #SleepAtNight @jorodev © Zuhlke 2019
33. #SleepAtNight @jorodev © Zuhlke 2019
34. #SleepAtNight @jorodev © Zuhlke 2019 “Oh, but that’s not us.
 We’re just doing …”
35. #SleepAtNight @jorodev © Zuhlke 2019
36. #SleepAtNight @jorodev © Zuhlke 2019
37. #SleepAtNight @jorodev © Zuhlke 2019
38. #SleepAtNight @jorodev © Zuhlke 2019 Censored!
39. #SleepAtNight @jorodev © Zuhlke 2019
40. #SleepAtNight @jorodev © Zuhlke 2019
41. #SleepAtNight @jorodev © Zuhlke 2019
42. #SleepAtNight @jorodev © Zuhlke 2019
43. #SleepAtNight @jorodev © Zuhlke 2019 only works if you are ✅ white ✅ cisgender
44. #SleepAtNight @jorodev © Zuhlke 2019
45. #SleepAtNight @jorodev © Zuhlke 2019
46. #SleepAtNight @jorodev © Zuhlke 2019
47. #SleepAtNight @jorodev © Zuhlke 2019
48. #SleepAtNight @jorodev © Zuhlke 2019 algorithm
 n. a process or set of rules to be followed in calculations or other problem-solving operations, especially by a computer.
49. #SleepAtNight @jorodev © Zuhlke 2019 “the ALGORITHM”
50. #SleepAtNight @jorodev © Zuhlke 2019 “the ALGORITHM” NEUTRAL or Not Neutral ?
51. #SleepAtNight @jorodev © Zuhlke 2019
52. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT IF name longer than 20 characters THEN name is not valid ELSE name is valid
53. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT IF name longer than 20 characters THEN name is not valid ELSE name is valid NEUTRAL or Not Neutral ?
54. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT IF name longer than 20 characters THEN name is not valid ELSE name is valid NEUTRAL or Not Neutral ?
55. #SleepAtNight @jorodev © Zuhlke 2019
56. #SleepAtNight @jorodev © Zuhlke 2019
57. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs.
 passport biometrics
58. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs.
 passport biometrics NEUTRAL or Not Neutral ?
59. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs.
 passport biometrics NEUTRAL or Not Neutral ?
60. #SleepAtNight @jorodev © Zuhlke 2019 Photo credit: Emma Obanye @eobanye
61. #SleepAtNight @jorodev © Zuhlke 2019
62. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging
63. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging NEUTRAL or Not Neutral ?
64. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging NEUTRAL or Not Neutral ?
65. #SleepAtNight @jorodev © Zuhlke 2019
66. #SleepAtNight @jorodev © Zuhlke 2019 “the ALGORITHM”
67. #SleepAtNight @jorodev © Zuhlke 2019 “the ALGORITHM”
68. #SleepAtNight @jorodev © Zuhlke 2019
69. #SleepAtNight @jorodev © Zuhlke 2019 Garbage in, garbage out
70. #SleepAtNight @jorodev © Zuhlke 2019
71. #SleepAtNight @jorodev © Zuhlke 2019
72. #SleepAtNight @jorodev © Zuhlke 2019
73. #SleepAtNight @jorodev © Zuhlke 2019 Why did we make technology that will live for 18 months, die, and never rot?
74. #SleepAtNight @jorodev © Zuhlke 2019 Image: Marlenenapoli / Wikimedia Commons
75. #SleepAtNight @jorodev © Zuhlke 2019
76. #SleepAtNight @jorodev © Zuhlke 2019 – Maciej Cegłowski, The Moral Economy of Tech Approaching the world as a software problem is a category error that has led us into some terrible habits of mind.
77. #SleepAtNight @jorodev © Zuhlke 2019
78. #SleepAtNight @jorodev © Zuhlke 2019
79. #SleepAtNight @jorodev © Zuhlke 2019 ཞེ་sdང Hatred
80. #SleepAtNight @jorodev © Zuhlke 2019
81. #SleepAtNight @jorodev © Zuhlke 2019 Some people do 
 consciously bad things.
82. #SleepAtNight @jorodev © Zuhlke 2019
83. #SleepAtNight @jorodev © Zuhlke 2019 But... Most people don’t believe they’re ‘bad people.’
84. #SleepAtNight @jorodev © Zuhlke 2019
85. #SleepAtNight @jorodev © Zuhlke 2019
86. #SleepAtNight @jorodev © Zuhlke 2019 “The company defended itself by explaining that “people have to expressly agree to use” the opt-in feature for importing their contacts on Messenger or Facebook Lite to allow Facebook to collect the cell phone data, but Sunday’s statement is the first time the social media company actually spelled out that practice in clear terms for users.”
87. #SleepAtNight @jorodev © Zuhlke 2019
88. #SleepAtNight @jorodev © Zuhlke 2019 What about people who 
 aren’t our users and customers, 
 but are still affected by our software? “As an <X>, I want <Y>, so that <Z>”
89. #SleepAtNight @jorodev © Zuhlke 2019
90. #SleepAtNight @jorodev © Zuhlke 2019 –Jacob Bronowski, The Ascent of Man, BBC “This is where people were turned into numbers. Into this pond were flushed the ashes of some four million people. And that was not done by gas. It was done by arrogance, it was done by dogma, it was done by ignorance.”
91. #SleepAtNight @jorodev © Zuhlke 2019
92. #SleepAtNight @jorodev © Zuhlke 2019
93. #SleepAtNight @jorodev © Zuhlke 2019
94. #SleepAtNight @jorodev © Zuhlke 2019
95. #SleepAtNight @jorodev © Zuhlke 2019
96. #SleepAtNight @jorodev © Zuhlke 2019
97. #SleepAtNight @jorodev © Zuhlke 2019
98. #SleepAtNight @jorodev © Zuhlke 2019
99. #SleepAtNight @jorodev © Zuhlke 2019
100. #SleepAtNight @jorodev © Zuhlke 2019 –Jacob Bronowski, The Ascent of Man, BBC “This is where people were turned into numbers. Into this pond were flushed the ashes of some four million people. And that was not done by gas. It was done by arrogance, it was done by dogma, it was done by ignorance.”
102. #SleepAtNight @jorodev © Zuhlke 2019 What can we do?
103. #SleepAtNight @jorodev © Zuhlke 2019 Buddhism’s “Three Poisons” Ignorance Attachment Hatred CC https://en.wikipedia.org/wiki/File:Bhavacakra_Thikse.jpg
104. #SleepAtNight @jorodev © Zuhlke 2019 གཏི་muག་མེད་པ། Non-Delusion
105. #SleepAtNight @jorodev © Zuhlke 2019
106. #SleepAtNight @jorodev © Zuhlke 2019
107. #SleepAtNight @jorodev © Zuhlke 2019 We could all do with examining how the system unfairly benefits us personally… An uncomfortable truth dawned on me as I lugged my bike up and down flights of stairs in commuter-town train stations […] no ramps, no lifts. Before I’d had my own wheels to carry, I’d never noticed this problem. -Reni Eddo-Lodge Why I’m No Longer Talking To White People About Race
108. #SleepAtNight @jorodev © Zuhlke 2019
109. #SleepAtNight @jorodev © Zuhlke 2019 By UKBlackTech - https://ukblacktech.com/stockphotos/
110. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team By UKBlackTech - https://ukblacktech.com/stockphotos/
111. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND
 foster an inclusive environment By UKBlackTech - https://ukblacktech.com/stockphotos/
112. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND
 foster an inclusive environment AND
 check your privilege By UKBlackTech - https://ukblacktech.com/stockphotos/
113. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND
 foster an inclusive environment AND
 check your privilege AND
 use it to elevate marginalised voices By UKBlackTech - https://ukblacktech.com/stockphotos/
114. #SleepAtNight @jorodev © Zuhlke 2019 མ་ཆགས་པ། Non-Attachment
115. #SleepAtNight @jorodev © Zuhlke 2019
116. #SleepAtNight @jorodev © Zuhlke 2019 shiny
117. #SleepAtNight @jorodev © Zuhlke 2019
118. #SleepAtNight @jorodev © Zuhlke 2019
119. #SleepAtNight @jorodev © Zuhlke 2019 When things in real life that sound like satire start to happen, it's way past the time where we need to start reflecting on how we are fostering a society built on hate [...] We cannot let people advance tech for their own selfish reasons while they pretend that they're doing it for everyone. — Jenn Schiffer
120. #SleepAtNight @jorodev © Zuhlke 2019 Just because you can, it doesn’t mean you should
121. #SleepAtNight @jorodev © Zuhlke 2019 ཞེས་sdང་མེད་པ། Non-Hatred
122. #SleepAtNight @jorodev © Zuhlke 2019
123. #SleepAtNight @jorodev © Zuhlke 2019 Try to be nice… Always be kind.
124. #SleepAtNight @jorodev © Zuhlke 2019
125. #SleepAtNight @jorodev © Zuhlke 2019
126. #SleepAtNight @jorodev © Zuhlke 2019 The one effective lever we have against tech companies is employee pressure […] Unfortunately, the enemy is complacency. […] A workplace free of 'politics' is just one of the many perks the tech industry offers its pampered employees. – Maciej Cegłowski
127. #SleepAtNight @jorodev © Zuhlke 2019 Should we use our position to • resist handing control to a few dominant players? • break down, rather than entrench, social barriers and injustice?
128. #SleepAtNight @jorodev © Zuhlke 2019 But… who do we think we are to make this call?
129. #SleepAtNight @jorodev © Zuhlke 2019 But… who else is there?
130. #SleepAtNight @jorodev © Zuhlke 2019 Single In a Relationship Engaged Married In a civil partnership In an open relationship ✓It’s complicated Separated Divorced Widowed
131. #SleepAtNight @jorodev © Zuhlke 2019 Non-Delusion •Be humble. It’s OK to be wrong •Think about your concerns. Put them in writing.
 Make a conscious decision to do things •Have a life outside tech
132. #SleepAtNight @jorodev © Zuhlke 2019 Non-Attachment •Try not to be seduced by that framework, buzzword etc. •Think about hostile users, not just attackers •Be brave—make a conscious decision not to do things
133. #SleepAtNight @jorodev © Zuhlke 2019 Non-Hatred •Have a visible ethical charter—and back it up •Say no, or leave •Try to be nice—but always be kind to people
134. #SleepAtNight @jorodev © Zuhlke 2019 Be humble. Be brave. Be kind.
135. So you can sleep at night written and presented by Jonathan Rothwell @jorodev co-written with Steve Freeman @sf105 © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight
136. #SleepAtNight @jorodev © Zuhlke 2019 Responsibly Smashing Pandora’s Box by Yanqing Cheng @YanqingCheng https://www.infoq.com/ presentations/philosophy- personal-role Consequences of an Insightful Algorithm by Carina C. Zona @cczona https://www.youtube.com/ watch?v=YRI40A4tyWU Jenn Schiffer @jennschiffer at XOXO
 https://www.youtube.com/ watch?v=DrCD1wmK9oM Meeting Resistance and Moving Forward by Linda Rising @RisingLinda https://www.youtube.com/ watch?v=DrCD1wmK9oM
137. #SleepAtNight @jorodev © Zuhlke 2019 Links •http://idlewords.com/talks/notes_from_an_emergency.htm •http://idlewords.com/talks/sase_panel.htm •http://idlewords.com/talks/build_a_better_monster.htm •https://www.schneier.com/blog/archives/2013/10/the_battle_for_1.html •https://www.theguardian.com/technology/2017/may/07/the-great-british-brexit-robbery- hijacked-democracy •http://www-formal.stanford.edu/jmc/little/little.html •https://medium.com/s/story/data-violence-and-how-bad-engineering-choices-can-damage- society-39e44150e1d4
138. #SleepAtNight @jorodev © Zuhlke 2019 •Better - A Surgeon’s Notes On Performance
 written by Atul Gawande •IBM and the Holocaust: The Strategic Alliance between Nazi Germany and America's Most Powerful Corporation
 written by Edwin Black •Why I’m No Longer Talking To White People About Race
 written by Reni Eddo-Lodge Recommended Reading
139. #SleepAtNight @jorodev © Zuhlke 2019 With thanks to: • Dr Sal Freudenburg • Emma Obanye • Katherine Kirk • Kate Gray • Sebastiaan Peters • James Thomson • Ruth Yakubu for their help and encouragement • J. Rosenbaum • Dan Hon • Ivan Moore • Asad Razvi • Adrian Kosmaczewski • Paul Hudson • Friends and colleagues at Zuhlke
140. So you can sleep at night written and presented by Jonathan Rothwell @jorodev co-written with Steve Freeman @sf105 Thank you for coming ✨ © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight