So you can sleep at night… ethics in software engineering (Devoxx UK 2018)

Mine and Steve Freeman's slides from Devoxx UK 2018 in London.

In a post-truth world, software doesn’t have to be malicious to be dangerous. For example, in 2016, Facebook replaced its (human) news editors with an algorithm. Within days, conspiracy theories were at the top of the trending feeds, presented as 'news.'

In this talk, we'll discuss the ethical considerations of building software in a world that looks more like an unfolding episode of 'Black Mirror' with every passing week. No software exists in a vacuum. As developers, we have to understand the social implications of our software to help us make ethical decisions about what we get involved in, and how we build software in general. And we have to be brave and speak up when we think something could go wrong. And, if someone else speaks up, to be willing to listen.

In short: we must be humble, we must be watchful, and we must ensure the decisions we make mean we can sleep at night.

Transcript

1. #SleepAtNight @jorodev @sf105 So you can sleep at night… Ethics In Software Engineering Jonathan Rothwell & Steve Freeman @jorodev @sf105 Zuhlke Engineering Ltd. www.zuhlke.com
2. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
3. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Thomas Midgley Jr. • 1922 Nichols Medal • 1925 Longstreth Medal • 1937 Perkin Medal • 1941 Priestley Medal • 1942 Willard Gibbs Award • 1944 President American Chemical Society • 2 Honorary degrees • Over 100 patents • Member of US Academy of Sciences
4. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Thomas Midgley Jr. Inventor of leaded petrol …and CFCs “[had] more impact on the atmosphere than any other single organism in Earth’s history”
 —J. R. McNeill, Georgetown U
5. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 –Maciej Cegłowski, Build a Better Monster April 2017 http://idlewords.com/talks/build_a_better_monster.htm This year especially there’s an uncomfortable feeling in the tech industry that we did something wrong, that in following our credo of “move fast and break things”, some of what we knocked down were the load-bearing walls of our democracy.
6. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
7. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
8. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 “I’m just an engineer.”
9. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
10. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 No software exists in a vacuum. Software has context.
11. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
12. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 We do have codes of conduct
13. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 It’s impossible to get struck off 
 as a software developer
14. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Buddhism’s “Three Poisons” Ignorance Attachment Aversion CC https://en.wikipedia.org/wiki/File:Bhavacakra_Thikse.jpg
15. CONTAINS: SCARY STORIES, SPECULATIONS FREE FROM: CONCLUSIONS PRODUCED IN A FACILITY CONTAINING NUTS ⚠
16. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 གཏི་muག Ignorance
17. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
18. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
19. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Bad software ruins people’s lives.
20. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 འདོད་ཆགས་ Attachment
21. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Wouldn’t it be cool if…
22. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
23. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
24. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 “Oh, but that’s not us.
 We’re just doing …”
25. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
26. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Censored!
27. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
28. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
29. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
30. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
31. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
32. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
33. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
34. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
35. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 – Maciej Cegłowski, The Moral Economy of Tech Approaching the world as a software problem is a category error that has led us into some terrible habits of mind.
36. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
37. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 ཞེ་sdང Aversion
38. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Some people do 
 consciously bad things.
39. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Most people don’t believe they’re ‘bad people.’ But…
40. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
41. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
42. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
43. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 “The company defended itself by explaining that “people have to expressly agree to use” the opt-in feature for importing their contacts on Messenger or Facebook Lite to allow Facebook to collect the cell phone data, but Sunday’s statement is the first time the social media company actually spelled out that practice in clear terms for users.”
44. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 What about people who 
 aren’t our users and customers, 
 but are still affected by our software? “As an <X>, I want <Y>, so that <Z>”
45. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 6 million Jews 2—3 milllion Soviet POWs 1.8—1.9 million ethnic Poles 150,000 Disabled people 90,000—220,000 Romani 5,000—15,000 Homosexuals 1,400—2,500 Jehovah’s Witnesses Total: 17 million people killed
46. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 –Jacob Bronowski, The Ascent of Man, BBC “This is the concentration camp and crematorium at Auschwitz. This is where people were turned into numbers. Into this pond were flushed the ashes of some four million people. And that was not done by gas. It was done by arrogance, it was done by dogma, it was done by ignorance.”
47. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
48. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 What if someone wanted
 to do that today?
49. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
50. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 “This is the concentration camp and crematorium at Auschwitz. This is where people were turned into numbers. Into this pond were flushed the ashes of some four million people. And that was not done by gas. It was done by arrogance, it was done by dogma, it was done by ignorance.” –Jacob Bronowski, The Ascent of Man, BBC
52. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 What can we do?
53. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Buddhism’s “Three Poisons” Ignorance Attachment Aversion CC https://en.wikipedia.org/wiki/File:Bhavacakra_Thikse.jpg
54. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 གཏི་muག་མེད་པ། Non-Delusion
55. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
56. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
57. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
58. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
59. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 མ་ཆགས་པ། Non-Attachment
60. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Metcalfe’s Law “the value of a telecommunications network is proportional to the square of the number of connected users of the system (n2)”
61. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 SHINY OBJECT SYNDROME
62. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
63. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 When things in real life that sound like satire start to happen, it's way past the time where we need to start reflecting on how we are fostering a society built on hate [...] We cannot let people advance tech for their own selfish reasons while they pretend that they're doing it for everyone. — Jenn Schiffer
64. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 We can’t all have access to young people’s blood. — Jenn Schiffer
65. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Just because you can, it doesn’t mean you should
66. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 ཞེས་sdང་མེད་པ། Non-Hatred
67. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
68. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Be civil to people
69. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
70. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
71. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 The one effective lever we have against tech companies is employee pressure […] Unfortunately, the enemy is complacency. […] A workplace free of 'politics' is just one of the many perks the tech industry offers its pampered employees. – Maciej Cegłowski
72. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
73. #SleepAtNight @jorodev @sf105 © Zuhlke 2018
74. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Should we use our position
 to resist handing control to
 a few dominant players?
75. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 But… who do we think we are to make this call?
76. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 But… who else is there?
77. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Single In a Relationship Engaged Married In a civil partnership In an open relationship ✓It’s complicated Separated Divorced Widowed
78. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Non-Delusion •Be humble •Talk about your concerns •Have a life outside tech
79. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Non-Attachment •Try not to be seduced by that framework, buzzword etc. •Track what people actually do with your system •Think about hostile users, not just attackers
80. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Non-Hatred •Have a visible ethical charter—and back it up •Leave or refuse •Try to be civil to people
81. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 By Oliver Spalt - CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=202801 Make a 
 conscious decision. Don’t drift into a world
 you didn’t choose.
82. So you can sleep at night written and presented by
 Jonathan Rothwell @jorodev Steve Freeman @sf105 © Zuhlke Engineering Ltd. 2017-2018 #SleepAtNight
83. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Responsibly Smashing Pandora’s Box by Yanqing Cheng @YanqingCheng https://www.infoq.com/ presentations/philosophy- personal-role Consequences of an Insightful Algorithm by Carina C. Zona @cczona https://www.youtube.com/ watch?v=YRI40A4tyWU Jenn Schiffer @jennschiffer at XOXO
 https://www.youtube.com/ watch?v=DrCD1wmK9oM Meeting Resistance and Moving Forward by Linda Rising @RisingLinda https://www.youtube.com/ watch?v=DrCD1wmK9oM
84. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 Links •http://idlewords.com/talks/notes_from_an_emergency.htm •http://idlewords.com/talks/sase_panel.htm •http://idlewords.com/talks/build_a_better_monster.htm •https://www.schneier.com/blog/archives/2013/10/the_battle_for_1.html •https://www.theguardian.com/technology/2017/may/07/the-great-british-brexit-robbery- hijacked-democracy •http://www-formal.stanford.edu/jmc/little/little.html •https://medium.com/s/story/data-violence-and-how-bad-engineering-choices-can-damage- society-39e44150e1d4
85. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 •Better - A Surgeon’s Notes On Performance
 written by Atul Gawande •IBM and the Holocaust: The Strategic Alliance between Nazi Germany and America's Most Powerful Corporation
 written by Edwin Black •Why I’m No Longer Talking To White People About Race
 written by Reni Eddo-Lodge Recommended Reading
86. #SleepAtNight @jorodev @sf105 © Zuhlke 2018 With thanks to: • Sal Freudenburg • Katherine Kirk • Kate Gray • Sebastiaan Peters • James Thomson • Ruth Yakubu for their help and encouragement • J. Rosenbaum • Dan Hon • Ivan Moore • Asad Razvi • Adrian Kosmaczewski • Friends and colleagues at Zuhlke
87. So you can sleep at night written and presented by
 Jonathan Rothwell @jorodev Steve Freeman @sf105 © Zuhlke Engineering Ltd. 2017-2018 #SleepAtNight