So you can sleep at night… ethics in software development (May 2019)

So you can sleep at night… ethics in software development (May 2019)

Given at Agile Manchester on 8 May 2019.

As software teams in a post-truth world, what are the moral implications of what we do when software doesn’t have to be malicious to be dangerous? None of what we build exists in a vacuum. To make ethical decisions, we must understand the implications of what we’re building. We must be well-informed, aware of the risks, brave, humble, willing to speak up and to listen. Come along and think about the ethical implications of the software we build.

Dd62e0ded3420cb332cc7d87505bc9c5?s=128

Jonathan Rothwell

May 08, 2019
Tweet

Transcript

  1. So you can sleep at night… ethics in software development

    Jonathan Rothwell @jorodev (he/him) © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight
  2. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com

  3. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com

  4. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com

  5. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com

  6. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com

  7. #SleepAtNight @jorodev © Zuhlke 2019 3dman_eu / pixabay.com

  8. #SleepAtNight @jorodev © Zuhlke 2019 Photo © European Parliament

  9. #SleepAtNight @jorodev © Zuhlke 2019 –Greta Thunberg Adults keep saying:

    “We owe it to the young people to give them hope.” But I don’t want your hope. I don’t want you to be hopeful. I want you to panic. I want you to feel the fear I feel every day. I want you to act. I want you to act as you would in a crisis. I want you to act as if the house is on fire, because it is. Photo © European Parliament
  10. #SleepAtNight @jorodev © Zuhlke 2019

  11. CONTAINS: SCARY STORIES, SPECULATIONS FREE FROM: CONCLUSIONS PRODUCED IN A

    FACILITY CONTAINING NUTS ⚠
  12. #SleepAtNight @jorodev © Zuhlke 2019 –Maciej Cegłowski, Build a Better

    Monster April 2017 http://idlewords.com/talks/build_a_better_monster.htm This year especially there’s an uncomfortable feeling in the tech industry that we did something wrong, that in following our credo of “move fast and break things”, some of what we knocked down were the load-bearing walls of our democracy.
  13. #SleepAtNight @jorodev © Zuhlke 2019 “I’m just an engineer,” he

    said.
  14. #SleepAtNight @jorodev © Zuhlke 2019 “I’m just an engineer,” he

    said.
  15. #SleepAtNight @jorodev © Zuhlke 2019 No software exists in a

    vacuum. Software has context.
  16. #SleepAtNight @jorodev © Zuhlke 2019

  17. #SleepAtNight @jorodev © Zuhlke 2019 ACM Code of Ethics and

    Professional Conduct Affi rming our obligation to use our skills to benefi t society Code Code Code The Code • General Ethical Principles • Professional Responsibilities • Professional Leadership Principles • Compliance with the Code • Case Studies • Using the Code c1_ACM_Code0830.indd 1 8/30/18 10:44 AM We do have codes of conduct…
  18. #SleepAtNight @jorodev © Zuhlke 2019 It’s impossible to get struck

    off 
 as a software developer.
  19. #SleepAtNight @jorodev © Zuhlke 2019 Buddhism’s “Three Poisons” Ignorance Attachment

    Hatred CC https://en.wikipedia.org/wiki/File:Bhavacakra_Thikse.jpg
  20. #SleepAtNight @jorodev © Zuhlke 2019 གཏི་muག Ignorance

  21. #SleepAtNight @jorodev © Zuhlke 2019

  22. #SleepAtNight @jorodev © Zuhlke 2019

  23. #SleepAtNight @jorodev © Zuhlke 2019

  24. #SleepAtNight @jorodev © Zuhlke 2019

  25. #SleepAtNight @jorodev © Zuhlke 2019

  26. #SleepAtNight @jorodev © Zuhlke 2019 “computer algorithm failure”

  27. #SleepAtNight @jorodev © Zuhlke 2019 Bad software ruins people’s lives.

  28. #SleepAtNight @jorodev © Zuhlke 2019 འདོད་ཆགས་ Attachment

  29. #SleepAtNight @jorodev © Zuhlke 2019 Wouldn’t it be cool if…

  30. #SleepAtNight @jorodev © Zuhlke 2019

  31. #SleepAtNight @jorodev © Zuhlke 2019

  32. #SleepAtNight @jorodev © Zuhlke 2019

  33. #SleepAtNight @jorodev © Zuhlke 2019

  34. #SleepAtNight @jorodev © Zuhlke 2019 “Oh, but that’s not us.


    We’re just doing …”
  35. #SleepAtNight @jorodev © Zuhlke 2019

  36. #SleepAtNight @jorodev © Zuhlke 2019

  37. #SleepAtNight @jorodev © Zuhlke 2019

  38. #SleepAtNight @jorodev © Zuhlke 2019 Censored!

  39. #SleepAtNight @jorodev © Zuhlke 2019

  40. #SleepAtNight @jorodev © Zuhlke 2019

  41. #SleepAtNight @jorodev © Zuhlke 2019

  42. #SleepAtNight @jorodev © Zuhlke 2019

  43. #SleepAtNight @jorodev © Zuhlke 2019 only works if you are

    ✅ white ✅ cisgender
  44. #SleepAtNight @jorodev © Zuhlke 2019

  45. #SleepAtNight @jorodev © Zuhlke 2019

  46. #SleepAtNight @jorodev © Zuhlke 2019

  47. #SleepAtNight @jorodev © Zuhlke 2019

  48. #SleepAtNight @jorodev © Zuhlke 2019 algorithm
 n. a process or

    set of rules to be followed in calculations or other problem-solving operations, especially by a computer.
  49. #SleepAtNight @jorodev © Zuhlke 2019 “the ALGORITHM”

  50. #SleepAtNight @jorodev © Zuhlke 2019 “the ALGORITHM” NEUTRAL or Not

    Neutral ?
  51. #SleepAtNight @jorodev © Zuhlke 2019

  52. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT

    IF name longer than 20 characters THEN name is not valid ELSE name is valid
  53. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT

    IF name longer than 20 characters THEN name is not valid ELSE name is valid NEUTRAL or Not Neutral ?
  54. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT

    IF name longer than 20 characters THEN name is not valid ELSE name is valid NEUTRAL or Not Neutral ?
  55. #SleepAtNight @jorodev © Zuhlke 2019

  56. #SleepAtNight @jorodev © Zuhlke 2019

  57. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT

    PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs.
 passport biometrics
  58. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT

    PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs.
 passport biometrics NEUTRAL or Not Neutral ?
  59. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT

    PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs.
 passport biometrics NEUTRAL or Not Neutral ?
  60. #SleepAtNight @jorodev © Zuhlke 2019 Photo credit: Emma Obanye @eobanye

  61. #SleepAtNight @jorodev © Zuhlke 2019

  62. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A

    WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging
  63. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A

    WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging NEUTRAL or Not Neutral ?
  64. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A

    WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging NEUTRAL or Not Neutral ?
  65. #SleepAtNight @jorodev © Zuhlke 2019

  66. #SleepAtNight @jorodev © Zuhlke 2019 “the ALGORITHM”

  67. #SleepAtNight @jorodev © Zuhlke 2019 “the ALGORITHM”

  68. #SleepAtNight @jorodev © Zuhlke 2019

  69. #SleepAtNight @jorodev © Zuhlke 2019 Garbage in, garbage out

  70. #SleepAtNight @jorodev © Zuhlke 2019

  71. #SleepAtNight @jorodev © Zuhlke 2019

  72. #SleepAtNight @jorodev © Zuhlke 2019

  73. #SleepAtNight @jorodev © Zuhlke 2019 Why did we make technology

    that will live for 18 months, die, and never rot?
  74. #SleepAtNight @jorodev © Zuhlke 2019 Image: Marlenenapoli / Wikimedia Commons

  75. #SleepAtNight @jorodev © Zuhlke 2019

  76. #SleepAtNight @jorodev © Zuhlke 2019 – Maciej Cegłowski, The Moral

    Economy of Tech Approaching the world as a software problem is a category error that has led us into some terrible habits of mind.
  77. #SleepAtNight @jorodev © Zuhlke 2019

  78. #SleepAtNight @jorodev © Zuhlke 2019

  79. #SleepAtNight @jorodev © Zuhlke 2019 ཞེ་sdང Hatred

  80. #SleepAtNight @jorodev © Zuhlke 2019

  81. #SleepAtNight @jorodev © Zuhlke 2019 Some people do 
 consciously

    bad things.
  82. #SleepAtNight @jorodev © Zuhlke 2019

  83. #SleepAtNight @jorodev © Zuhlke 2019 But... Most people don’t believe

    they’re ‘bad people.’
  84. #SleepAtNight @jorodev © Zuhlke 2019

  85. #SleepAtNight @jorodev © Zuhlke 2019

  86. #SleepAtNight @jorodev © Zuhlke 2019 “The company defended itself by

    explaining that “people have to expressly agree to use” the opt-in feature for importing their contacts on Messenger or Facebook Lite to allow Facebook to collect the cell phone data, but Sunday’s statement is the first time the social media company actually spelled out that practice in clear terms for users.”
  87. #SleepAtNight @jorodev © Zuhlke 2019

  88. #SleepAtNight @jorodev © Zuhlke 2019 What about people who 


    aren’t our users and customers, 
 but are still affected by our software? “As an <X>, I want <Y>, so that <Z>”
  89. #SleepAtNight @jorodev © Zuhlke 2019

  90. #SleepAtNight @jorodev © Zuhlke 2019 –Jacob Bronowski, The Ascent of

    Man, BBC “This is where people were turned into numbers. Into this pond were flushed the ashes of some four million people. And that was not done by gas. It was done by arrogance, it was done by dogma, it was done by ignorance.”
  91. #SleepAtNight @jorodev © Zuhlke 2019

  92. #SleepAtNight @jorodev © Zuhlke 2019

  93. #SleepAtNight @jorodev © Zuhlke 2019

  94. #SleepAtNight @jorodev © Zuhlke 2019

  95. #SleepAtNight @jorodev © Zuhlke 2019

  96. #SleepAtNight @jorodev © Zuhlke 2019

  97. #SleepAtNight @jorodev © Zuhlke 2019

  98. #SleepAtNight @jorodev © Zuhlke 2019

  99. #SleepAtNight @jorodev © Zuhlke 2019

  100. #SleepAtNight @jorodev © Zuhlke 2019 –Jacob Bronowski, The Ascent of

    Man, BBC “This is where people were turned into numbers. Into this pond were flushed the ashes of some four million people. And that was not done by gas. It was done by arrogance, it was done by dogma, it was done by ignorance.”
  101. None
  102. #SleepAtNight @jorodev © Zuhlke 2019 What can we do?

  103. #SleepAtNight @jorodev © Zuhlke 2019 Buddhism’s “Three Poisons” Ignorance Attachment

    Hatred CC https://en.wikipedia.org/wiki/File:Bhavacakra_Thikse.jpg
  104. #SleepAtNight @jorodev © Zuhlke 2019 གཏི་muག་མེད་པ། Non-Delusion

  105. #SleepAtNight @jorodev © Zuhlke 2019

  106. #SleepAtNight @jorodev © Zuhlke 2019

  107. #SleepAtNight @jorodev © Zuhlke 2019 We could all do with

    examining how the system unfairly benefits us personally… An uncomfortable truth dawned on me as I lugged my bike up and down flights of stairs in commuter-town train stations […] no ramps, no lifts. Before I’d had my own wheels to carry, I’d never noticed this problem. -Reni Eddo-Lodge Why I’m No Longer Talking To White People About Race
  108. #SleepAtNight @jorodev © Zuhlke 2019

  109. #SleepAtNight @jorodev © Zuhlke 2019 By UKBlackTech - https://ukblacktech.com/stockphotos/

  110. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team By

    UKBlackTech - https://ukblacktech.com/stockphotos/
  111. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND


    foster an inclusive environment By UKBlackTech - https://ukblacktech.com/stockphotos/
  112. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND


    foster an inclusive environment AND
 check your privilege By UKBlackTech - https://ukblacktech.com/stockphotos/
  113. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND


    foster an inclusive environment AND
 check your privilege AND
 use it to elevate marginalised voices By UKBlackTech - https://ukblacktech.com/stockphotos/
  114. #SleepAtNight @jorodev © Zuhlke 2019 མ་ཆགས་པ། Non-Attachment

  115. #SleepAtNight @jorodev © Zuhlke 2019

  116. #SleepAtNight @jorodev © Zuhlke 2019 shiny

  117. #SleepAtNight @jorodev © Zuhlke 2019

  118. #SleepAtNight @jorodev © Zuhlke 2019

  119. #SleepAtNight @jorodev © Zuhlke 2019 When things in real life

    that sound like satire start to happen, it's way past the time where we need to start reflecting on how we are fostering a society built on hate [...] We cannot let people advance tech for their own selfish reasons while they pretend that they're doing it for everyone. — Jenn Schiffer
  120. #SleepAtNight @jorodev © Zuhlke 2019 Just because you can, it

    doesn’t mean you should
  121. #SleepAtNight @jorodev © Zuhlke 2019 ཞེས་sdང་མེད་པ། Non-Hatred

  122. #SleepAtNight @jorodev © Zuhlke 2019

  123. #SleepAtNight @jorodev © Zuhlke 2019 Try to be nice… Always

    be kind.
  124. #SleepAtNight @jorodev © Zuhlke 2019

  125. #SleepAtNight @jorodev © Zuhlke 2019

  126. #SleepAtNight @jorodev © Zuhlke 2019 The one effective lever we

    have against tech companies is employee pressure […] Unfortunately, the enemy is complacency. […] A workplace free of 'politics' is just one of the many perks the tech industry offers its pampered employees. – Maciej Cegłowski
  127. #SleepAtNight @jorodev © Zuhlke 2019 Should we use our position

    to • resist handing control to a few dominant players? • break down, rather than entrench, social barriers and injustice?
  128. #SleepAtNight @jorodev © Zuhlke 2019 But… who do we think

    we are to make this call?
  129. #SleepAtNight @jorodev © Zuhlke 2019 But… who else is there?

  130. #SleepAtNight @jorodev © Zuhlke 2019 Single In a Relationship Engaged

    Married In a civil partnership In an open relationship ✓It’s complicated Separated Divorced Widowed
  131. #SleepAtNight @jorodev © Zuhlke 2019 Non-Delusion •Be humble. It’s OK

    to be wrong •Think about your concerns. Put them in writing.
 Make a conscious decision to do things •Have a life outside tech
  132. #SleepAtNight @jorodev © Zuhlke 2019 Non-Attachment •Try not to be

    seduced by that framework, buzzword etc. •Think about hostile users, not just attackers •Be brave—make a conscious decision not to do things
  133. #SleepAtNight @jorodev © Zuhlke 2019 Non-Hatred •Have a visible ethical

    charter—and back it up •Say no, or leave •Try to be nice—but always be kind to people
  134. #SleepAtNight @jorodev © Zuhlke 2019 Be humble. Be brave. Be

    kind.
  135. So you can sleep at night written and presented by

    Jonathan Rothwell @jorodev co-written with Steve Freeman @sf105 © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight
  136. #SleepAtNight @jorodev © Zuhlke 2019 Responsibly Smashing Pandora’s Box by

    Yanqing Cheng @YanqingCheng https://www.infoq.com/ presentations/philosophy- personal-role Consequences of an Insightful Algorithm by Carina C. Zona @cczona https://www.youtube.com/ watch?v=YRI40A4tyWU Jenn Schiffer @jennschiffer at XOXO
 https://www.youtube.com/ watch?v=DrCD1wmK9oM Meeting Resistance and Moving Forward by Linda Rising @RisingLinda https://www.youtube.com/ watch?v=DrCD1wmK9oM
  137. #SleepAtNight @jorodev © Zuhlke 2019 Links •http://idlewords.com/talks/notes_from_an_emergency.htm •http://idlewords.com/talks/sase_panel.htm •http://idlewords.com/talks/build_a_better_monster.htm •https://www.schneier.com/blog/archives/2013/10/the_battle_for_1.html

    •https://www.theguardian.com/technology/2017/may/07/the-great-british-brexit-robbery- hijacked-democracy •http://www-formal.stanford.edu/jmc/little/little.html •https://medium.com/s/story/data-violence-and-how-bad-engineering-choices-can-damage- society-39e44150e1d4
  138. #SleepAtNight @jorodev © Zuhlke 2019 •Better - A Surgeon’s Notes

    On Performance
 written by Atul Gawande •IBM and the Holocaust: The Strategic Alliance between Nazi Germany and America's Most Powerful Corporation
 written by Edwin Black •Why I’m No Longer Talking To White People About Race
 written by Reni Eddo-Lodge Recommended Reading
  139. #SleepAtNight @jorodev © Zuhlke 2019 With thanks to: • Dr

    Sal Freudenburg • Emma Obanye • Katherine Kirk • Kate Gray • Sebastiaan Peters • James Thomson • Ruth Yakubu for their help and encouragement • J. Rosenbaum • Dan Hon • Ivan Moore • Asad Razvi • Adrian Kosmaczewski • Paul Hudson • Friends and colleagues at Zuhlke
  140. So you can sleep at night written and presented by

    Jonathan Rothwell @jorodev co-written with Steve Freeman @sf105 Thank you for coming ✨ © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight