Upgrade to Pro — share decks privately, control downloads, hide ads and more …

So You Can Sleep At Night (NSLondon)

So You Can Sleep At Night (NSLondon)

Jonathan Rothwell

July 18, 2019
Tweet

More Decks by Jonathan Rothwell

Other Decks in Technology

Transcript

  1. So you can sleep at night… ethics in software development

    Jonathan Rothwell (he/him) @jorodev © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight
  2. #SleepAtNight @jorodev © Zuhlke 2019 I don’t want you to

    be hopeful. I want you to panic. 
 –Greta Thunberg Photo © European Parliament
  3. #SleepAtNight @jorodev © Zuhlke 2019 –Maciej Cegłowski, Build a Better

    Monster April 2017 http://idlewords.com/talks/build_a_better_monster.htm …in following our credo of “move fast and break things”, some of what we knocked down were the load-bearing walls of our democracy.
  4. #SleepAtNight @jorodev © Zuhlke 2019 No software exists in a

    vacuum. If software is art, software has context.
  5. #SleepAtNight @jorodev © Zuhlke 2019 ACM Code of Ethics and

    Professional Conduct Affi rming our obligation to use our skills to benefi t society Code Code Code The Code • General Ethical Principles • Professional Responsibilities • Professional Leadership Principles • Compliance with the Code • Case Studies • Using the Code c1_ACM_Code0830.indd 1 8/30/18 10:44 AM We do have codes of conduct…
  6. #SleepAtNight @jorodev © Zuhlke 2019 Does ACM’s Code of Ethics

    Change Ethical Decision Making in Software Development? Andrew McNamara North Carolina State University Raleigh, North Carolina, USA [email protected] Justin Smith North Carolina State University Raleigh, North Carolina, USA [email protected] Emerson Murphy-Hill North Carolina State University Raleigh, North Carolina, USA [email protected] ABSTRACT Ethical decisions in software development can substantially impact end-users, organizations, and our environment, as is evidenced by recent ethics scandals in the news. Organizations, like the ACM, publish codes of ethics to guide software-related ethical decisions. In fact, the ACM has recently demonstrated renewed interest in its code of ethics and made updates for the first time since 1992. To better understand how the ACM code of ethics changes software- related decisions, we replicated a prior behavioral ethics study with 63 software engineering students and 105 professional software developers, measuring their responses to 11 ethical vignettes. We found that explicitly instructing participants to consider the ACM code of ethics in their decision making had no observed effect when compared with a control group. Our findings suggest a challenge to the research community: if not a code of ethics, what techniques can improve ethical decision making in software engineering? CCS CONCEPTS • Social and professional topics ! Codes of ethics; KEYWORDS ACM code of ethics, software engineering ACM Reference Format: Andrew McNamara, Justin Smith, and Emerson Murphy-Hill. 2018. Does ACM’s Code of Ethics Change Ethical Decision Making in Software Develop- ment?. In Proceedings of the 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE ’18), November 4–9, 2018, Lake Buena Vista, FL, USA. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/3236024.3264833 1 INTRODUCTION Software developers must constantly make ethical considerations, including deciding the proper amount of user data to collect; balanc- ing added functionality with potential adverse environmental effects; and performing due diligence to reduce the risks of critical security bugs. Such ethical decisions can cause substantial harm to people, to organizations, and to our planet. Consider two recent examples. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM The first example is the Uber versus Waymo dispute [26], in which a software engineer at Waymo took self-driving car code to his home. Shortly thereafter, the engineer left Waymo to work for a competing company with a self-driving car business, Uber. When Waymo realized that their own code had been taken by their former employee, Waymo sued Uber. Even though the code was not ap- parently used for Uber’s competitive advantage, the two companies settled the lawsuit for $245 million dollars. The second example is the “Dieselgate” scandal [21], where soft- ware inside certain diesel Volkswagen vehicles was programmed to run in one of two modes. In one mode, the car operated under normal, day-to-day driving conditions, but emitted pollution at levels above what is allowed by US and international regulators. In the other mode, the car emitted allowable pollution levels, but only when it detected that it was being inspected by regulators. Although software engineers raised objections to management about the devices, they did not bring these concerns to authorities [19]. Consequently, the company was forced to pay $30 billion dollars in compensation so far [31] and an estimated 59 people suffered early deaths as a result of the excess emitted pollution in the US alone [5]. As early as 1913, organizations have published codes of ethics to guide people facing such ethical situations [3]. In 1972, the Asso- ciation for Computing Machinery (ACM) adopted a code of ethics designed to specifically to apply to software development. In 2018, the ACM code of ethics was updated for the first time since 1992 [2]. In light of recent software ethics scandals, like Dieselgate and the Uber versus Waymo dispute, and ACM’s renewed interest in revising its guidelines, we are motivated to study the effect of ACM’s code of ethics on ethical decision making in software development. While the ACM claims its code of ethics is “intended to serve as a basis for ethical decision making” [1] to our knowledge the effectiveness of this claim has never been tested. We asked 63 software engineering students and 105 professional software engineers to consider 11 software-related ethical decisions. We derived these decisions from real ethical dilemmas faced by software developers. To assess how much the ACM code of ethics in- fluenced decision making, participants were divided into two groups, a control group, and a group explicitly instructed to use the ACM code of ethics. The primary contribution of this paper is a better understanding of ethical decision making in software development and the influence of the ACM code of ethics on those decisions.
  7. #SleepAtNight @jorodev © Zuhlke 2019 end-users, organizations, and our environment,

    as is evidenced by recent ethics scandals in the news. Organizations, like the ACM, publish codes of ethics to guide software-related ethical decisions. In fact, the ACM has recently demonstrated renewed interest in its code of ethics and made updates for the first time since 1992. To better understand how the ACM code of ethics changes software- related decisions, we replicated a prior behavioral ethics study with 63 software engineering students and 105 professional software developers, measuring their responses to 11 ethical vignettes. We found that explicitly instructing participants to consider the ACM code of ethics in their decision making had no observed effect when compared with a control group. Our findings suggest a challenge to the research community: if not a code of ethics, what techniques can improve ethical decision making in software engineering? CCS CONCEPTS • Social and professional topics ! Codes of ethics; his home. Sh competing co Waymo realiz employee, W parently used settled the law The secon ware inside c run in one of day-to-day dr what is allow mode, the ca detected that engineers rai did not bring company wa far [31] and a of the excess
  8. #SleepAtNight @jorodev © Zuhlke 2019 end-users, organizations, and our environment,

    as is evidenced by recent ethics scandals in the news. Organizations, like the ACM, publish codes of ethics to guide software-related ethical decisions. In fact, the ACM has recently demonstrated renewed interest in its code of ethics and made updates for the first time since 1992. To better understand how the ACM code of ethics changes software- related decisions, we replicated a prior behavioral ethics study with 63 software engineering students and 105 professional software developers, measuring their responses to 11 ethical vignettes. We found that explicitly instructing participants to consider the ACM code of ethics in their decision making had no observed effect when compared with a control group. Our findings suggest a challenge to the research community: if not a code of ethics, what techniques can improve ethical decision making in software engineering? CCS CONCEPTS • Social and professional topics ! Codes of ethics; his home. Sh competing co Waymo realiz employee, W parently used settled the law The secon ware inside c run in one of day-to-day dr what is allow mode, the ca detected that engineers rai did not bring company wa far [31] and a of the excess
  9. #SleepAtNight @jorodev © Zuhlke 2019 algorithm
 n. a process or

    set of rules to be followed in calculations or other problem-solving operations, especially by a computer.
  10. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT

    IF name longer than 20 characters THEN name is not valid ELSE name is valid
  11. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT

    IF name longer than 20 characters THEN name is not valid ELSE name is valid NEUTRAL or Not Neutral ?
  12. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT

    IF name longer than 20 characters THEN name is not valid ELSE name is valid NEUTRAL or Not Neutral ?
  13. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT

    PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs.
 passport biometrics
  14. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT

    PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs.
 passport biometrics NEUTRAL or Not Neutral ?
  15. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT

    PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs.
 passport biometrics NEUTRAL or Not Neutral ?
  16. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A

    WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging
  17. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A

    WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging NEUTRAL or Not Neutral ?
  18. #SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A

    WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging NEUTRAL or Not Neutral ?
  19. #SleepAtNight @jorodev © Zuhlke 2019 Why did we make technology

    that will live for 18 months, die, and never rot?
  20. #SleepAtNight @jorodev © Zuhlke 2019 “The company defended itself by

    explaining that “people have to expressly agree to use” the opt-in feature for importing their contacts on Messenger or Facebook Lite to allow Facebook to collect the cell phone data, but Sunday’s statement is the first time the social media company actually spelled out that practice in clear terms for users.”
  21. #SleepAtNight @jorodev © Zuhlke 2019 What about people who 


    aren’t our users and customers, 
 but are still affected by our software? “As an <X>, I want <Y>, so that <Z>”
  22. #SleepAtNight @jorodev © Zuhlke 2019 Is Microsoft good or evil?

    Is Google good or evil? Is Apple good or evil? Is Amazon good or evil? Is Uber good or evil? Is Netflix good or evil?
  23. #SleepAtNight @jorodev © Zuhlke 2019 Is Microsoft good or evil?

    Is Google good or evil? Is Apple good or evil? Is Amazon good or evil? Is Uber good or evil? Is Netflix good or evil?
  24. #SleepAtNight @jorodev © Zuhlke 2019 - C J Silvero, The

    Economics of Open Source Companies do not love you. Not even companies that make something you like.
 
 Npm does not love you. It can't love you. It's a Delaware corporation founded as a financial instrument intended to turn money into more money for a handful of people you don't know.
  25. #SleepAtNight @jorodev © Zuhlke 2019 –Jacob Bronowski, The Ascent of

    Man, BBC “This is where people were turned into numbers. Into this pond were flushed the ashes of some four million people. And that was not done by gas. It was done by arrogance, it was done by dogma, it was done by ignorance.”
  26. #SleepAtNight @jorodev © Zuhlke 2019 Are you willing to provide

    the supporting apparatus for
 human rights abuses?
  27. #SleepAtNight @jorodev © Zuhlke 2019 We could all do with

    examining how the system unfairly benefits us personally. -Reni Eddo-Lodge Why I’m No Longer Talking To White People About Race
  28. male white Anglo-European heritage cisgender able-bodied middle class university educated

    southern English (received pron.) accent masculine presenting stable income stable housing arrangement Image: Devoxx London
  29. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team By

    UKBlackTech - https://ukblacktech.com/stockphotos/
  30. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND


    foster an inclusive environment By UKBlackTech - https://ukblacktech.com/stockphotos/
  31. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND


    foster an inclusive environment AND
 check your privilege By UKBlackTech - https://ukblacktech.com/stockphotos/
  32. #SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND


    foster an inclusive environment AND
 check your privilege AND
 use it to elevate marginalised voices By UKBlackTech - https://ukblacktech.com/stockphotos/
  33. #SleepAtNight @jorodev © Zuhlke 2019 When things in real life

    that sound like satire start to happen, it's way past the time where we need to start reflecting on how we are fostering a society built on hate [...] We cannot let people advance tech for their own selfish reasons while they pretend that they're doing it for everyone. — Jenn Schiffer
  34. #SleepAtNight @jorodev © Zuhlke 2019 Be nice to people Be

    civil to people Don’t feed the trolls Don’t be nasty to people Don’t be a d*ck etc. etc.etc.
  35. #SleepAtNight @jorodev © Zuhlke 2019 Be nice to people Be

    civil to people Don’t feed the trolls Don’t be nasty to people Don’t be a d*ck etc. etc.etc.
  36. #SleepAtNight @jorodev © Zuhlke 2019 Should we use our position

    to • resist handing control of our industry and society to a few dominant players? • resist being the supporting apparatus for oppression and human rights abuses? • break down, rather than entrench, social barriers and injustice?
  37. #SleepAtNight @jorodev © Zuhlke 2019 Single In a Relationship Engaged

    Married In a civil partnership In an open relationship ✓It’s complicated Separated Divorced Widowed
  38. #SleepAtNight @jorodev © Zuhlke 2019 Be humble •Remember: it’s OK

    to be wrong •Think about your concerns. Put them in writing.
 Make a conscious decision to do things •Have a life outside tech
  39. #SleepAtNight @jorodev © Zuhlke 2019 Be brave •Try not to

    be seduced by that framework, buzzword etc. •Think about hostile users, not just attackers •Take ownership for what you’re building.
 Make a conscious decision not to do things
  40. #SleepAtNight @jorodev © Zuhlke 2019 Be kind •Have a visible

    ethical charter and code of conduct—
 and enforce it •Say no, or leave •Make a conscious decision to acknowledge, own, and address your own behaviour
  41. So you can sleep at night written and presented by

    Jonathan Rothwell @jorodev co-written with Steve Freeman @sf105 © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight
  42. #SleepAtNight @jorodev © Zuhlke 2019 Responsibly Smashing Pandora’s Box by

    Yanqing Cheng @YanqingCheng https://www.infoq.com/ presentations/philosophy- personal-role Consequences of an Insightful Algorithm by Carina C. Zona @cczona https://www.youtube.com/ watch?v=YRI40A4tyWU Jenn Schiffer @jennschiffer at XOXO
 https://www.youtube.com/ watch?v=DrCD1wmK9oM Meeting Resistance and Moving Forward by Linda Rising @RisingLinda https://www.youtube.com/ watch?v=DrCD1wmK9oM
  43. #SleepAtNight @jorodev © Zuhlke 2019 •Better - A Surgeon’s Notes

    On Performance
 written by Atul Gawande •IBM and the Holocaust: The Strategic Alliance between Nazi Germany and America's Most Powerful Corporation
 written by Edwin Black •Why I’m No Longer Talking To White People About Race
 written by Reni Eddo-Lodge Recommended Reading
  44. #SleepAtNight @jorodev © Zuhlke 2019 With thanks to: • Dr

    Sal Freudenburg • Emma Obanye • Katherine Kirk • Kate Gray • Sebastiaan Peters • James Thomson • Ruth Yakubu for their help and encouragement • J. Rosenbaum • Dan Hon • Ivan Moore • Asad Razvi • Adrian Kosmaczewski • Paul Hudson • Friends and colleagues at Zuhlke
  45. So you can sleep at night written and presented by

    Jonathan Rothwell @jorodev co-written with Steve Freeman @sf105 Thank you for coming ✨ © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight