Share
Embed
So You Can Sleep At Night (NSLondon)
Jonathan Rothwell
July 18, 2019
Transcript
157 slides
-
So you can sleep at night… ethics in software development
Jonathan Rothwell (he/him) @jorodev © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight -
#SleepAtNight @jorodev © Zuhlke 2019 I don’t want you to
be hopeful. I want you to panic. –Greta Thunberg Photo © European Parliament -
#SleepAtNight @jorodev © Zuhlke 2019 –Maciej Cegłowski, Build a Better
Monster April 2017 http://idlewords.com/talks/build_a_better_monster.htm …in following our credo of “move fast and break things”, some of what we knocked down were the load-bearing walls of our democracy. -
CONTAINS: SCARY STORIES, SPECULATIONS FREE FROM: CONCLUSIONS PRODUCED IN A
FACILITY CONTAINING NUTS ⚠ -
#SleepAtNight @jorodev © Zuhlke 2019 No software exists in a
vacuum. If software is art, software has context. -
#SleepAtNight @jorodev © Zuhlke 2019 ACM Code of Ethics and
Professional Conduct Affi rming our obligation to use our skills to benefi t society Code Code Code The Code • General Ethical Principles • Professional Responsibilities • Professional Leadership Principles • Compliance with the Code • Case Studies • Using the Code c1_ACM_Code0830.indd 1 8/30/18 10:44 AM We do have codes of conduct… -
#SleepAtNight @jorodev © Zuhlke 2019 Does ACM’s Code of Ethics
Change Ethical Decision Making in Software Development? Andrew McNamara North Carolina State University Raleigh, North Carolina, USA [email protected] Justin Smith North Carolina State University Raleigh, North Carolina, USA [email protected] Emerson Murphy-Hill North Carolina State University Raleigh, North Carolina, USA [email protected] ABSTRACT Ethical decisions in software development can substantially impact end-users, organizations, and our environment, as is evidenced by recent ethics scandals in the news. Organizations, like the ACM, publish codes of ethics to guide software-related ethical decisions. In fact, the ACM has recently demonstrated renewed interest in its code of ethics and made updates for the first time since 1992. To better understand how the ACM code of ethics changes software- related decisions, we replicated a prior behavioral ethics study with 63 software engineering students and 105 professional software developers, measuring their responses to 11 ethical vignettes. We found that explicitly instructing participants to consider the ACM code of ethics in their decision making had no observed effect when compared with a control group. Our findings suggest a challenge to the research community: if not a code of ethics, what techniques can improve ethical decision making in software engineering? CCS CONCEPTS • Social and professional topics ! Codes of ethics; KEYWORDS ACM code of ethics, software engineering ACM Reference Format: Andrew McNamara, Justin Smith, and Emerson Murphy-Hill. 2018. Does ACM’s Code of Ethics Change Ethical Decision Making in Software Develop- ment?. In Proceedings of the 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE ’18), November 4–9, 2018, Lake Buena Vista, FL, USA. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/3236024.3264833 1 INTRODUCTION Software developers must constantly make ethical considerations, including deciding the proper amount of user data to collect; balanc- ing added functionality with potential adverse environmental effects; and performing due diligence to reduce the risks of critical security bugs. Such ethical decisions can cause substantial harm to people, to organizations, and to our planet. Consider two recent examples. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM The first example is the Uber versus Waymo dispute [26], in which a software engineer at Waymo took self-driving car code to his home. Shortly thereafter, the engineer left Waymo to work for a competing company with a self-driving car business, Uber. When Waymo realized that their own code had been taken by their former employee, Waymo sued Uber. Even though the code was not ap- parently used for Uber’s competitive advantage, the two companies settled the lawsuit for $245 million dollars. The second example is the “Dieselgate” scandal [21], where soft- ware inside certain diesel Volkswagen vehicles was programmed to run in one of two modes. In one mode, the car operated under normal, day-to-day driving conditions, but emitted pollution at levels above what is allowed by US and international regulators. In the other mode, the car emitted allowable pollution levels, but only when it detected that it was being inspected by regulators. Although software engineers raised objections to management about the devices, they did not bring these concerns to authorities [19]. Consequently, the company was forced to pay $30 billion dollars in compensation so far [31] and an estimated 59 people suffered early deaths as a result of the excess emitted pollution in the US alone [5]. As early as 1913, organizations have published codes of ethics to guide people facing such ethical situations [3]. In 1972, the Asso- ciation for Computing Machinery (ACM) adopted a code of ethics designed to specifically to apply to software development. In 2018, the ACM code of ethics was updated for the first time since 1992 [2]. In light of recent software ethics scandals, like Dieselgate and the Uber versus Waymo dispute, and ACM’s renewed interest in revising its guidelines, we are motivated to study the effect of ACM’s code of ethics on ethical decision making in software development. While the ACM claims its code of ethics is “intended to serve as a basis for ethical decision making” [1] to our knowledge the effectiveness of this claim has never been tested. We asked 63 software engineering students and 105 professional software engineers to consider 11 software-related ethical decisions. We derived these decisions from real ethical dilemmas faced by software developers. To assess how much the ACM code of ethics in- fluenced decision making, participants were divided into two groups, a control group, and a group explicitly instructed to use the ACM code of ethics. The primary contribution of this paper is a better understanding of ethical decision making in software development and the influence of the ACM code of ethics on those decisions. -
#SleepAtNight @jorodev © Zuhlke 2019 end-users, organizations, and our environment,
as is evidenced by recent ethics scandals in the news. Organizations, like the ACM, publish codes of ethics to guide software-related ethical decisions. In fact, the ACM has recently demonstrated renewed interest in its code of ethics and made updates for the first time since 1992. To better understand how the ACM code of ethics changes software- related decisions, we replicated a prior behavioral ethics study with 63 software engineering students and 105 professional software developers, measuring their responses to 11 ethical vignettes. We found that explicitly instructing participants to consider the ACM code of ethics in their decision making had no observed effect when compared with a control group. Our findings suggest a challenge to the research community: if not a code of ethics, what techniques can improve ethical decision making in software engineering? CCS CONCEPTS • Social and professional topics ! Codes of ethics; his home. Sh competing co Waymo realiz employee, W parently used settled the law The secon ware inside c run in one of day-to-day dr what is allow mode, the ca detected that engineers rai did not bring company wa far [31] and a of the excess -
#SleepAtNight @jorodev © Zuhlke 2019 end-users, organizations, and our environment,
as is evidenced by recent ethics scandals in the news. Organizations, like the ACM, publish codes of ethics to guide software-related ethical decisions. In fact, the ACM has recently demonstrated renewed interest in its code of ethics and made updates for the first time since 1992. To better understand how the ACM code of ethics changes software- related decisions, we replicated a prior behavioral ethics study with 63 software engineering students and 105 professional software developers, measuring their responses to 11 ethical vignettes. We found that explicitly instructing participants to consider the ACM code of ethics in their decision making had no observed effect when compared with a control group. Our findings suggest a challenge to the research community: if not a code of ethics, what techniques can improve ethical decision making in software engineering? CCS CONCEPTS • Social and professional topics ! Codes of ethics; his home. Sh competing co Waymo realiz employee, W parently used settled the law The secon ware inside c run in one of day-to-day dr what is allow mode, the ca detected that engineers rai did not bring company wa far [31] and a of the excess -
#SleepAtNight @jorodev © Zuhlke 2019 It’s impossible to get struck
off as a software developer. -
#SleepAtNight @jorodev © Zuhlke 2019 “Oh, but that’s not us.
We’re just doing …” -
#SleepAtNight @jorodev © Zuhlke 2019 only works if you are
✅ white ✅ cisgender -
#SleepAtNight @jorodev © Zuhlke 2019 algorithm n. a process or
set of rules to be followed in calculations or other problem-solving operations, especially by a computer. -
#SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT
IF name longer than 20 characters THEN name is not valid ELSE name is valid -
#SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT
IF name longer than 20 characters THEN name is not valid ELSE name is valid NEUTRAL or Not Neutral ? -
#SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR VALIDATING NAME INPUT
IF name longer than 20 characters THEN name is not valid ELSE name is valid NEUTRAL or Not Neutral ? -
#SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT
PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs. passport biometrics -
#SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT
PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs. passport biometrics NEUTRAL or Not Neutral ? -
#SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETECTING FACES AT
PASSPORT GATES IF camera detects a beige/brown oval // maybe a face? THEN compare fleshy oval vs. passport biometrics NEUTRAL or Not Neutral ? -
#SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A
WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging -
#SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A
WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging NEUTRAL or Not Neutral ? -
#SleepAtNight @jorodev © Zuhlke 2019 ALGORITHM FOR DETERMINING IF A
WOMAN IS A WITCH Dunk woman into a lake IF woman survives THEN declare her a witch AND execute her by hanging NEUTRAL or Not Neutral ? -
#SleepAtNight @jorodev © Zuhlke 2019 Why did we make technology
that will live for 18 months, die, and never rot? -
#SleepAtNight @jorodev © Zuhlke 2019 But... Most people don’t believe
they’re ‘bad people.’ -
#SleepAtNight @jorodev © Zuhlke 2019 “The company defended itself by
explaining that “people have to expressly agree to use” the opt-in feature for importing their contacts on Messenger or Facebook Lite to allow Facebook to collect the cell phone data, but Sunday’s statement is the first time the social media company actually spelled out that practice in clear terms for users.” -
#SleepAtNight @jorodev © Zuhlke 2019 What about people who
aren’t our users and customers, but are still affected by our software? “As an <X>, I want <Y>, so that <Z>” -
#SleepAtNight @jorodev © Zuhlke 2019 Is Microsoft good or evil?
Is Google good or evil? Is Apple good or evil? Is Amazon good or evil? Is Uber good or evil? Is Netflix good or evil? -
#SleepAtNight @jorodev © Zuhlke 2019 Is Microsoft good or evil?
Is Google good or evil? Is Apple good or evil? Is Amazon good or evil? Is Uber good or evil? Is Netflix good or evil? -
#SleepAtNight @jorodev © Zuhlke 2019 - C J Silvero, The
Economics of Open Source Companies do not love you. Not even companies that make something you like. Npm does not love you. It can't love you. It's a Delaware corporation founded as a financial instrument intended to turn money into more money for a handful of people you don't know. -
#SleepAtNight @jorodev © Zuhlke 2019 –Jacob Bronowski, The Ascent of
Man, BBC “This is where people were turned into numbers. Into this pond were flushed the ashes of some four million people. And that was not done by gas. It was done by arrogance, it was done by dogma, it was done by ignorance.” -
#SleepAtNight @jorodev © Zuhlke 2019 The President of the United
States is a racist. -
#SleepAtNight @jorodev © Zuhlke 2019 The President of the United
States is a racist. -
#SleepAtNight @jorodev © Zuhlke 2019 The President of the United
States is a racist. -
#SleepAtNight @jorodev © Zuhlke 2019 The President of the United
States is a racist. -
#SleepAtNight @jorodev © Zuhlke 2019 The President of the United
States is a racist. -
#SleepAtNight @jorodev © Zuhlke 2019 The President of the United
States is a racist. -
#SleepAtNight @jorodev © Zuhlke 2019 The President of the United
States is a racist. -
#SleepAtNight @jorodev © Zuhlke 2019 The President of the United
States is a racist. -
#SleepAtNight @jorodev © Zuhlke 2019 Are you willing to provide
the supporting apparatus for human rights abuses? -
#SleepAtNight @jorodev © Zuhlke 2019 We could all do with
examining how the system unfairly benefits us personally. -Reni Eddo-Lodge Why I’m No Longer Talking To White People About Race -
male white Anglo-European heritage cisgender able-bodied middle class university educated
southern English (received pron.) accent masculine presenting stable income stable housing arrangement Image: Devoxx London -
#SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team By
UKBlackTech - https://ukblacktech.com/stockphotos/ -
#SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND
foster an inclusive environment By UKBlackTech - https://ukblacktech.com/stockphotos/ -
#SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND
foster an inclusive environment AND check your privilege By UKBlackTech - https://ukblacktech.com/stockphotos/ -
#SleepAtNight @jorodev © Zuhlke 2019 hire a diverse team AND
foster an inclusive environment AND check your privilege AND use it to elevate marginalised voices By UKBlackTech - https://ukblacktech.com/stockphotos/ -
#SleepAtNight @jorodev © Zuhlke 2019 When things in real life
that sound like satire start to happen, it's way past the time where we need to start reflecting on how we are fostering a society built on hate [...] We cannot let people advance tech for their own selfish reasons while they pretend that they're doing it for everyone. — Jenn Schiffer -
#SleepAtNight @jorodev © Zuhlke 2019 Just because you can, it
doesn’t mean you should -
#SleepAtNight @jorodev © Zuhlke 2019 Be nice to people Be
civil to people Don’t feed the trolls Don’t be nasty to people Don’t be a d*ck etc. etc.etc. -
#SleepAtNight @jorodev © Zuhlke 2019 Be nice to people Be
civil to people Don’t feed the trolls Don’t be nasty to people Don’t be a d*ck etc. etc.etc. -
#SleepAtNight @jorodev © Zuhlke 2019 Try to be nice… Always
be kind to people. -
#SleepAtNight @jorodev © Zuhlke 2019 “if I don’t do it,
someone else will” -
#SleepAtNight @jorodev © Zuhlke 2019 Should we use our position
to • resist handing control of our industry and society to a few dominant players? • resist being the supporting apparatus for oppression and human rights abuses? • break down, rather than entrench, social barriers and injustice? -
#SleepAtNight @jorodev © Zuhlke 2019 But… who do we think
we are to make this call? -
#SleepAtNight @jorodev © Zuhlke 2019 Single In a Relationship Engaged
Married In a civil partnership In an open relationship ✓It’s complicated Separated Divorced Widowed -
#SleepAtNight @jorodev © Zuhlke 2019 Be humble •Remember: it’s OK
to be wrong •Think about your concerns. Put them in writing. Make a conscious decision to do things •Have a life outside tech -
#SleepAtNight @jorodev © Zuhlke 2019 Be brave •Try not to
be seduced by that framework, buzzword etc. •Think about hostile users, not just attackers •Take ownership for what you’re building. Make a conscious decision not to do things -
#SleepAtNight @jorodev © Zuhlke 2019 Be kind •Have a visible
ethical charter and code of conduct— and enforce it •Say no, or leave •Make a conscious decision to acknowledge, own, and address your own behaviour -
So you can sleep at night written and presented by
Jonathan Rothwell @jorodev co-written with Steve Freeman @sf105 © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight -
#SleepAtNight @jorodev © Zuhlke 2019 Responsibly Smashing Pandora’s Box by
Yanqing Cheng @YanqingCheng https://www.infoq.com/ presentations/philosophy- personal-role Consequences of an Insightful Algorithm by Carina C. Zona @cczona https://www.youtube.com/ watch?v=YRI40A4tyWU Jenn Schiffer @jennschiffer at XOXO https://www.youtube.com/ watch?v=DrCD1wmK9oM Meeting Resistance and Moving Forward by Linda Rising @RisingLinda https://www.youtube.com/ watch?v=DrCD1wmK9oM -
#SleepAtNight @jorodev © Zuhlke 2019 Links •http://idlewords.com/talks/notes_from_an_emergency.htm •https://www.theguardian.com/technology/2017/may/07/the- great-british-brexit-robbery-hijacked-democracy •https://medium.com/s/story/data-violence-and-how-bad-
engineering-choices-can-damage-society-39e44150e1d4 •https://people.engr.ncsu.edu/ermurph3/papers/fse18nier.pdf -
#SleepAtNight @jorodev © Zuhlke 2019 •Better - A Surgeon’s Notes
On Performance written by Atul Gawande •IBM and the Holocaust: The Strategic Alliance between Nazi Germany and America's Most Powerful Corporation written by Edwin Black •Why I’m No Longer Talking To White People About Race written by Reni Eddo-Lodge Recommended Reading -
#SleepAtNight @jorodev © Zuhlke 2019 With thanks to: • Dr
Sal Freudenburg • Emma Obanye • Katherine Kirk • Kate Gray • Sebastiaan Peters • James Thomson • Ruth Yakubu for their help and encouragement • J. Rosenbaum • Dan Hon • Ivan Moore • Asad Razvi • Adrian Kosmaczewski • Paul Hudson • Friends and colleagues at Zuhlke -
So you can sleep at night written and presented by
Jonathan Rothwell @jorodev co-written with Steve Freeman @sf105 Thank you for coming ✨ © Zuhlke Engineering Ltd. 2017-2019 #SleepAtNight