Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Navigating the Ethical Minefield of Digital Design

Navigating the Ethical Minefield of Digital Design

Talk given at Camp Digital in 2018.

Per Axbom is a designer, coach and visual explainer working out of Stockholm, Sweden. He has dabbled in usability, accessibility and UX since the mid-nineties. His passion for interviews, sketches and prototypes allow him to understand vividly and explain comprehensibly. His passion for listening helps people get unstuck and move forward in work and life.
Over the years, Per has consulted for over 50 organizations, startups as well as leading international corporations. The past three years he has worked as the UX lead for Sweden’s national platform for online behavioral therapy.
Regularly contributing to the UX community through blog posts and podcasts, Per is very keen on promoting the concept of design ethics and is working on a book to help designers embrace ethics thinking.
NAVIGATING THE ETHICAL MINEFIELD OF DIGITAL DESIGN
I started working with usability hoping to help create a better world by making it easier and more enjoyable to use digital technology. I was driven by a desire to help humans feel less pain when interacting with online tools and services.

Some years ago I started acknowledging ways in which people fell in harm’s way by my design. A notification was perhaps intended as a friendly reminder and call to act on important events. The same notification triggered across a week could increase stress and cause lower work satisfaction. This could be true even when the notification was a direct result of explicit user demands. Worse still, the receivers of these notifications were doctors in charge of patients who could potentially harbor suicidal tendencies.

Even as positive outcomes can be ascribed to a design project we need to look one step further and consider negative impact. Negative impact will occur in many ways: increased anxiety, outspoken misogyny and racism, reduced mental and physical health, diminishing privacy, poor decision-making, deception, loss of trust and more. Yes, even death.

As a designer, you are today pivotal in creating the future we live in. Innovation is progressing faster than public policy or ethical considerations can keep pace. If you want to take responsibility for the impact of your own work you need to incorporate a moral framework in your design process.

Adding to the problem, many designers are using methods borrowed from popular psychology in an attempt to influence decision-making when using a digital service. Concepts such as nudging and behavioral economics encourage designers to manipulate without a full grasp of the consequences.

Last year, Nextdoor experienced negative impact in a big way. Nextdoor is an online space offering private areas for chatting with people in your neighborhood. People are required to use real names and addresses to sign up. In 2015 they were getting concerned reports about racial profiling on their platform as neighbors were reporting suspicious behaviour and only reporting this as "black people".

Imagine having to realize: the platform you built to bring communities closer was in fact creating great divides within them.

Nextdoor took this problem seriously and experienced success in managing the issue. In a walkthrough of the components required for successfully managing negative impact I will use the example of Nextdoor and other use cases to guide you through a process for designing with a conscience.

Avatar for Per Axbom

Per Axbom PRO

June 12, 2018

More Decks by Per Axbom

Other Decks in Design

Transcript

  1. @axbom o bir mühendis he is an engineer o bir

    doktor he is a doctor o bir asker he is a soldier o bir hemşire she is a nurse o bir öğretmen she’s a teacher o bir polis He-she is a police Hän on ruma He’s ugly Hän on kaunis She is beautiful Hän on insinööri He is an engineer Hän on sihteeri She’s a secretary Hän on asianajaja He is a lawyer Hän on lastentarhanopetta She is a kindergarten teacher
  2. @axbom Come on @net ix, I don’t have kids!! Also,

    how would this feel if you lost a child or had trouble conceiving - no button here to remove it. fl @audreylouisa on Twitter
  3. Right Doing the right thing the right way D Wrong

    What we do @axbom Doing the right thing the wrong way Doing the wrong thing the wrong way Wrong Right How we do it
  4. Right Doing the right thing the right way D Wrong

    What we do @axbom Doing the right thing the wrong way Doing the wrong thing the wrong way Doing the wrong thing the right way Wrong Right How we do it
  5. @axbom Do you want the success of your product to

    be determined by how mindless you can make it?
  6. @axbom Loss aversion Anchoring Decoy e ect Sunk co fallacy

    ff st ff fi Con rmation bias Scarcity Expectation bias Bandwagon e ect
  7. @axbom Exploiting biases for pro t 1) It’s immoral /

    not tran arent sp st fi 2) It’s wildly misunder ood
  8. @axbom reported for • self-plagiarism and potential data misrepresentation •

    reporting positive results from a dataset of primarily null outcomes
  9. @axbom Raise your hand (and keep it up until you

    do not agree) I have lied. I have lied to someone I care about. When I lie it is often to protect the feelings of the person I am talking to. I believe I will continue to lie. I have based design decisions on too little data. I will continue to design with too little data. Lying is unethical.
  10. @axbom Ethics? Altruism Asceticism Cognitivism Consequentialism Cynicism Relativism - Absolutism

    Situation ethics Utilitarianism Deontology Egoism Epicureanism Ethical Naturalism Ethical Non-Naturalism Ethical Subjectivism Eudaimonism Hedonism Humanism Individualism Moral Absolutism Moral Anti-Realism Moral Nihilism Kantian ethics Virtue ethics Token absolutism Deontology Moral Realism Moral Relativism Moral Skepticism Moral Universalism Non-Cognitivism Utilitarianism Virtue Ethics
  11. @axbom why? call for ethics why? people are being harmed

    the harm is invisible why? the harm isn’t evaluated
  12. @axbom why? call for ethics why? people are being harmed

    why? the harm is invisible Measure and evaluate how people are being harmed by your solutions. the harm isn’t evaluated
  13. @axbom PERSON 1: “Does my hair look good.” PERSON 2:

    (lying) “Yes.” PERSON 1: “Awesome; I’ll keep going to this new hairdresser I’ve found then.” Impact: Person 1 will keep having bad hair.
  14. @axbom Why you make bad decisions •• •• • Lack

    of knowledge Too fast External pressure Lack of self-awareness Health issues
  15. @axbom Why people make bad decisions •• •• • Lack

    of knowledge Too fast External pressure Lack of self-awareness Health issues
  16. @axbom In the 1960’s to the early 1980’s, the clothing

    and shoes industrial sector in East Africa was thriving and producing for both the local markets as well as the export market. Over the years, the clothing and shoes manufacturing industries have collapsed with the emergence of an informal sector dealing in second-hand clothes and shoes (SHC).
  17. @axbom “Ours is better than yours.” Ghana: Broni wa wo

    (a white person has died) Togo: Dead yovo clothing (dead white people’s clothing) The used-clothing industry is Kenya's seventh largest import, raking in well over 60 million euros per year. Hundreds of thousands of African workers have lost their jobs as a result of these imports.
  18. @axbom ACTION NEGATIVE IMPACT I saw that coming but I

    don’t care. I saw that coming but didn’t have the power to stop it. Whoa! I did not see that coming…
  19. @axbom ACTION NEGATIVE IMPACT I saw that coming but I

    don’t care. I saw that coming but didn’t have the power to stop it. Whoa! I did not see that coming… What can I do to gain power? What can I do before it happens? What can I do after it happens?
  20. @axbom ACTION I saw that coming but I don’t care.

    NEGATIVE IMPACT How do I even know this? I saw that coming but didn’t have the power to stop it. Whoa! I did not see that coming… What can I do to gain power? What can I do before it happens? What can I do after it happens?
  21. @axbom ACTION I saw that coming but I don’t care.

    We allow people to tell us or we nd out ourselves. NEGATIVE IMPACT How do I even know this? I saw that coming but didn’t have the power to stop it. Whoa! I did not see that coming… What can I do to gain power? What can I do before it happens? fi What can I do after it happens?
  22. st @axbom ory 4. CLIMAX 1. PERSON 2. GOAL/PROBLEM 5.

    GAIN 3. ENCOUNTER Design/solution/product
  23. @axbom The other ory 4. CLIMAX/CRISIS 1. PERSON 2. GOAL/PROBLEM

    5. IMPACT/GAIN/LOSS 3. ENCOUNTER st Design/solution/product
  24. st @axbom ory Tries out exposure exercises 36 yo developer

    4. CLIMAX 1. PERSON social anxiety 2. GOAL/PROBLEM Makes a new friend AI therapist app 3. ENCOUNTER Design/solution/product 5. GAIN
  25. @axbom The other ory Unloads on the AI therapist Person

    unable to conceive Child 4. CLIMAX/CRISIS 1. PERSON Abusive parents Never helped 2. GOAL/PROBLEM 5. IMPACT/GAIN/LOSS AI therapist app 3. ENCOUNTER st Design/solution/product
  26. @axbom ACTION A. Eat chocolate OUTPUT OUTCOME IMPACT 1 2

    3 1st order 2nd order 3nd order Energy boost Hungry again Addiction
  27. @axbom ACTION A. Eat chocolate B. Work out OUTPUT OUTCOME

    IMPACT 1 2 3 1st order 2nd order 3nd order Energy boost Hungry again Addiction Exertion/pain Improved physiology Better at work
  28. @axbom ACTION A. Eat chocolate B. Work out ffi C.

    Donate clothes OUTPUT OUTCOME IMPACT 1 2 3 1st order 2nd order 3nd order Energy boost Hungry again Addiction Exertion/pain Improved physiology Better at work Feel better, can buy new Cheap clothes for poor Repressed self-su ciency
  29. @axbom ACTION A. Eat chocolate B. Work out C. Donate

    clothes ffi D. Recycle OUTPUT OUTCOME IMPACT 1 2 3 1st order 2nd order 3rd order Energy boost Hungry again Addiction Exertion/pain Improved physiology Better at work Feel better, can buy new Cheap clothes for poor Repressed self-su ciency Recycling space at home Visit recycling station regularly Waste reduced
  30. @axbom non-participants Excluded ff Inclusive panda. No e ort to

    include Participating but you don’t see them participants Included Unwanted Can get hurt Can hurt others Participating but you don’t see them
  31. @axbom Who? ▢ Ability divergent ▢ Age disfavored ▢ Appearance

    disfavored ▢ Crime/distress endurer ▢ Ethnic minority ▢ Faith disfavored ▢ Gender disfavored ▢ Illness endurer ▢ Non-citizen ▢ Racism endurer ▢ Social class disfavored ▢ Sexuality disfavored What? ▢ Environment ▢ Esteem ▢ Finance ▢ Health ▢ Privacy ▢ Relationships ▢ Safety ▢ Self-actualization ▢ Self-worth ▢ Social belonging ▢ How? ▢ Dangerous defaults ▢ False assumptions ▢ False expectations ▢ Excluded by design ▢ External pressure ▢ Lack of choice ▢ Lack of knowledge ▢ Lack of understanding ▢ Sensory overload ▢ Time constraints ▢ version 0.9 of the cards
  32. @axbom "unprofessional hair” Who? ▢ Ability divergent ▢ Age disfavored

    ▢ Appearance disfavored ▢ Crime/distress endurer ▢ Ethnic minority ▢ Faith disfavored ▢ Gender disfavored ▢ Illness endurer ▢ Non-citizen ▢ Racism endurer ▢ Social class disfavored ▢ Sexuality disfavored What? ▢ Environment ▢ Esteem ▢ Finance ▢ Health ▢ Privacy ▢ Relationships ▢ Safety ▢ Self-actualization ▢ Self-worth ▢ Social belonging ▢ How? ▢ Dangerous defaults ▢ False assumptions ▢ False expectations ▢ Excluded by design ▢ External pressure ▢ Lack of choice ▢ Lack of knowledge ▢ Lack of understanding ▢ Sensory overload ▢ Time constraints ▢ downward-spiralling algorithms version 0.9 of the cards
  33. @axbom WHAT HOW MUCH negative positive marginal deep for few

    for many brief long-lasting WHO well-served under-served CONTRIBUTION much worse much better RISK low impactmanagementproject.com high
  34. n a c i r f A s u o

    i c i p s u s o w t w a p s u t s r u a j c d r i n e e i h r t f y e o v o b r y d y “M e h T . r a c a n ” i . d n e e k r m a n p d n a , n r America u t U a e d a m , t e e r t s e th Imagine having to realize that the platform you built to bring communities closer is in fact creating great divides within them. The private social network for your neighborhood.
  35. "I'm a person of color so it really cut deep.

    We hated the idea that something we built would be viewed as racist… I hadn't seen it in my own neighborhood's Nextdoor and so didn't realize it was an issue for us. Once I got past that, I was powered by the challenge to do something about it." Nirav Tolia, CEO of Nextdoor
  36. How NextDoor managed negative impact ✓ Listening ✓ Assuming ownership

    ✓ Transparency ✓ Positive friction ✓ Leadership buy-in
  37. IMPACT ASSESSMENT Racial pro ling Important negative outcome Neutral Important

    positive outcome What How much Marginal effect Deep effect For few For many Short-term Long-term Slowly Quickly Racism endurers Well-served Underserved Who fi Model adapted from www.impactmanagementproject.com
  38. @axbom 76 work days to read all privacy policies you

    encounter in a year http://techland.time.com/2012/03/06/youd-need-76-work-days-to-read-all-your-privacy-policies-each-year/
  39. @axbom “…[I]f they made it more easy then people would

    actually read it and think twice about the app.” — a 13-year-old girl after being shown a revised policy for Instagram, rewritten in a child-friendly language by lawyer Jenny Afia at Schillings. https://qz.com/878790/a-lawyer-rewrote-instagrams-terms-of-service-forkids-now-you-can-understand-all-of-the-private-data-you-and-your-teenare-giving-up-to-social-media/
  40. You will find the password by reading the welcome information

    carefully. g in g g lo r fo d r o w s s a p e th In this film you will find r u o Y . s r te c a r a h c ll a m s in to the booking system, 5 . il a -m e n o ti a m r fi n o c e username is in th Code of conduct explained whilst giving out one letter of the password every thirty seconds.
  41. Pre-scripting. @axbom We need to do this. Hang on, I’m

    concerned about the impact of this decision. Maybe you have the answer. ALTERNATIVE PATHS SHORT TERM / LONG TERM ADDICTIVE CYCLES COST CONSIDERATION THE OTHER STORY REPUTATION AND TRUST
  42. @axbom What can I do? 1. Anticipate conflict with others

    2. Practice responding to arguments (pre-scripting) 3. Explain and show how the business can suffer from the wrong choice 4. Suggest alternative design paths 5. Run workshops to get the discussion going
  43. @axbom What can I do? 1. Anticipate conflict with others

    2. Practice responding to arguments (pre-scripting) 3. Explain and show how the business can suffer from the wrong choice 4. Suggest alternative design paths 5. Run workshops to get the discussion going Find your allies.
  44. @axbom In every work situation you are in, ask yourself:

    1. Who is in this room? 2. Whose voice is being heard? 3. Whose voice(s) are missing? Then, for each question, ask why.
  45. @axbom We say work with human-centric design. Ensuring human safety

    is our re onsibility. It’s why we go to work. sp If our profession is built on human safety and we are failing to deliver, then how do we claim our right to be taken seriously?
  46. @axbom When you have the opportunity to contribute to good,

    but do nothing, you are contributing to harm. st Harm doesn’t have to be something you do intentionally, sometimes it’s ju the absence of good.
  47. @axbom It continues with us. Per Axbom Coach & Designer

    Re onsible Innovation Advocate Follow @axbom on Twitter sp axbom.eu