Designing for Ethical Online Discourse

Notes at https://github.com/Aupajo/social-risk-and-responsibility/tree/master/talks/designing-for-ethical-online-discourse

Transcript

1. This talk will include discussion about hate groups and their behaviour online.
2. Designing for Ethical Online Discourse
3. Design designare • to designate
4. Design is not just what it looks like and feels like. Design is how it works.” “
5. Who is responsible for design? Business? Designers? Developers? Analysts?
6. The responsibility is shared. And the choices we make have consequences.
7. Ethics ēthikē • the science of morals
8. Most of us have an intuitive understanding that we have an ethical obligation to do well by our employer. Do we have other ethical responsibilities?
9. Yes. Society is increasingly dependent on software. We build that software, and so we have a ethical obligation to build responsibly.
11. As skilled professionals, we are hired for our expertise and experience. Part are expected to mitigate threats that may not be obvious to non-experts.
12. As ethical professionals, we should consider issues as they affect business, individuals, and society. We should identify and protect design against risk on all three fronts.
13. Discourse discursus • running to and fro
14. Person persona • mask, character in a play
16. “People are dumb.” express different personas (some of which can be dumb)
19. the bit you have control over…
20. …affects this
21. Recap Design involves ethical thinking. We are ethically bound to design responsibly, and we are professionally entitled to do so. Discourse is heavily influenced by its context. We are responsible for designing this context.
22. Designing Discourse
23. There is tension. What is desirable may not always align. We want to align these. Business Society Individuals
24. S(R)DK Social Risk Development Kit
25. Business Designers Developers Analysts
26. Business Designers Developers Analysts Ownership Strategy Threat Modeling Policies UI Enforcement Response Implementation
27. Ownership
28. Community interactions count as customer experience.
29. Your community should reflect the company’s values.
30. Invest in your customer experience.
31. “Today we removed 28 accounts because of their clear affiliation with known hate groups. The use of Slack by hate groups runs counter to everything we believe in at Slack and is not welcome on our platform. Slack is designed to help businesses communicate better and more collaboratively so people can do their best work. Using Slack to encourage or incite hatred and violence against groups or individuals because of who they are is antithetical to our values and the very purpose of Slack.”
32. 2) Are you aware of the nature of the material this website hosts? “No, nor would it be right for us to monitor the content that flows through our network and make determinations on what is and what is not politically appropriate. Frankly, that would be creepy. The blogger may be confused about the nature of Cloudflare. We are not a hosting provider. Removing this, or any other site, from our network wouldn't remove the content from the Internet: it would simply slow its performance and make it more vulnerable to attack.” 3) What safeguards do you have in place to ensure that Cloudflare does not support illegal terrorist activity? “This question assumes the answer. A website is speech. It is not a bomb. There is no imminent danger it creates and no provider has an affirmative obligation to monitor and make determinations about the theoretically harmful nature of speech a site may contain.”
33. The Pub Metaphor
34. Strategy
35. Danger = Risk ✖ Threat Source: Daniel Fone et al. Likelihood Bad Thing Scariness
36. What is the danger to the company? Negatively impacts the brand Costs of monitoring community Customers revolt
37. Opportunity = Likelihood ✖ Benefit Likelihood Good Thing Excitingness
38. What is the opportunity to the company of a well-executed community? Positively impacts the brand Engagement Community manages itself
39. Establish Goals Desirable business outcomes Desirable community Impacts to business Approach to build and support
40. Threat Modeling
41. NIST Cybersecurity Framework
42. Identify your community goals Protect your community Detect incidents Respond to incidents Recover from incidents
43. Social Harm Risk Assessment What is the potential hazard?
 What is the likelihood of the hazard occurring? How do you mitigate the hazard?
 How do you respond during an incident? How do you recover after an incident?
44. What is the potential hazard?
 Someone leaves hate speech on a blog comment. Social Harm Risk Assessment
45. What is the likelihood of the hazard occurring? Low, but will need to be continually monitored. Social Harm Risk Assessment
46. How do you mitigate the hazard? Remove anonymity. Present the user with a clear policies. Have a moderation team. Keyword detection. Social Harm Risk Assessment
47. How do you mitigate the hazard? (cont’d) Allow users to flag comments. Consider not having comments. Social Harm Risk Assessment
48. Social Harm Risk Assessment How do you respond during an incident? Delete the comment. Disable comments on the post.
49. Social Harm Risk Assessment How do you recover after an incident? Post-mortem. Review processes and policies. Re-enable comments.
50. Discourse Classification Commenting File sharing Chat Long-form Platforms Blogs Imgur Messenger Medium Facebook, Twitter, etc.
51. Policies
52. Develop Situational Awareness
54. Enforcement
55. Reputation
56. Resourcing and outsourcing
57. Response & Recovery
58. “If a search returns largely polluted results that violate our policies, we will stop serving the query, either temporarily or permanently. We started blocking certain searches related to vaccinations and cancer cures last year because results were leading to harmful misinformation.” 
 “It’s better not to serve those results than to lead people down what is like a recommendation rabbit hole.”
 “We want Pinterest to be an inspiring place for people, and there’s nothing inspiring about misinformation.”
60. “We have abolished the ability for readers to upvote or downvote comments. That feature too often led to the most divisive comments being promoted. You can say goodbye to the thumbs icons – now, comments will simply be displayed chronologically.”
 
 “And we've introduced an 'Editor's Pick' function to set a positive example by recognising the best comments. We hope that highlighting the most intelligent, wittiest and most constructive comments will start a virtuous circle.”
61. User Interface
62. Segmentation
63. Setting Expectations
64. Delivery
66. Don’t Auto-Play Media
67. Patterns and anti-patterns
68. Implementation
69. Algorithm design checklist Does this algorithm create engagement by targeting what may be surprising, alarming, or extremist views? Does this algorithm serve users as well as it serves the company? Can it be brought into better alignment?
70. AI and ML to combat hate speech
71. Standards for transmitting speech for review
72. “Responsible by default” libraries
73. Summary
74. You have a responsibility to your users, your organisation, and society to design a safe, secure places. You have the right and the obligation as professionals to challenge the design of the system beyond merely technical considerations. Be smart about how you broach the subject and start the conversation – understand all points of view, present risks, costs, and benefits in a way that everyone can grasp. Use these tools to guide a productive conversation that leads to a beneficial outcome for for your users and your organisation.
75. What’s next?
76. Ownership Strategy Threat Modeling Policies UI Enforcement Response Implementation Examples of leadership, essays, executive whitepapers Examples of strategy, guidance, risk-focused white papers Social harm risk assessment framework Policy blueprints, guidance Social risk design anti-patterns, essays On-demand services (people, AI) for prevention & monitoring Response guides Essays, algorithm design checklist, responsible-by-default libraries Concern Resource
77. github.com/Aupajo/social-risk-and-responsibility