Erin malone - Mapping Hate - Deconstructing the Ecosystem

UX Australia 2021 DAY 4

What does the landscape of online hate look like and how do you visualize it? Join Erin Malone as she talks about her project with ADL’s Center for Technology and Society to map the ecosystems of hate in digital social spaces.

Transcript

1. Deconstructing Online Hate UX AUSTRALIA 2021 carnagenyc, Hater, https://flic.kr/p/93xwdV
2. DECONSTRUCTING ONLINE HATE Who Am I ▸ 25+ year veteran interaction designer ▸ Currently consulting with Anti-Defamation League’s (ADL) Center for Technology and Society (CTS) ▸ Chair, Interaction Design BFA program at California College of the Arts ▸ Co-author Designing Social Interfaces, 1st & 2nd editions from O’Reilly Media ▸ Design specialties include modeling systems & architectures, designing and advising on social interfaces ▸ Writer ▸ Photographer ▸ Linoleum block printer & letterpress printer ▸ t: @emalone ▸ ig: erin_8pawpress ▸ erin@experiencemattersdesign.com
3. DECONSTRUCTING ONLINE HATE Who is the ADL? ▸ Oldest anti-hate org in world and one of leading 
 civil rights groups in US ▸ Mission: Stop the defamation of the Jewish people and secure justice and fair treatment to all ▸ www.adl.org
4. DECONSTRUCTING ONLINE HATE What is the Center for Technology & Society ▸ ADL actively working with leading companies across Silicon Valley to problem-solve around fighting hate ▸ Actively advising Congress and state elected officials on policy ▸ Tracking & measuring hate online in social platforms and games ▸ Partner with the FairPlay Alliance, a group of over 160 game developers to create a framework for addressing toxic behavior in games (includes definitions, policies, ux design & moderation recommendations) ▸ Building a kinder, gentler Social Patterns interaction design library as a reference for teams battling these issues ▸ https://www.adl.org/who-we-are/ our-organization/advocacy-centers/ center-for-technology-and-society
5. What is the online hate ecosystem? London, England, United Kingdom - June 6th, 2020: Group of people at BLM protest in London by Avel Shah from Noun Project
6. STAKEHOLDER INTERVIEWS
7. How might we visualize DECONSTRUCTING ONLINE HATE
8. 2x2 Buckets of categories Bubble Diagram Cluster Model Concept Model Ecosystem Diagram Genealogy of Hate Honeycomb Diagram Inflows & Outflows Journeymap Mindmap Networks Relationships Taxonomy Timeline Venn diagrams
9. Who are the players WHO•WHAT•WHY•WHERE•WHEN•HOW Group of men and women on anti-government protest by Jacob Lund from Noun Project
10. How I ended up down a rathole EXTREMISTS Craig O'Neal from Florida - KKK Rally in Georgia, Wikimedia Commons
13. Alt-Right Anti-Abortion Anti-Feminist Anti-Gypsyism Anti-Immigrant Anti-LGBT Anti-Muslim Anti-Semitism Antigovernment Movement Black Nationalist Christian Identity General Hate Hate Music Holocaust Denial Identitarian Movement Islamaphobia Isolationism Ku Klux Klan Male Supremacy Misogyny Militias Nativism Neo-Confederate Neo-Fascim Neo-Nazi Neo-Nationalism Neo Volkisch Paleoconservatism Paleolibertarianism Phineas Priesthood Pro-Life Protectionism Racist Skinhead Radical Right Radical Traditional Catholicism Right Wing Populism Sovereign Citizens 
 Movement White Nationalist White Ethnostate White Separatist White Supremacy Anti-American Indian or Alaska Native Anti-Arab Anti-Asian Anti-Black or African American Anti-Hispanic or Latino Anti-Multiple Races, Group Anti-Native Hawaiian or Other Pacific Islander Anti-Other Race/Ethnicity/Ancestry Anti-White Anti-Buddhist Anti-Catholic Anti-Eastern Orthodox 
 (Russian, Greek, Other) Anti-Hindu Anti-Islamic Anti-Jehovah’s Witness Anti-Jewish Anti-Mormon Anti-Multiple Religions, Group Anti-Other Christian Anti-Other Religion Anti-Protestant Anti-Atheism/Agnosticism/etc. Anti-Bisexual Anti-Gay (Male) Anti-Heterosexual Anti-Lesbian Anti-Lesbian, Gay, Bisexual, or 
 Transgender (Mixed Group) Anti-Mental Disability Anti-Physical Disability Anti-Male Anti-Female Gender Identity Anti-Transgender Anti-Gender Non-Conforming
19. First find the parts BACK TO ONLINE HATE ECOSYSTEM Photo by Mika Baumeister on Unsplash
20. Othering Amplifiers Perpetrators Location Relationships Biases People Organizations Dating Apps Sentiment Victimhood Offline Online Anonymous Incentives Section 230 Dehumanization US vs Europe Privacy Data Ideologies Hate Groups Propaganda Mitigators Inflows & Outflows Money Social Platforms Governments Online Games Liability Infrastructure Community Narratives 1st Amendment Conspiracy Theories Non-Profits Schools Misinformation Memes Trolls Mainstream Fringe Extremists Academics Activists Alt-Right NGOs Targets Harassment Discrimination Conspiracy Theories Discrimination Religion Gender Sex Immigrant Anger Fear Ability Rejection Marginalized Marginalized Research Silenced Racism Misogyny Abuse Doxing Violence Death Genocide Stress Education Counter-Speech Players Patriotism Intelligence Hate Manosphere Bigotry Race Stalking Brainwashing Toxic Threats Stereotyping Slurs Ridicule Women LGBTQ+ Reputation Identity Virality Engagement RedPill
21. Othering Amplifiers Perpetrators Location Relationships Biases People Organizations Dating Apps Sentiment Victimhood Offline Online Anonymous Incentives Section 230 Dehumanization US vs Europe Privacy Data Ideologies Hate Groups Propaganda Mitigators Inflows & Outflows Money Social Platforms Governments Online Games Liability Infrastructure Community Narratives 1st Amendment Conspiracy Theories Non-Profits Schools Misinformation Memes Trolls Mainstream Fringe Extremists Academics Activists Alt-Right NGOs Targets Harassment Discrimination Conspiracy Theories Discrimination Religion Gender Sex Immigrant Anger Fear Ability Rejection Marginalized Marginalized Research Silenced Racism Misogyny Abuse Doxing Violence Death Genocide Stress Education Counter-Speech Players Patriotism Intelligence Hate Manosphere Bigotry Race Stalking Brainwashing Toxic Threats Stereotyping Slurs Ridicule Women LGBTQ+ Reputation Identity Virality Engagement RedPill
22. Robert Thivierge, Aryan Guard in Kensington 2, https://flic.kr/p/2UMZUh Ideologies & Biases
23. Large group of people at BLM protest at the Lincoln Memorial, Washington, D.C. by Carol M. Highsmith from Noun Project Race/Ethnicity/ Ancestry Anti-American Indian or Alaska Native Anti-Arab Anti-Asian Anti-Black or African American Anti-Hispanic or Latino Anti-Multiple Races, Group Anti-Native Hawaiian or Other Pacific Islander Anti-Other Race/ Ethnicity/Ancestry Anti-White Religion Anti-Buddhist Anti-Catholic Anti-Eastern Orthodox (Russian, Greek, Other) Anti-Hindu Anti-Islamic Anti-Jehovah’s Witness Anti-Jewish Anti-Mormon Anti-Multiple Religions, Group Anti-Other Christian Anti-Other Religion Anti-Protestant Anti-Atheism/ Agnosticism/etc. Sexual Orientation Anti-Bisexual Anti-Gay (Male) Anti-Heterosexual Anti-Lesbian Anti-Lesbian, Gay, Bisexual, or Transgender (Mixed Group) Disability Anti-Mental Disability Anti-Physical Disability Gender Anti-Male Anti-Female Gender Identity Anti-Transgender Anti-Gender Non- Conforming FBI LIST
24. carnagenyc, Hater, https://flic.kr/p/93xwdV Ageism Ableism Anti-LGBTQ+ Conspiracy Theories Misogyny Nativism Racism Religious Bigotry Xenophobia White Supremacy Hate Groups Individuals Institutions Governments Media Who What
25. Janusz Kaliszczak, 2018-07-14-AntiFascist-005, https://flic.kr/p/M161hm Why Bullied Lashing Out Fear Victim of Hate Looking for and Finding Community Religious Doctrine Radicalized Suffered Abuse Rejection Blaming Others Pain Masquerading as Hate Protection Violent Childhood Mistrust of Other Belief in Stereotypes Learned Cultural Biases Feeling Threatened War
26. Biases & Hate Ideologies Perpetrators
27. Why are people targeted? London, England, United Kingdom - July 7th, 2019: Revelers pictured atop the Pride Bus in London by Avel Shah from Noun Project
28. Photo by Crawford Jolly on Unsplash Racist Hate Speech Death Threats Sexist Hate Speech Discrimination Dehumanization Doxxing Insults Blocking Play Othering Rape Threats Harassment Identity Impersonation Non-consensual Pornography Deepfakes Swatting Bullying Throwing the Game Mobbing Overt Negativity Digital Voyeurism Cyber Stalking Race Religion Gender Gender Identity Sexual Orientation Age National Origin Ethnicity Ability Profession Why How
29. super awesome, riot, https://flic.kr/p/9fNVGx Outcomes Apathy Discrimination Suicides Ethnic Cleansing Silenced Voices Lost Reputation Lost Income Undermined Elections Radicalization Offline Attacks Escalation & Retaliation
30. Targets Behaviors Aga inst Targets
31. Woman taking selfie with her friend for her social media post by Jacob Lund from Noun Project Where is this happening?
32. Photo by Thais Do Rio on Unsplash Arcade Games Role Playing Games Collaborative Games Action & Adventure Games 1st Person Shooter Games Sports Massively Multiplayer Simulations Photo Sharing Videos Live Streaming Podcasting Blogging Chat Messaging Email Message Boards Groups Collecting Events Professional Traditional Media Online Radio Online Games Social Platforms & Apps Dating Apps Media Where Types
33. Three friends hanging out and playing video games by Jacob Lund from Noun Project Activities Live Streams Text Chats Voice Chats Comments Connections Conversations Comments Hashtags Text Photos
34. Amplifiers Activities That Am plify Hate Speech
35. Photo by Sushil Nash on Unsplash Who is working to stop online hate?
36. Community Guidelines Terms of Service Moderators Best Practices Artificial Intelligence Algorithms Product Goals Ethics Principles Designers Journalists Editors Publishers Researchers Academics Authors Activists Youth Groups Schools Teachers & Parents Civil Rights Feminist Policy Immigration Human Rights Discrimination International Federal State Local Social Platforms Online Games Dating Apps Media Individuals NGOs & Non-Profits Governments Who How Photo by Matt Popovich on Unsplash
37. Outputs Rewarding Desired Behavior User Reputation Light Friction Built-in User Tools Algorithmic Filtering Moderator Tools Celebrity Influence Articles Research Papers Books Fact Checking Counter Speech Market Reform Campaigns Workshops Policies Laws Photo by Anna Sullivan on Unsplash
38. Mitigators Mitigator Tactics & Tools
39. Mitigator Tactics & Tools OUTP UTS Activities That Am plify Hate Speech ACTIV ITIES Biases & Hate Ideologies W H Y Behaviors Aga inst Targets OUTC OM ES Hate Groups Media Governments Individuals Institutions Race Death Threats Racist Hate Speech Anti-LGBTQ+ Racism Misogyny Sexist Hate Speech Insults Blocking Play Othering Dehumanization Doxing Rape Threats Harassm ent Identity Im personation Non-consensual Pornography Deepfakes Threats of Violence Swatting Bullying Throwing the Game Mobbing Overt Negativity Digital Voyeurism Stereotyping Gender National Origin Ethnicity Ability Social Platforms & Games Individuals NGOs & Non-Profits Governments Media Social Platform s & Social Apps Online Games Nativism Xenophobia Ageism Religious Bigotry Conspiracy Theories Ableism Religion Traditional Media Blogs Podcasts Online Radio VLogs Academ ics Activists Journalists Civil Rights Feminist Policy Immigration Human Rights Discrimination International Federal State Local Community Guidelines Best Practices Moderators Terms of Service Researchers Artifical Intelligence Algorithms Cyber Stalking Discrimination Photo Sharing Videos Podcasting Live Streaming Blogging Collecting Dating Professional Events M essage Boards M essaging Em ail Groups Chat Perpetrators M itigators Targets A m plifiers Live Streams Text Chats Voice Chats Comments Role Playing Arcade Collaborative Action & Adventure 1st Person Shooter Sports Massively Multiplayer Simulation Youth Groups Schools Teachers & Parents Conversations Photos Com m ents Hashtags Audio Video Text Connections Designers Product Goals Age Market Reform Campaigns User Tools Algorithmic Filtering User Reputation Light Friction Built-in Rewarding Desired Behavior Moderator Tools Counter Speech Celebrity Influence Laws Policies Workshops Research Papers Fact Checking Silenced Voices Radicalization Undermined Elections Ethnic Cleansing Suicides Offline Attacks Escalation & Retaliation Lost Reputation Lost Income Radicalized Suffered Abuse Rejection Bullied Looking for and Finding Community Lashing Out Protection Mistrust of Other Violent Childhood Belief in Stereotypes Learned Cultural Biases Feeling Threatened Victim of Hate Religious Doctrine Pain m asquerading as Hate War Fear Blam ing Others Discrimination Apathy Online Hate Media Editors Authors Publishers Gender Identity Sexual Orientation Profession Articles White Supremacy
40. Understanding the relationships ONLINE HATE ECOSYSTEM Photo by Diego Lozano on Unsplash
41. ONLINE HATE ECOSYSTEM - THE RELATIONSHIPS What are the relationships between the parts? What paths can we see connecting them? Where do we see perpetrators in the system? Where do we see mitigators the system? How do these all interconnect? Photo by Nitish Meena on Unsplash
54. Follow the money ONLINE HATE ECOSYSTEM Photo by Diego Lozano on Unsplash
58. Anthony Crider, CC BY 2.0 , via Wikimedia Commons Escalation path A POTENTIAL
59. ONLINE HATE ECOSYSTEM - A POTENTIAL ESCALATION PATH What is a single person’s experience? Where do they get immersed? Where do bad actors target them? Where might we find opportunities for intervention? Where can platforms intervene or suppress hate content? Who should be intervening and disrupting the cycle? How can design intervene? Photo by Clay Banks on Unsplash
60. A straight path of negative possibility
61. and consumes which in fluences and continues Influences, amplification & confirmation biases reinforce the path
62. Interventions & places of disruption invite discussion around programs, education and other initiatives. and consumes which in fluences and continues
63. Platforms can interrupt & suppress before hate is posted and consumes which in fluences and continues
64. Research HOW DID I GET HERE Photo by Gayatri Malhotra on Unsplash
65. UNDERSTANDING BY THE NUMBERS - PRIMARY & SECONDARY RESEARCH carnagenyc, Hater, https://flic.kr/p/93xwdV Mainstream articles, NGO reports, academic papers & expert authored books read 300+ NGO and Non-profit websites, reports, articles and missions reviewed 50+ Left-leaning & mainstream online magazines, news sites & blogs 19+ & a handful of Alt-Right media sites
66. UNDERSTANDING BY THE NUMBERS - PRIMARY & SECONDARY RESEARCH Janusz Kaliszczak, 2018-07-14-AntiFascist-005, https://flic.kr/p/M161hm External feedback and validation reviewer sessions 17+ Companies, disciplines and expertises represented in reviewer & feedback sessions 15 ADL Stakeholder & Academic interviews to start the research process 17+
67. Questions STILL TO ANSWER Photo by Gayatri Malhotra on Unsplash
68. How do games differ than social platforms? How should our approach differ? QUESTIONS
69. How can we manipulate “engagement” and “virality” for good (kinder) outcomes? QUESTIONS
70. How can we continue to encourage connections but design better guardrails? 
 How much is too much? QUESTIONS
71. How do we know if our design intentions make positive change? QUESTIONS
72. How can understanding the ecosystem help “design” improve in small places for big impact? QUESTIONS
73. Where are the most salient opportunities for change? Where are there time delays? QUESTIONS
74. What policies, laws & legislation should I understand in my role as a designer? QUESTIONS
75. What is our responsibility for pushing back — for keeping kindness & community at the forefront over profits? QUESTIONS
76. Thank You ▸ t: @emalone ▸ ig: erin_8pawpress ▸ www.experiencemattersdesign.com Photo by Gayatri Malhotra on Unsplash Diagrams: https://www.adl.org/online-hate-ecosystem-primer
77. Questions? Group of protestors with their fists raised up in the air by Jacob Lund from Noun Project
78. REFERENCES Royalty Free & Creative Commons Images from: ▸ London, England, United Kingdom - June 6th, 2020: Group of people at BLM protest in London by Avel Shah from Noun Project ▸ Anti-government protest by the social activist group by Jacob Lund from Noun Project ▸ Group of men and women on anti-government protest by Jacob Lund from Noun Project ▸ Large group of people at BLM protest at the Lincoln Memorial, Washington, D.C. by Carol M. Highsmith from Noun Project ▸ Three friends hanging out and playing video games by Jacob Lund from Noun Project ▸ Woman taking selfie with her friend for her social media post by Jacob Lund from Noun Project ▸ London, England, United Kingdom - July 7th, 2019: Revelers pictured atop the Pride Bus in London by Avel Shah from Noun Project ▸ Robert Thivierge, Aryan Guard in Kensington 2, https://flic.kr/p/2UMZUh ▸ Janusz Kaliszczak, 2018-07-14-AntiFascist-005, https://flic.kr/p/M161hm ▸ carnagenyc, Hater, https://flic.kr/p/93xwdV ▸ super awesome, riot, https://flic.kr/p/9fNVGx ▸ Newspaper boxes, photo by Matt Popovich on Unsplash ▸ Courthouse, Photo by Anna Sullivan on Unsplash ▸ Black Lives Matter Signs, Photo by Diego Lozano on Unsplash ▸ Black Lives Matters protesters, Photo by Sushil Nash on Unsplash ▸ Stalking mannequins, Photo by Mika Baumeister on Unsplash ▸ Southern confederate protesters, Anthony Crider, CC BY 2.0 , via Wikimedia Commons ▸ White Pride, Robert Thivierge, Aryan Guard in Kensington 2, https://flic.kr/p/2UMZUh ▸ KKK Rally in Georgia, Craig O'Neal from Florida, CC BY 2.0 , via Wikimedia Commons ▸ Anton Fredrikson / Keski-Suomen museo, Public domain, via Wikimedia Commons ▸ Activists, Photo by Nitish Meena on Unsplash ▸ Black Lives Matters protesters & activists, Photo by Clay Banks on Unsplash ▸ Black Lives Matters protesters, Photo by Gayatri Malhotra on Unsplash ▸ Group of protestors with their fists raised up in the air by Jacob Lund from Noun Project Anton Fredrikson / Keski-Suomen museo, Public domain, via Wikimedia Commons Typography - Font design: www.vocaltype.co - Headings Martin, Subheads/body VTC Carrie, Body text Avenir Next Condensed