Anyone Can Become a Troll

In online communities, antisocial behavior such as trolling disrupts constructive discussion. While prior work suggests that trolling behavior is confined to a vocal and antisocial minority, we demonstrate that ordinary people can engage in such behavior as well. We propose two primary trigger mechanisms: the individual's mood, and the surrounding context of a discussion (e.g., exposure to prior trolling behavior). Through an experiment simulating an online discussion, we find that both negative mood and seeing troll posts by others significantly increases the probability of a user trolling, and together double this probability. To support and extend these results, we study how these same mechanisms play out in the wild via a data-driven, longitudinal analysis of a large online news discussion community. This analysis reveals temporal mood effects, and explores long range patterns of repeated exposure to trolling. A predictive model of trolling behavior shows that mood and discussion context together can explain trolling behavior better than an individual's history of trolling. These results combine to suggest that ordinary people can, under the right circumstances, behave like trolls.

Presented at CSCW 2017.

Transcript

1. (even jcccf, msbernst, cristian, and jure) Anyone can become a troll. 1
2. Time (2016) “How trolls are ruining the internet” 2 News headlines
3. The Atlantic (2016) “When will the internet be safe for women?” “How trolls are ruining the internet” 3 News headlines
4. Salon (2014) “Furious trolls are everywhere” “When will the internet be safe for women?” “How trolls are ruining the internet” 4 News headlines
5. Pew Research (2014) 40% of online users have been harassed 5
6. Popular Science (2013) “Why we’re shutting off our comments” 6 More headlines
7. The Verge (2013) “We’re turning comments off for a while” “Why we’re shutting off our comments” 7 More headlines
8. Chicago Sun-Times (2014) “Sick of internet comments? Us, too” “We’re turning comments off for a while” “Why we’re shutting off our comments” 8 More headlines
9. Why is trolling so prevalent? RQ 9
10. Understanding trolling lets us design ❤ communities Implication 10 healthier prosocial
11. What is trolling? 11
12. Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008) 1. Engaging in negatively marked online behavior? What is trolling? 12
13. Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008) 2. Not following the rules? 1. Engaging in negatively marked online behavior? What is trolling? 13
14. Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008) 3. Taking pleasure in upsetting others? 2. Not following the rules? 1. Engaging in negatively marked online behavior? What is trolling? 14
15. Trolling is behavior outside community norms. 3. Taking pleasure in upsetting others? 2. Not following the rules? 1. Engaging in negatively marked online behavior? What is trolling? 15 name-calling personal attacks threats hate speech ethnically/racially offensive material
16. Donath (1999); Hardaker (2010); Buckels, et al. (2014) Trolling is largely due to sociopaths Prior work 16
17. Trolling is due to ordinary people This work 17
18. How much do trolls troll? 18
19. 16M posts on 16K articles from .com 19 Data
20. How much do trolls troll? Proportion of Banned Users 0 0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 20
21. The distribution of trolls is bimodal Proportion of Banned Users 0 0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 21
22. Are there two types of trolls? Proportion of Banned Users 0 0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Situational trolling? 22 Lifelong trolling?
23. How to show that trolling is situational? 23 Challenge
24. Observational data isn’t causal How to show that trolling is situational? 24
25. Experiments are hard to generalize Observational data isn’t causal How to show that trolling is situational? 25
26. Solution: online experiment + observational study Experiments are hard to generalize Observational data isn’t causal How to show that trolling is situational? 26
27. Anyone can become a troll 27 Hypothesis
28. Zimbardo (1969); Wilson (1982) Theory 1: “Broken windows” 28
29. Jones & Bogat (1978); Rotton & Frey (1985) Theory 2: Unpleasant stimuli increase aggression 29
30. Online experiment simulating a discussion forum 30 N=667 (40% female)
31. Complete a quiz, then participate in a discussion Online experiment simulating a discussion forum 31
32. We manipulated quiz difficulty and discussion context Complete a quiz, then participate in a discussion Online experiment simulating a discussion forum 32
33. The quiz was either easy or difficult We manipulated quiz difficulty and discussion context Complete a quiz, then participate in a discussion Online experiment simulating a discussion forum 33
34. Discussion context was either positive or negative The quiz was either easy or difficult We manipulated quiz difficulty and discussion context Complete a quiz, then participate in a discussion Online experiment simulating a discussion forum 34
35. Here’s the easy quiz (positive mood condition) 35
36. Here’s the easy quiz (positive mood condition) 36
37. Here’s the difficult quiz (negative mood condition) 37
38. Here’s the positive discussion context condition 38
39. Here’s negative discussion context condition 39
40. People were in a worse mood after the difficult quiz Easy Quiz Difficult Quiz POMS Mood Disturbance 0 10 20 30 40 50 40 Manipulation checks
41. People also perceived seed troll posts as worse People were in a worse mood after the difficult quiz Seed Troll Seed Non-Troll Proportion of Upvotes 0 0.2 0.4 0.6 0.8 1 41 Manipulation checks
42. How much trolling was there in each condition? 42 posts independently labeled by two expert raters using community guidelines
43. How much trolling was there in each condition? Positive Mood Negative Mood Positive Context Negative Context % Troll Posts 43
44. Trolling is lowest in the positive conditions… Positive Mood Negative Mood Positive Context 35% Negative Context % Troll Posts 44
45. …increases with either negative condition… Positive Mood Negative Mood Positive Context 35% 49% Negative Context 47% % Troll Posts 45
46. …and almost doubles in the negative conditions (p < 0.05 using a mixed effects logistic regression model) Positive Mood Negative Mood Positive Context 35% 49% Negative Context 47% 68% % Troll Posts 46
47. Negative affect also triples (p < 0.05) Positive Mood Negative Mood Positive Context 1.1% 1.4% Negative Context 2.3% 2.9% % Neg. Affect Words (LIWC) 47
48. Comment from the positive mood/context condition: 48
49. “Hilary is a solid candidate. As a woman, I appreciate that she's a woman, but it's not the only reason I think she would do well in office.” Comment from the positive mood/context condition: 49
50. Comment from the negative mood/context condition: 50
51. “Anyone who votes for her is a complete idiot. These supporters are why this country is in such bad shape now. Uneducated people.” Comment from the negative mood/context condition: 51
52. Bad mood and negative context increase trolling 52
53. But does this generalize? 53 Bad mood and negative context increase trolling
54. Online experiment + Observational study 54 But does this generalize? Bad mood and negative context increase trolling .com
55. Golder & Macy (2011) Can trolling vary with the time of day or day of week? 55 Replicating mood Neg. Affect Time of Day ?
56. Can trolling vary with the time of day or day of week? 56 Proportion of Flagged Posts 0.03 0.033 0.036 0.039 0.042 Time of Day 0 6 12 18 24
57. Trolling peaks in the evening… 57 Proportion of Flagged Posts 0.03 0.033 0.036 0.039 0.042 Time of Day 0 6 12 18 24 Golder & Macy Our Data
58. …and early in the work week 58 Proportion of Flagged Posts 0.03 0.032 0.034 0.036 0.038 0.04 Day of Week Mon Tue Wed Thu Fri Sat Sun
59. Trolling peaks when moods are worse 59 Time of Day Proportion of Flagged Posts Negative Affect Proportion of Downvotes Day of Week
60. Does mood spill over from prior discussions? 60 Replicating mood
61. Does mood spill over from prior discussions? 61 (Discussion) … …
62. Does mood spill over from prior discussions? 62 (Discussion) … … … … (Unrelated Discussions) … … ? ?
63. Mood spills over from prior discussions 63 … … (Discussion) … … (Unrelated Discussions) … …
64. Trolling is twice as likely in unrelated discussions (p < 0.01) 64 … … (Discussion) … … (Unrelated Discussions) … … 2x
65. Does the initial post affect subsequent trolling? 65 Replicating context
66. Does the initial post affect subsequent trolling? 66 (Separate discussions of same article) … … ? ? ? ? ? ?
67. An initial post increases later trolling by over 1.5x (p < 0.01) 67 (Separate discussions of same article) … … ? ?
68. Does increased trolling have an additive effect? 68 Replicating context
69. Does increased trolling have an additive effect? 69 1 2 3 4 5 … (Discussion)
70. Does increased trolling have an additive effect? 70 1 2 3 4 5 … (Discussion)
71. Does increased trolling have an additive effect? 71 1 2 3 4 5 … (Discussion) ?
72. Does increased trolling have an additive effect? 72 1 2 3 4 5 … (Discussion) ?
73. More initial trolling means more future trolling 73 Pr(5th Post is Flagged) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 # of Flagged Posts in First 4 Posts 0 1 2 3 4 New participant Posted previously
74. Can we predict trolling before it happens? 74
75. Logistic regression on 120K posts 75 Can we predict trolling before it happens?
76. Three sets of features: mood, context, or user-specific 76 Trolling is situational Trolling is innate Logistic regression on 120K posts Can we predict trolling before it happens?
77. How predictable is trolling? 77
78. How predictable is trolling? 78 User-specific Mood Discussion Context Combined AUC 0.00 0.20 0.40 0.60 0.80 0.78 0.74 0.60 0.66
79. How predictable is trolling? 79 User-specific Mood Discussion Context Combined AUC 0.00 0.20 0.40 0.60 0.80 0.78 0.74 0.60 0.66
80. How predictable is trolling? 80 User-specific Mood Discussion Context Combined AUC 0.00 0.20 0.40 0.60 0.80 0.78 0.74 0.60 0.66
81. 81 Troll or not? User
82. 82 Troll or not? User Mood
83. 83 Troll or not? User Mood Other users { }
84. Couldn’t we just ban trolls? 84
85. But many “trolls” are ordinary people! 85 Couldn’t we just ban trolls?
86. Important to also curb situational trolling: 86 But many “trolls” are ordinary people! Couldn’t we just ban trolls?
87. Through inferring mood… 87 Important to also curb situational trolling: But many “trolls” are ordinary people! Couldn’t we just ban trolls? Rate limiting Enabling “undo”
88. Mazar, Amir, & Ariely (2008) Or altering the context of a discussion. 88 Through inferring mood… Important to also curb situational trolling: But many “trolls” are ordinary people! Couldn’t we just ban trolls? Prioritizing constructive comments Ethical/moral reminders
89. (even @jcccf, @msbernst, cristian, and @jure. read the paper: bit.ly/anyonepaper) Because trolling is situational, anyone can become a troll. 89