Anyone Can Become a Troll

8480b47e733a040fba07c32da414b0e0?s=47 Justin Cheng
February 28, 2017

Anyone Can Become a Troll

In online communities, antisocial behavior such as trolling disrupts constructive discussion. While prior work suggests that trolling behavior is confined to a vocal and antisocial minority, we demonstrate that ordinary people can engage in such behavior as well. We propose two primary trigger mechanisms: the individual's mood, and the surrounding context of a discussion (e.g., exposure to prior trolling behavior). Through an experiment simulating an online discussion, we find that both negative mood and seeing troll posts by others significantly increases the probability of a user trolling, and together double this probability. To support and extend these results, we study how these same mechanisms play out in the wild via a data-driven, longitudinal analysis of a large online news discussion community. This analysis reveals temporal mood effects, and explores long range patterns of repeated exposure to trolling. A predictive model of trolling behavior shows that mood and discussion context together can explain trolling behavior better than an individual's history of trolling. These results combine to suggest that ordinary people can, under the right circumstances, behave like trolls.

Presented at CSCW 2017.

8480b47e733a040fba07c32da414b0e0?s=128

Justin Cheng

February 28, 2017
Tweet

Transcript

  1. (even jcccf, msbernst, cristian, and jure) Anyone can become a

    troll. 1
  2. Time (2016) “How trolls are ruining the internet” 2 News

    headlines
  3. The Atlantic (2016) “When will the internet be safe for

    women?” “How trolls are ruining the internet” 3 News headlines
  4. Salon (2014) “Furious trolls are everywhere” “When will the internet

    be safe for women?” “How trolls are ruining the internet” 4 News headlines
  5. Pew Research (2014) 40% of online users have been harassed

    5
  6. Popular Science (2013) “Why we’re shutting off our comments” 6

    More headlines
  7. The Verge (2013) “We’re turning comments off for a while”

    “Why we’re shutting off our comments” 7 More headlines
  8. Chicago Sun-Times (2014) “Sick of internet comments? Us, too” “We’re

    turning comments off for a while” “Why we’re shutting off our comments” 8 More headlines
  9. Why is trolling so prevalent? RQ 9

  10. Understanding trolling lets us design ❤ communities Implication 10 healthier

    prosocial
  11. What is trolling? 11

  12. Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008) 1. Engaging

    in negatively marked online behavior? What is trolling? 12
  13. Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008) 2. Not

    following the rules? 1. Engaging in negatively marked online behavior? What is trolling? 13
  14. Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008) 3. Taking

    pleasure in upsetting others? 2. Not following the rules? 1. Engaging in negatively marked online behavior? What is trolling? 14
  15. Trolling is behavior outside community norms. 3. Taking pleasure in

    upsetting others? 2. Not following the rules? 1. Engaging in negatively marked online behavior? What is trolling? 15 name-calling personal attacks threats hate speech ethnically/racially offensive material
  16. Donath (1999); Hardaker (2010); Buckels, et al. (2014) Trolling is

    largely due to sociopaths Prior work 16
  17. Trolling is due to ordinary people This work 17

  18. How much do trolls troll? 18

  19. 16M posts on 16K articles from .com 19 Data

  20. How much do trolls troll? Proportion of Banned Users 0

    0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 20
  21. The distribution of trolls is bimodal Proportion of Banned Users

    0 0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 21
  22. Are there two types of trolls? Proportion of Banned Users

    0 0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Situational trolling? 22 Lifelong trolling?
  23. How to show that trolling is situational? 23 Challenge

  24. Observational data isn’t causal How to show that trolling is

    situational? 24
  25. Experiments are hard to generalize Observational data isn’t causal How

    to show that trolling is situational? 25
  26. Solution: online experiment + observational study Experiments are hard to

    generalize Observational data isn’t causal How to show that trolling is situational? 26
  27. Anyone can become a troll 27 Hypothesis

  28. Zimbardo (1969); Wilson (1982) Theory 1: “Broken windows” 28

  29. Jones & Bogat (1978); Rotton & Frey (1985) Theory 2:

    Unpleasant stimuli increase aggression 29
  30. Online experiment simulating a discussion forum 30 N=667 (40% female)

  31. Complete a quiz, then participate in a discussion Online experiment

    simulating a discussion forum 31
  32. We manipulated quiz difficulty and discussion context Complete a quiz,

    then participate in a discussion Online experiment simulating a discussion forum 32
  33. The quiz was either easy or difficult We manipulated quiz

    difficulty and discussion context Complete a quiz, then participate in a discussion Online experiment simulating a discussion forum 33
  34. Discussion context was either positive or negative The quiz was

    either easy or difficult We manipulated quiz difficulty and discussion context Complete a quiz, then participate in a discussion Online experiment simulating a discussion forum 34
  35. Here’s the easy quiz (positive mood condition) 35

  36. Here’s the easy quiz (positive mood condition) 36

  37. Here’s the difficult quiz (negative mood condition) 37

  38. Here’s the positive discussion context condition 38

  39. Here’s negative discussion context condition 39

  40. People were in a worse mood after the difficult quiz

    Easy Quiz Difficult Quiz POMS Mood Disturbance 0 10 20 30 40 50 40 Manipulation checks
  41. People also perceived seed troll posts as worse People were

    in a worse mood after the difficult quiz Seed Troll Seed Non-Troll Proportion of Upvotes 0 0.2 0.4 0.6 0.8 1 41 Manipulation checks
  42. How much trolling was there in each condition? 42 posts

    independently labeled by two expert raters using community guidelines
  43. How much trolling was there in each condition? Positive Mood

    Negative Mood Positive Context Negative Context % Troll Posts 43
  44. Trolling is lowest in the positive conditions… Positive Mood Negative

    Mood Positive Context 35% Negative Context % Troll Posts 44
  45. …increases with either negative condition… Positive Mood Negative Mood Positive

    Context 35% 49% Negative Context 47% % Troll Posts 45
  46. …and almost doubles in the negative conditions (p < 0.05

    using a mixed effects logistic regression model) Positive Mood Negative Mood Positive Context 35% 49% Negative Context 47% 68% % Troll Posts 46
  47. Negative affect also triples (p < 0.05) Positive Mood Negative

    Mood Positive Context 1.1% 1.4% Negative Context 2.3% 2.9% % Neg. Affect Words (LIWC) 47
  48. Comment from the positive mood/context condition: 48

  49. “Hilary is a solid candidate. As a woman, I appreciate

    that she's a woman, but it's not the only reason I think she would do well in office.” Comment from the positive mood/context condition: 49
  50. Comment from the negative mood/context condition: 50

  51. “Anyone who votes for her is a complete idiot. These

    supporters are why this country is in such bad shape now. Uneducated people.” Comment from the negative mood/context condition: 51
  52. Bad mood and negative context increase trolling 52

  53. But does this generalize? 53 Bad mood and negative context

    increase trolling
  54. Online experiment + Observational study 54 But does this generalize?

    Bad mood and negative context increase trolling .com
  55. Golder & Macy (2011) Can trolling vary with the time

    of day or day of week? 55 Replicating mood Neg. Affect Time of Day ?
  56. Can trolling vary with the time of day or day

    of week? 56 Proportion of Flagged Posts 0.03 0.033 0.036 0.039 0.042 Time of Day 0 6 12 18 24
  57. Trolling peaks in the evening… 57 Proportion of Flagged Posts

    0.03 0.033 0.036 0.039 0.042 Time of Day 0 6 12 18 24 Golder & Macy Our Data
  58. …and early in the work week 58 Proportion of Flagged

    Posts 0.03 0.032 0.034 0.036 0.038 0.04 Day of Week Mon Tue Wed Thu Fri Sat Sun
  59. Trolling peaks when moods are worse 59 Time of Day

    Proportion of Flagged Posts Negative Affect Proportion of Downvotes Day of Week
  60. Does mood spill over from prior discussions? 60 Replicating mood

  61. Does mood spill over from prior discussions? 61 (Discussion) …

  62. Does mood spill over from prior discussions? 62 (Discussion) …

    … … … (Unrelated Discussions) … … ? ?
  63. Mood spills over from prior discussions 63 … … (Discussion)

    … … (Unrelated Discussions) … …
  64. Trolling is twice as likely in unrelated discussions (p <

    0.01) 64 … … (Discussion) … … (Unrelated Discussions) … … 2x
  65. Does the initial post affect subsequent trolling? 65 Replicating context

  66. Does the initial post affect subsequent trolling? 66 (Separate discussions

    of same article) … … ? ? ? ? ? ?
  67. An initial post increases later trolling by over 1.5x (p

    < 0.01) 67 (Separate discussions of same article) … … ? ?
  68. Does increased trolling have an additive effect? 68 Replicating context

  69. Does increased trolling have an additive effect? 69 1 2

    3 4 5 … (Discussion)
  70. Does increased trolling have an additive effect? 70 1 2

    3 4 5 … (Discussion)
  71. Does increased trolling have an additive effect? 71 1 2

    3 4 5 … (Discussion) ?
  72. Does increased trolling have an additive effect? 72 1 2

    3 4 5 … (Discussion) ?
  73. More initial trolling means more future trolling 73 Pr(5th Post

    is Flagged) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 # of Flagged Posts in First 4 Posts 0 1 2 3 4 New participant Posted previously
  74. Can we predict trolling before it happens? 74

  75. Logistic regression on 120K posts 75 Can we predict trolling

    before it happens?
  76. Three sets of features: mood, context, or user-specific 76 Trolling

    is situational Trolling is innate Logistic regression on 120K posts Can we predict trolling before it happens?
  77. How predictable is trolling? 77

  78. How predictable is trolling? 78 User-specific Mood Discussion Context Combined

    AUC 0.00 0.20 0.40 0.60 0.80 0.78 0.74 0.60 0.66
  79. How predictable is trolling? 79 User-specific Mood Discussion Context Combined

    AUC 0.00 0.20 0.40 0.60 0.80 0.78 0.74 0.60 0.66
  80. How predictable is trolling? 80 User-specific Mood Discussion Context Combined

    AUC 0.00 0.20 0.40 0.60 0.80 0.78 0.74 0.60 0.66
  81. 81 Troll or not? User

  82. 82 Troll or not? User Mood

  83. 83 Troll or not? User Mood Other users { }

  84. Couldn’t we just ban trolls? 84

  85. But many “trolls” are ordinary people! 85 Couldn’t we just

    ban trolls?
  86. Important to also curb situational trolling: 86 But many “trolls”

    are ordinary people! Couldn’t we just ban trolls?
  87. Through inferring mood… 87 Important to also curb situational trolling:

    But many “trolls” are ordinary people! Couldn’t we just ban trolls? Rate limiting Enabling “undo”
  88. Mazar, Amir, & Ariely (2008) Or altering the context of

    a discussion. 88 Through inferring mood… Important to also curb situational trolling: But many “trolls” are ordinary people! Couldn’t we just ban trolls? Prioritizing constructive comments Ethical/moral reminders
  89. (even @jcccf, @msbernst, cristian, and @jure. read the paper: bit.ly/anyonepaper)

    Because trolling is situational, anyone can become a troll. 89