Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Anyone Can Become a Troll

Justin Cheng
February 28, 2017

Anyone Can Become a Troll

In online communities, antisocial behavior such as trolling disrupts constructive discussion. While prior work suggests that trolling behavior is confined to a vocal and antisocial minority, we demonstrate that ordinary people can engage in such behavior as well. We propose two primary trigger mechanisms: the individual's mood, and the surrounding context of a discussion (e.g., exposure to prior trolling behavior). Through an experiment simulating an online discussion, we find that both negative mood and seeing troll posts by others significantly increases the probability of a user trolling, and together double this probability. To support and extend these results, we study how these same mechanisms play out in the wild via a data-driven, longitudinal analysis of a large online news discussion community. This analysis reveals temporal mood effects, and explores long range patterns of repeated exposure to trolling. A predictive model of trolling behavior shows that mood and discussion context together can explain trolling behavior better than an individual's history of trolling. These results combine to suggest that ordinary people can, under the right circumstances, behave like trolls.

Presented at CSCW 2017.

Justin Cheng

February 28, 2017
Tweet

More Decks by Justin Cheng

Other Decks in Research

Transcript

  1. The Atlantic (2016) “When will the internet be safe for

    women?” “How trolls are ruining the internet” 3 News headlines
  2. Salon (2014) “Furious trolls are everywhere” “When will the internet

    be safe for women?” “How trolls are ruining the internet” 4 News headlines
  3. The Verge (2013) “We’re turning comments off for a while”

    “Why we’re shutting off our comments” 7 More headlines
  4. Chicago Sun-Times (2014) “Sick of internet comments? Us, too” “We’re

    turning comments off for a while” “Why we’re shutting off our comments” 8 More headlines
  5. Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008) 1. Engaging

    in negatively marked online behavior? What is trolling? 12
  6. Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008) 2. Not

    following the rules? 1. Engaging in negatively marked online behavior? What is trolling? 13
  7. Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008) 3. Taking

    pleasure in upsetting others? 2. Not following the rules? 1. Engaging in negatively marked online behavior? What is trolling? 14
  8. Trolling is behavior outside community norms. 3. Taking pleasure in

    upsetting others? 2. Not following the rules? 1. Engaging in negatively marked online behavior? What is trolling? 15 name-calling personal attacks threats hate speech ethnically/racially offensive material
  9. How much do trolls troll? Proportion of Banned Users 0

    0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 20
  10. The distribution of trolls is bimodal Proportion of Banned Users

    0 0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 21
  11. Are there two types of trolls? Proportion of Banned Users

    0 0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Situational trolling? 22 Lifelong trolling?
  12. Solution: online experiment + observational study Experiments are hard to

    generalize Observational data isn’t causal How to show that trolling is situational? 26
  13. Jones & Bogat (1978); Rotton & Frey (1985) Theory 2:

    Unpleasant stimuli increase aggression 29
  14. We manipulated quiz difficulty and discussion context Complete a quiz,

    then participate in a discussion Online experiment simulating a discussion forum 32
  15. The quiz was either easy or difficult We manipulated quiz

    difficulty and discussion context Complete a quiz, then participate in a discussion Online experiment simulating a discussion forum 33
  16. Discussion context was either positive or negative The quiz was

    either easy or difficult We manipulated quiz difficulty and discussion context Complete a quiz, then participate in a discussion Online experiment simulating a discussion forum 34
  17. People were in a worse mood after the difficult quiz

    Easy Quiz Difficult Quiz POMS Mood Disturbance 0 10 20 30 40 50 40 Manipulation checks
  18. People also perceived seed troll posts as worse People were

    in a worse mood after the difficult quiz Seed Troll Seed Non-Troll Proportion of Upvotes 0 0.2 0.4 0.6 0.8 1 41 Manipulation checks
  19. How much trolling was there in each condition? 42 posts

    independently labeled by two expert raters using community guidelines
  20. How much trolling was there in each condition? Positive Mood

    Negative Mood Positive Context Negative Context % Troll Posts 43
  21. Trolling is lowest in the positive conditions… Positive Mood Negative

    Mood Positive Context 35% Negative Context % Troll Posts 44
  22. …increases with either negative condition… Positive Mood Negative Mood Positive

    Context 35% 49% Negative Context 47% % Troll Posts 45
  23. …and almost doubles in the negative conditions (p < 0.05

    using a mixed effects logistic regression model) Positive Mood Negative Mood Positive Context 35% 49% Negative Context 47% 68% % Troll Posts 46
  24. Negative affect also triples (p < 0.05) Positive Mood Negative

    Mood Positive Context 1.1% 1.4% Negative Context 2.3% 2.9% % Neg. Affect Words (LIWC) 47
  25. “Hilary is a solid candidate. As a woman, I appreciate

    that she's a woman, but it's not the only reason I think she would do well in office.” Comment from the positive mood/context condition: 49
  26. “Anyone who votes for her is a complete idiot. These

    supporters are why this country is in such bad shape now. Uneducated people.” Comment from the negative mood/context condition: 51
  27. Online experiment + Observational study 54 But does this generalize?

    Bad mood and negative context increase trolling .com
  28. Golder & Macy (2011) Can trolling vary with the time

    of day or day of week? 55 Replicating mood Neg. Affect Time of Day ?
  29. Can trolling vary with the time of day or day

    of week? 56 Proportion of Flagged Posts 0.03 0.033 0.036 0.039 0.042 Time of Day 0 6 12 18 24
  30. Trolling peaks in the evening… 57 Proportion of Flagged Posts

    0.03 0.033 0.036 0.039 0.042 Time of Day 0 6 12 18 24 Golder & Macy Our Data
  31. …and early in the work week 58 Proportion of Flagged

    Posts 0.03 0.032 0.034 0.036 0.038 0.04 Day of Week Mon Tue Wed Thu Fri Sat Sun
  32. Trolling peaks when moods are worse 59 Time of Day

    Proportion of Flagged Posts Negative Affect Proportion of Downvotes Day of Week
  33. Does mood spill over from prior discussions? 62 (Discussion) …

    … … … (Unrelated Discussions) … … ? ?
  34. Mood spills over from prior discussions 63 … … (Discussion)

    … … (Unrelated Discussions) … …
  35. Trolling is twice as likely in unrelated discussions (p <

    0.01) 64 … … (Discussion) … … (Unrelated Discussions) … … 2x
  36. An initial post increases later trolling by over 1.5x (p

    < 0.01) 67 (Separate discussions of same article) … … ? ?
  37. More initial trolling means more future trolling 73 Pr(5th Post

    is Flagged) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 # of Flagged Posts in First 4 Posts 0 1 2 3 4 New participant Posted previously
  38. Three sets of features: mood, context, or user-specific 76 Trolling

    is situational Trolling is innate Logistic regression on 120K posts Can we predict trolling before it happens?
  39. Important to also curb situational trolling: 86 But many “trolls”

    are ordinary people! Couldn’t we just ban trolls?
  40. Through inferring mood… 87 Important to also curb situational trolling:

    But many “trolls” are ordinary people! Couldn’t we just ban trolls? Rate limiting Enabling “undo”
  41. Mazar, Amir, & Ariely (2008) Or altering the context of

    a discussion. 88 Through inferring mood… Important to also curb situational trolling: But many “trolls” are ordinary people! Couldn’t we just ban trolls? Prioritizing constructive comments Ethical/moral reminders
  42. (even @jcccf, @msbernst, cristian, and @jure. read the paper: bit.ly/anyonepaper)

    Because trolling is situational, anyone can become a troll. 89