Antisocial Computing

Transcript

1. Justin Cheng / @jcccf / clr3.com
   with Jure Leskovec, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Tim Hsieh
   ANTISOCIAL COMPUTING
   Explaining, Predicting, and Mediating Online Negative Behavior
   v1.1
   

   View Slide

2. Vieweg, et al. (2010); Kittur, et al. (2013); Burke & Kraut (2016)
   

   View Slide

3. View Slide

4. 47% of online users
   have been harassed
   Data & Society (2017)
   

   View Slide

5. Popular Science (2013); The Verge (2015); Chicago Sun-Times (2014)
   

   View Slide

6. Why is bad behavior so prevalent?

   (╯°□°)╯︵ ┻━┻
   Research Question
   

   View Slide

7. Understanding bad behavior helps
   us build healthier communities
   Implications
   Systems
   Guidelines Interventions
   

   View Slide

8. Antisocial behavior is largely
   due to sociopaths
   Prior Work
   Donath (1999); Hardaker (2010); Buckels, et al. (2014)
   

   View Slide

9. Antisocial behavior is largely
   due to ordinary people
   This Work
   

   View Slide

10. Antisocial Computing
    Talk Outline
    What causes antisocial behavior?
    How do systems mediate it?
    Does it worsen over time?
    1
    2
    3
    WIP!
    

    View Slide

11. Antisocial Computing
    Talk Outline
    What causes antisocial behavior?
    How do systems mediate it?
    Does it worsen over time?
    1
    2
    3
    WIP!
    

    View Slide

12. CSCW 2017 (Best Paper); ICWSM 2015 (Honorable Mention)
    with M. Bernstein, C. Danescu-Niculescu-Mizil, J. Leskovec
    CAN ANYONE
    BECOME A TROLL?
    Causes of Antisocial Behavior in Online Discussions
    

    View Slide

13. CONTENT WARNING!

    This talk contains depictions of trolling that use strong language.
    !
    

    View Slide

14. View Slide

15. It also shows that Islam and
    Christianity teaching women to
    dress modest could be right
    afterall.
    

    View Slide

16. It also shows that Islam and
    Christianity teaching women to
    dress modest could be right
    afterall.
    Clearly that is the only logical
    conclusion to this article. Now if
    you'll excuse me, I need to iron
    my tarp. I have work on Monday,
    and I want to appear 'modest'.
    fail at life. go bomb yourself.
    Religious nut alert
    

    View Slide

17. We studied multiple large comment-
    based news communities.
    470M posts 831M votes
    76M users
    

    View Slide

18. What is trolling?
    

    View Slide

19. What is trolling?
    Engaging in negatively marked online
    behavior
    Taking pleasure in upsetting others
    Not following the rules
    Disrupting a group while staying
    undercover
    Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008)
    

    View Slide

20. Trolling is behavior that occurs
    outside community norms.
    Defined using community guidelines
    Our Definition
    e.g., name-calling, personal attacks, profanity, threats, hate speech, ethnically/racially offensive material
    

    View Slide

21. Are trolls just a vocal minority?
    Donath (1999); Hardaker (2010); Shachaf & Hara (2010); NYT (2008); Wired (2014); Vox (2014)
    

    View Slide

22. How much do trolls troll?
    Proportion of Banned Users
    0
    0.1
    0.2
    0.3
    0.4
    Proportion of Deleted Posts
    0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
    

    View Slide

23. The distribution of trolls is bimodal
    Proportion of Banned Users
    0
    0.1
    0.2
    0.3
    0.4
    Proportion of Deleted Posts
    0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
    

    View Slide

24. Are there two types of trolls?
    Proportion of Banned Users
    0
    0.1
    0.2
    0.3
    0.4
    Proportion of Deleted Posts
    0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
    Situational trolling?
    Lifelong trolling?
    

    View Slide

25. What if antisocial behavior is
    situational?
    

    View Slide

26. Challenge: how to show that
    antisocial behavior is situational?
    Observational data isn’t causal
    

    View Slide

27. Challenge: how to show that
    antisocial behavior is situational?
    Experiments hard to generalize
    

    View Slide

28. Simulated Discussion Experiment Large-Scale Analysis
    Solution: Experiment + Observational Study
    

    View Slide

29. Anyone can become a troll
    Our Hypothesis
    

    View Slide

30. “Broken windows” theory
    Zimbardo (1969); Wilson (1982)
    

    View Slide

31. Unpleasant stimuli increase aggression
    Jones & Bogat (1978); Rotton & Frey (1985)
    

    View Slide

32. Experiment: simulated discussion forum
    

    View Slide

33. N=667, 40% female
    Quiz Discussion
    Experimental method
    

    View Slide

34. Quiz Discussion
    Experimental method
    ×
    

    View Slide

35. Positive/Negative Mood Positive/Negative Context
    Experimental method
    ×
    Quiz Discussion
    

    View Slide

36. Easy quiz (positive mood)
    

    View Slide

37. Difficult quiz (negative mood)
    

    View Slide

38. Positive discussion context
    

    View Slide

39. Negative discussion context
    

    View Slide

40. How did trolling differ across
    conditions?
    Two expert raters labeled posts independently
    

    View Slide

41. How did trolling differ across conditions?
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    Negative
    Context
    % Troll Posts
    

    View Slide

42. Trolling is lowest with positive conditions…
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    35%
    Negative
    Context
    % Troll Posts
    

    View Slide

43. …increases with either negative condition…
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    35% 49%
    Negative
    Context
    47%
    % Troll Posts
    

    View Slide

44. …and almost doubles in the worst case
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    35% 49%
    Negative
    Context
    47% 68%
    % Troll Posts
    (p < 0.05 using a mixed effects logistic regression model)
    

    View Slide

45. Negative affect almost triples
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    1.1% 1.4%
    Negative
    Context
    2.3% 2.9%
    % Negative Affect Words (LIWC)
    (p < 0.05)
    

    View Slide

46. Hilary is a solid candidate. As a woman, I
    appreciate that she's a woman, but it's not
    the only reason I think she would do well in
    office.
    Positive Mood + Context
    

    View Slide

47. Anyone who votes for her is a complete
    idiot. These supporters are why this country
    is in such bad shape now. Uneducated
    people.
    Negative Mood + Context
    

    View Slide

48. Bad mood and negative discussion
    context increase trolling
    

    View Slide

49. Simulated Discussion Experiment Large-Scale Analysis of CNN.com
    Online Experiment + Observational Study
    

    View Slide

50. Can trolling, like mood, vary with
    the time of day and day of week?
    Replicating Mood
    Golder & Macy (2011)
    Neg. Affect
    Time of Day
    ?
    

    View Slide

51. How does trolling vary with time of day?
    Proportion of Flagged Posts
    0.03
    0.033
    0.036
    0.039
    0.042
    Time of Day
    0 6 12 18 24
    

    View Slide

52. Trolling peaks in the evening…
    Proportion of Flagged Posts
    0.03
    0.033
    0.036
    0.039
    0.042
    Time of Day
    0 6 12 18 24
    Negative Affect
    (Golder & Macy)
    

    View Slide

53. …and early in the work week.
    Proportion of Flagged Posts
    0.03
    0.032
    0.034
    0.036
    0.038
    0.04
    Day of Week
    Mon Tue Wed Thu Fri Sat Sun
    

    View Slide

54. Trolling peaks when moods are worse
    Time of Day
    Proportion of
    Flagged Posts
    Negative Affect
    Proportion of
    Downvotes
    Day of Week
    

    View Slide

55. Mood spills over from prior
    discussions
    Replicating Mood
    

    View Slide

56. …
    …
    (Discussion)
    Mood spills over from prior
    discussions
    

    View Slide

57. …
    …
    …
    ?
    …
    (Discussion)
    (Unrelated Discussions)
    …
    ?
    …
    Mood spills over from prior
    discussions
    

    View Slide

58. …
    …
    …
    …
    (Discussion)
    …
    …
    Mood spills over from prior
    discussions
    

    View Slide

59. A user who trolled in a previous
    discussion is twice as likely to troll
    in a later, unrelated discussion
    (p < 0.01)
    Replicating Mood
    

    View Slide

60. The initial post affects subsequent
    trolling
    Replicating Context
    

    View Slide

61. (Separate discussions of same article)
    …
    ? ? ?
    …
    ? ? ?
    The initial post affects subsequent
    trolling
    

    View Slide

62. (Separate discussions of same article)
    …
    …
    The initial post affects subsequent
    trolling
    

    View Slide

63. An initial troll post increases the
    subsequent trolling by 63%
    (p < 0.01)
    Replicating Context
    

    View Slide

64. Troll or not?
    User
    

    View Slide

65. Troll or not?
    User
    Mood
    

    View Slide

66. Troll or not?
    User
    Mood Other users
    { }
    

    View Slide

67. Because trolling is situational,
    ordinary people can end up trolling
    

    View Slide

68. View Slide

69. Can voting mitigate bad behavior?
    

    View Slide

70. Downvoting causes
    negative behavior to worsen
    Our Hypothesis
    

    View Slide

71. Antisocial Computing
    Talk Outline
    What causes antisocial behavior?
    How do systems mediate it?
    Does it worsen over time?
    1
    2
    3
    WIP!
    

    View Slide

72. How Antisocial Behavior Worsens
    ICWSM 2014
    with C. Danescu-Niculescu-Mizil, J. Leskovec
    CAN ANTISOCIAL
    BEHAVIOR SPI
    R
    A
    L?
    

    View Slide

73. Downvoting causes
    negative behavior to worsen
    Our Hypothesis
    

    View Slide

74. What effects do evaluations have?
    Positively
    evaluated
    ?
    ?
    Negatively
    evaluated
    

    View Slide

75. What is a positive or negative evaluation?
    

    View Slide

76. Defining positive and negative evaluations
    :
    9 1
    2 8
    N↑

    N↑
    + N↓
    2

    2+8
    = = 0.2 p↓
    ≤
    : N↑

    N↑
    + N↓
    9

    9+1
    = = 0.9 p↑
    ≥
    Positive Evaluation
    Negative Evaluation
    (validated using a crowdsourcing experiment)
    

    View Slide

77. What effects do evaluations have?
    Positively
    evaluated
    ?
    ?
    Negatively
    evaluated
    

    View Slide

78. Does feedback encourage better behavior?
    Skinner (1938)
    

    View Slide

79. Or is bad stronger than good?
    Brinko (1993); Baumeister, et al. (2001)
    

    View Slide

80. Four large comment-based news
    communities
    

    View Slide

81. What effects do evaluations have?
    Positively
    evaluated
    Negatively
    evaluated
    

    View Slide

82. What effects do evaluations have?
    …
    …
    …
    …
    

    View Slide

83. What effects do evaluations have?
    …
    …
    …
    …
    Before After
    vs.
    Before After
    vs.
    

    View Slide

84. Challenge: how to compare
    different users and posts?
    Aren’t downvoted users/posts inherently worse?
    

    View Slide

85. Solution: propensity score matching
    PSM: Rosenbaum (1983); CEM: Iacus, et al. (2012)
    Positively
    evaluated
    Negatively
    evaluated
    

    View Slide

86. Match on text quality
    Similar text
    quality q
    }
    ≈
    

    View Slide

87. Computing text quality
    (Validated using crowdsourcing)
    Learn p with bigrams
    1 3
    lorem ipsum…
    q = ?
    Lorem…
    ? ?
    9 2
    lorem ipsum…
    …
    Text quality q is
    predicted p
    

    View Slide

88. Match on text quality
    Similar text
    quality
    q(c↑
    )=q(c↓
    )
    }
    ≈
    

    View Slide

89. …as well as other covariates
    Similar history
    (# posts, overall
    proportion of
    upvotes, etc.)
    { ≈
    …
    …
    ≈
    

    View Slide

90. …as well as other covariates
    ≈
    …
    …
    …
    …
    ≈
    

    View Slide

91. How are subsequent posts evaluated?
    ≈
    …
    …
    …
    …
    ≈
    

    View Slide

92. How much are evaluations due to
    textual or community effects?
    

    View Slide

93. How much are evaluations due to
    textual effects (i.e., people writing worse)?
    f***ing a******
    i.e., downvoting because of post content
    

    View Slide

94. How much are evaluations due to
    community effects (i.e., inherent bias)?
    We dislike you.
    i.e., downvoting because of community dislikes author
    We dislike you.
    

    View Slide

95. Do people write better/worse after
    a positive/negative evaluation?
    Textual Effects
    

    View Slide

96. ≈
    …
    …
    …
    …
    Better/Worse?
    Do people write better/worse after
    a positive/negative evaluation?
    ≈
    

    View Slide

97. Text quality drops significantly after
    a negative evaluation…
    (p < 0.05, mean effect size r = 0.18)
    … …
    Negativity bias
    

    View Slide

98. …but doesn’t change after
    a positive evaluation
    … …
    (n.s.)
    Negativity bias
    

    View Slide

99. How does community bias
    change after an evaluation?
    Community Effects
    

    View Slide

100. … …
     … …
     How does community bias
     change after an evaluation?
     

     View Slide

101. Measuring community bias
     N↑ q
     N↓
     N↑

     N↑
     + N↓
     = 0.5
     p(c) =
     = 0.8
     q(c)
     p(c) q(c) = 0.3
     Prop. Upvotes
     Text Quality
     Community Bias − −
     

     View Slide

102. Community bias increase more after
     a negative than positive evaluation
     (p < 0.01, mean effect size r = 0.13)
     … …
     Halo effect
     

     View Slide

103. Troll or not?
     User
     Mood Other users
     { }
     

     View Slide

104. Troll or not?
     User
     Mood Other users
     { }
     

     View Slide

105. Troll or not?
     User
     Mood Other users
     { }
     

     View Slide

106. Trolls may start out normal, but tip
     into a spiral and never recover
     

     View Slide

107. Were downvotes a good idea in the
     first place?
     

     View Slide

108. Antisocial Computing
     Talk Outline
     What causes antisocial behavior?
     How do systems mediate it?
     Does it worsen over time?
     1
     2
     3
     WIP!
     

     View Slide

109. Work-in-Progress
     with J. Leskovec, T. Hsieh
     WIP!
     How Introducing Downvoting to Communities Impacts User Behavior
     CAN ENABLING
     NEGATIVE FEEDBACK
     HARM COMMUNITIES?
     

     View Slide

110. Downvoting causes negative
     behavior to worsen…
     Previously
     WIP!
     

     View Slide

111. …but does its mere presence
     affect communities?
     The Present Work
     WIP!
     

     View Slide

112. The ability to downvote
     negatively alters behavior
     Our Hypothesis
     WIP!
     

     View Slide

113. Challenge: how do we measure the
     impact of introducing downvoting?
     WIP!
     

     View Slide

114. Why the sudden increase in downvotes?
     Data: Worldstarhiphop circa. 2012
     # Votes
     0
     40000
     80000
     120000
     160000
     Time (Days)
     0 20 40 60 80 100 120 140 160 180 200 220
     Downvotes
     Upvotes
     ?
     WIP!
     

     View Slide

115. Early 2012 Late 2012
     Cause: a Disqus interface change
     WIP!
     

     View Slide

116. Early 2012 Late 2012
     Cause: a Disqus interface change
     +
     WIP!
     

     View Slide

117. A natural experiment studying the
     impact of introducing downvoting
     Opportunity
     

     View Slide

118. Twenty comment-based websites
     …
     …
     WIP!
     

     View Slide

119. How do we measure impact?
     WIP!
     Time (Weeks)
     0
     

     View Slide

120. How do we measure impact?
     WIP!
     Time (Weeks)
     1
     0
     

     View Slide

121. How do we measure impact?
     WIP!
     Time (Weeks)
     1 5
     -4 0
     Before After
     Repeat for each domain (x20), applying corrections as necessary
     

     View Slide

122. How did the introduction of
     downvoting affect user behavior?
     Votes Discussions Anonymity
     WIP!
     

     View Slide

123. Does introducing downvoting alter
     upvoting behavior?
     WIP!
     

     View Slide

124. Does introducing downvoting alter
     upvoting behavior?
     ?
     ?
     Downvotes don’t impact upvotes overall
     Upvotes increase to offset downvotes
     WIP!
     (Muchnik, et al. 2013)
     

     View Slide

125. Introducing downvoting has little
     effect on upvoting overall.
     (5 domains saw a significant increase / 12 saw no change / 3 saw a significant decrease, overall n.s.)
     WIP!
     

     View Slide

126. Does introducing downvoting
     spur discussion?
     WIP!
     

     View Slide

127. Does introducing downvoting
     spur discussion?
     ?
     ?
     Downvotes cause arguments
     Downvotes replace comments
     WIP!
     

     View Slide

128. Introducing downvoting
     spurs discussion.
     More replies, less
     top-level comments
     (13 + / 4 · / 3 —,
     overall increase [p < 0.01])
     WIP!
     

     View Slide

129. Introducing downvoting
     spurs discussion.
     More replies, less
     top-level comments
     Discussion length
     increases
     …
     (13 + / 4 · / 3 —,
     overall increase [p < 0.01])
     (15 + / 3 · / 2 —,
     overall increase [p < 0.01])
     WIP!
     

     View Slide

130. Introducing downvoting
     spurs discussion.
     More replies, less
     top-level comments
     Discussion length
     increases
     More back-and-forth
     discussion
     …
     @john…
     @mary…
     @john…
     @mary…
     (13 + / 4 · / 3 —,
     overall increase [p < 0.01])
     (15 + / 3 · / 2 —,
     overall increase [p < 0.01])
     (16 + / 2 · / 2 —,
     overall increase [p < 0.01])
     WIP!
     

     View Slide

131. Does introducing downvoting
     have a “chilling effect”?
     WIP!
     

     View Slide

132. Introducing downvoting
     increases anonymity.
     WIP!
     

     View Slide

133. Introducing downvoting
     increases anonymity.
     WIP!
     (18 sig. increase / 1 no change / 1 sig. decrease, overall increase [p < 0.01])
     

     View Slide

134. Negative feedback mechanisms can
     (unintentionally) encourage trolling
     

     View Slide

135. What we now know
     What we thought
     Trolls are a
     vocal minority
     Trolls can be
     ordinary people
     Trolling is innate
     Trolling can spiral from
     a single bad post
     Design mediates
     trolling behavior
     Design mediates
     a user’s experience
     ANTISOCIAL COMPUTING
     

     View Slide

136. Justin Cheng / @jcccf / clr3.com
     with Jure Leskovec, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Tim Hsieh
     ANTISOCIAL COMPUTING
     Explaining, Predicting, and Mediating Online Negative Behavior
     v1.1
     

     View Slide