Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Antisocial Computing

Justin Cheng
February 02, 2018

Antisocial Computing

Justin Cheng

February 02, 2018
Tweet

More Decks by Justin Cheng

Other Decks in Research

Transcript

  1. Justin Cheng / @jcccf / clr3.com
    with Jure Leskovec, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Tim Hsieh
    ANTISOCIAL COMPUTING
    Explaining, Predicting, and Mediating Online Negative Behavior
    v1.1

    View Slide

  2. Vieweg, et al. (2010); Kittur, et al. (2013); Burke & Kraut (2016)

    View Slide

  3. View Slide

  4. 47% of online users
    have been harassed
    Data & Society (2017)

    View Slide

  5. Popular Science (2013); The Verge (2015); Chicago Sun-Times (2014)

    View Slide

  6. Why is bad behavior so prevalent?

    (╯°□°)╯︵ ┻━┻
    Research Question

    View Slide

  7. Understanding bad behavior helps
    us build healthier communities
    Implications
    Systems
    Guidelines Interventions

    View Slide

  8. Antisocial behavior is largely
    due to sociopaths
    Prior Work
    Donath (1999); Hardaker (2010); Buckels, et al. (2014)

    View Slide

  9. Antisocial behavior is largely
    due to ordinary people
    This Work

    View Slide

  10. Antisocial Computing
    Talk Outline
    What causes antisocial behavior?
    How do systems mediate it?
    Does it worsen over time?
    1
    2
    3
    WIP!

    View Slide

  11. Antisocial Computing
    Talk Outline
    What causes antisocial behavior?
    How do systems mediate it?
    Does it worsen over time?
    1
    2
    3
    WIP!

    View Slide

  12. CSCW 2017 (Best Paper); ICWSM 2015 (Honorable Mention)
    with M. Bernstein, C. Danescu-Niculescu-Mizil, J. Leskovec
    CAN ANYONE
    BECOME A TROLL?
    Causes of Antisocial Behavior in Online Discussions

    View Slide

  13. CONTENT WARNING!

    This talk contains depictions of trolling that use strong language.
    !

    View Slide

  14. View Slide

  15. It also shows that Islam and
    Christianity teaching women to
    dress modest could be right
    afterall.

    View Slide

  16. It also shows that Islam and
    Christianity teaching women to
    dress modest could be right
    afterall.
    Clearly that is the only logical
    conclusion to this article. Now if
    you'll excuse me, I need to iron
    my tarp. I have work on Monday,
    and I want to appear 'modest'.
    fail at life. go bomb yourself.
    Religious nut alert

    View Slide

  17. We studied multiple large comment-
    based news communities.
    470M posts 831M votes
    76M users

    View Slide

  18. What is trolling?

    View Slide

  19. What is trolling?
    Engaging in negatively marked online
    behavior
    Taking pleasure in upsetting others
    Not following the rules
    Disrupting a group while staying
    undercover
    Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008)

    View Slide

  20. Trolling is behavior that occurs
    outside community norms.
    Defined using community guidelines
    Our Definition
    e.g., name-calling, personal attacks, profanity, threats, hate speech, ethnically/racially offensive material

    View Slide

  21. Are trolls just a vocal minority?
    Donath (1999); Hardaker (2010); Shachaf & Hara (2010); NYT (2008); Wired (2014); Vox (2014)

    View Slide

  22. How much do trolls troll?
    Proportion of Banned Users
    0
    0.1
    0.2
    0.3
    0.4
    Proportion of Deleted Posts
    0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

    View Slide

  23. The distribution of trolls is bimodal
    Proportion of Banned Users
    0
    0.1
    0.2
    0.3
    0.4
    Proportion of Deleted Posts
    0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

    View Slide

  24. Are there two types of trolls?
    Proportion of Banned Users
    0
    0.1
    0.2
    0.3
    0.4
    Proportion of Deleted Posts
    0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
    Situational trolling?
    Lifelong trolling?

    View Slide

  25. What if antisocial behavior is
    situational?

    View Slide

  26. Challenge: how to show that
    antisocial behavior is situational?
    Observational data isn’t causal

    View Slide

  27. Challenge: how to show that
    antisocial behavior is situational?
    Experiments hard to generalize

    View Slide

  28. Simulated Discussion Experiment Large-Scale Analysis
    Solution: Experiment + Observational Study

    View Slide

  29. Anyone can become a troll
    Our Hypothesis

    View Slide

  30. “Broken windows” theory
    Zimbardo (1969); Wilson (1982)

    View Slide

  31. Unpleasant stimuli increase aggression
    Jones & Bogat (1978); Rotton & Frey (1985)

    View Slide

  32. Experiment: simulated discussion forum

    View Slide

  33. N=667, 40% female
    Quiz Discussion
    Experimental method

    View Slide

  34. Quiz Discussion
    Experimental method
    ×

    View Slide

  35. Positive/Negative Mood Positive/Negative Context
    Experimental method
    ×
    Quiz Discussion

    View Slide

  36. Easy quiz (positive mood)

    View Slide

  37. Difficult quiz (negative mood)

    View Slide

  38. Positive discussion context

    View Slide

  39. Negative discussion context

    View Slide

  40. How did trolling differ across
    conditions?
    Two expert raters labeled posts independently

    View Slide

  41. How did trolling differ across conditions?
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    Negative
    Context
    % Troll Posts

    View Slide

  42. Trolling is lowest with positive conditions…
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    35%
    Negative
    Context
    % Troll Posts

    View Slide

  43. …increases with either negative condition…
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    35% 49%
    Negative
    Context
    47%
    % Troll Posts

    View Slide

  44. …and almost doubles in the worst case
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    35% 49%
    Negative
    Context
    47% 68%
    % Troll Posts
    (p < 0.05 using a mixed effects logistic regression model)

    View Slide

  45. Negative affect almost triples
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    1.1% 1.4%
    Negative
    Context
    2.3% 2.9%
    % Negative Affect Words (LIWC)
    (p < 0.05)

    View Slide

  46. Hilary is a solid candidate. As a woman, I
    appreciate that she's a woman, but it's not
    the only reason I think she would do well in
    office.
    Positive Mood + Context

    View Slide

  47. Anyone who votes for her is a complete
    idiot. These supporters are why this country
    is in such bad shape now. Uneducated
    people.
    Negative Mood + Context

    View Slide

  48. Bad mood and negative discussion
    context increase trolling

    View Slide

  49. Simulated Discussion Experiment Large-Scale Analysis of CNN.com
    Online Experiment + Observational Study

    View Slide

  50. Can trolling, like mood, vary with
    the time of day and day of week?
    Replicating Mood
    Golder & Macy (2011)
    Neg. Affect
    Time of Day
    ?

    View Slide

  51. How does trolling vary with time of day?
    Proportion of Flagged Posts
    0.03
    0.033
    0.036
    0.039
    0.042
    Time of Day
    0 6 12 18 24

    View Slide

  52. Trolling peaks in the evening…
    Proportion of Flagged Posts
    0.03
    0.033
    0.036
    0.039
    0.042
    Time of Day
    0 6 12 18 24
    Negative Affect
    (Golder & Macy)

    View Slide

  53. …and early in the work week.
    Proportion of Flagged Posts
    0.03
    0.032
    0.034
    0.036
    0.038
    0.04
    Day of Week
    Mon Tue Wed Thu Fri Sat Sun

    View Slide

  54. Trolling peaks when moods are worse
    Time of Day
    Proportion of
    Flagged Posts
    Negative Affect
    Proportion of
    Downvotes
    Day of Week

    View Slide

  55. Mood spills over from prior
    discussions
    Replicating Mood

    View Slide



  56. (Discussion)
    Mood spills over from prior
    discussions

    View Slide




  57. ?

    (Discussion)
    (Unrelated Discussions)

    ?

    Mood spills over from prior
    discussions

    View Slide





  58. (Discussion)


    Mood spills over from prior
    discussions

    View Slide

  59. A user who trolled in a previous
    discussion is twice as likely to troll
    in a later, unrelated discussion
    (p < 0.01)
    Replicating Mood

    View Slide

  60. The initial post affects subsequent
    trolling
    Replicating Context

    View Slide

  61. (Separate discussions of same article)

    ? ? ?

    ? ? ?
    The initial post affects subsequent
    trolling

    View Slide

  62. (Separate discussions of same article)


    The initial post affects subsequent
    trolling

    View Slide

  63. An initial troll post increases the
    subsequent trolling by 63%
    (p < 0.01)
    Replicating Context

    View Slide

  64. Troll or not?
    User

    View Slide

  65. Troll or not?
    User
    Mood

    View Slide

  66. Troll or not?
    User
    Mood Other users
    { }

    View Slide

  67. Because trolling is situational,
    ordinary people can end up trolling

    View Slide

  68. View Slide

  69. Can voting mitigate bad behavior?

    View Slide

  70. Downvoting causes
    negative behavior to worsen
    Our Hypothesis

    View Slide

  71. Antisocial Computing
    Talk Outline
    What causes antisocial behavior?
    How do systems mediate it?
    Does it worsen over time?
    1
    2
    3
    WIP!

    View Slide

  72. How Antisocial Behavior Worsens
    ICWSM 2014
    with C. Danescu-Niculescu-Mizil, J. Leskovec
    CAN ANTISOCIAL
    BEHAVIOR SPI
    R
    A
    L?

    View Slide

  73. Downvoting causes
    negative behavior to worsen
    Our Hypothesis

    View Slide

  74. What effects do evaluations have?
    Positively
    evaluated
    ?
    ?
    Negatively
    evaluated

    View Slide

  75. What is a positive or negative evaluation?

    View Slide

  76. Defining positive and negative evaluations
    :
    9 1
    2 8
    N↑

    N↑
    + N↓
    2

    2+8
    = = 0.2 p↓

    : N↑

    N↑
    + N↓
    9

    9+1
    = = 0.9 p↑

    Positive Evaluation
    Negative Evaluation
    (validated using a crowdsourcing experiment)

    View Slide

  77. What effects do evaluations have?
    Positively
    evaluated
    ?
    ?
    Negatively
    evaluated

    View Slide

  78. Does feedback encourage better behavior?
    Skinner (1938)

    View Slide

  79. Or is bad stronger than good?
    Brinko (1993); Baumeister, et al. (2001)

    View Slide

  80. Four large comment-based news
    communities

    View Slide

  81. What effects do evaluations have?
    Positively
    evaluated
    Negatively
    evaluated

    View Slide

  82. What effects do evaluations have?




    View Slide

  83. What effects do evaluations have?




    Before After
    vs.
    Before After
    vs.

    View Slide

  84. Challenge: how to compare
    different users and posts?
    Aren’t downvoted users/posts inherently worse?

    View Slide

  85. Solution: propensity score matching
    PSM: Rosenbaum (1983); CEM: Iacus, et al. (2012)
    Positively
    evaluated
    Negatively
    evaluated

    View Slide

  86. Match on text quality
    Similar text
    quality q
    }

    View Slide

  87. Computing text quality
    (Validated using crowdsourcing)
    Learn p with bigrams
    1 3
    lorem ipsum…
    q = ?
    Lorem…
    ? ?
    9 2
    lorem ipsum…

    Text quality q is
    predicted p

    View Slide

  88. Match on text quality
    Similar text
    quality
    q(c↑
    )=q(c↓
    )
    }

    View Slide

  89. …as well as other covariates
    Similar history
    (# posts, overall
    proportion of
    upvotes, etc.)
    { ≈



    View Slide

  90. …as well as other covariates






    View Slide

  91. How are subsequent posts evaluated?






    View Slide

  92. How much are evaluations due to
    textual or community effects?

    View Slide

  93. How much are evaluations due to
    textual effects (i.e., people writing worse)?
    f***ing a******
    i.e., downvoting because of post content

    View Slide

  94. How much are evaluations due to
    community effects (i.e., inherent bias)?
    We dislike you.
    i.e., downvoting because of community dislikes author
    We dislike you.

    View Slide

  95. Do people write better/worse after
    a positive/negative evaluation?
    Textual Effects

    View Slide






  96. Better/Worse?
    Do people write better/worse after
    a positive/negative evaluation?

    View Slide

  97. Text quality drops significantly after
    a negative evaluation…
    (p < 0.05, mean effect size r = 0.18)
    … …
    Negativity bias

    View Slide

  98. …but doesn’t change after
    a positive evaluation
    … …
    (n.s.)
    Negativity bias

    View Slide

  99. How does community bias
    change after an evaluation?
    Community Effects

    View Slide

  100. … …
    … …
    How does community bias
    change after an evaluation?

    View Slide

  101. Measuring community bias
    N↑ q
    N↓
    N↑

    N↑
    + N↓
    = 0.5
    p(c) =
    = 0.8
    q(c)
    p(c) q(c) = 0.3
    Prop. Upvotes
    Text Quality
    Community Bias − −

    View Slide

  102. Community bias increase more after
    a negative than positive evaluation
    (p < 0.01, mean effect size r = 0.13)
    … …
    Halo effect

    View Slide

  103. Troll or not?
    User
    Mood Other users
    { }

    View Slide

  104. Troll or not?
    User
    Mood Other users
    { }

    View Slide

  105. Troll or not?
    User
    Mood Other users
    { }

    View Slide

  106. Trolls may start out normal, but tip
    into a spiral and never recover

    View Slide

  107. Were downvotes a good idea in the
    first place?

    View Slide

  108. Antisocial Computing
    Talk Outline
    What causes antisocial behavior?
    How do systems mediate it?
    Does it worsen over time?
    1
    2
    3
    WIP!

    View Slide

  109. Work-in-Progress
    with J. Leskovec, T. Hsieh
    WIP!
    How Introducing Downvoting to Communities Impacts User Behavior
    CAN ENABLING
    NEGATIVE FEEDBACK
    HARM COMMUNITIES?

    View Slide

  110. Downvoting causes negative
    behavior to worsen…
    Previously
    WIP!

    View Slide

  111. …but does its mere presence
    affect communities?
    The Present Work
    WIP!

    View Slide

  112. The ability to downvote
    negatively alters behavior
    Our Hypothesis
    WIP!

    View Slide

  113. Challenge: how do we measure the
    impact of introducing downvoting?
    WIP!

    View Slide

  114. Why the sudden increase in downvotes?
    Data: Worldstarhiphop circa. 2012
    # Votes
    0
    40000
    80000
    120000
    160000
    Time (Days)
    0 20 40 60 80 100 120 140 160 180 200 220
    Downvotes
    Upvotes
    ?
    WIP!

    View Slide

  115. Early 2012 Late 2012
    Cause: a Disqus interface change
    WIP!

    View Slide

  116. Early 2012 Late 2012
    Cause: a Disqus interface change
    +
    WIP!

    View Slide

  117. A natural experiment studying the
    impact of introducing downvoting
    Opportunity

    View Slide

  118. Twenty comment-based websites


    WIP!

    View Slide

  119. How do we measure impact?
    WIP!
    Time (Weeks)
    0

    View Slide

  120. How do we measure impact?
    WIP!
    Time (Weeks)
    1
    0

    View Slide

  121. How do we measure impact?
    WIP!
    Time (Weeks)
    1 5
    -4 0
    Before After
    Repeat for each domain (x20), applying corrections as necessary

    View Slide

  122. How did the introduction of
    downvoting affect user behavior?
    Votes Discussions Anonymity
    WIP!

    View Slide

  123. Does introducing downvoting alter
    upvoting behavior?
    WIP!

    View Slide

  124. Does introducing downvoting alter
    upvoting behavior?
    ?
    ?
    Downvotes don’t impact upvotes overall
    Upvotes increase to offset downvotes
    WIP!
    (Muchnik, et al. 2013)

    View Slide

  125. Introducing downvoting has little
    effect on upvoting overall.
    (5 domains saw a significant increase / 12 saw no change / 3 saw a significant decrease, overall n.s.)
    WIP!

    View Slide

  126. Does introducing downvoting
    spur discussion?
    WIP!

    View Slide

  127. Does introducing downvoting
    spur discussion?
    ?
    ?
    Downvotes cause arguments
    Downvotes replace comments
    WIP!

    View Slide

  128. Introducing downvoting
    spurs discussion.
    More replies, less
    top-level comments
    (13 + / 4 · / 3 —,
    overall increase [p < 0.01])
    WIP!

    View Slide

  129. Introducing downvoting
    spurs discussion.
    More replies, less
    top-level comments
    Discussion length
    increases

    (13 + / 4 · / 3 —,
    overall increase [p < 0.01])
    (15 + / 3 · / 2 —,
    overall increase [p < 0.01])
    WIP!

    View Slide

  130. Introducing downvoting
    spurs discussion.
    More replies, less
    top-level comments
    Discussion length
    increases
    More back-and-forth
    discussion

    @john…
    @mary…
    @john…
    @mary…
    (13 + / 4 · / 3 —,
    overall increase [p < 0.01])
    (15 + / 3 · / 2 —,
    overall increase [p < 0.01])
    (16 + / 2 · / 2 —,
    overall increase [p < 0.01])
    WIP!

    View Slide

  131. Does introducing downvoting
    have a “chilling effect”?
    WIP!

    View Slide

  132. Introducing downvoting
    increases anonymity.
    WIP!

    View Slide

  133. Introducing downvoting
    increases anonymity.
    WIP!
    (18 sig. increase / 1 no change / 1 sig. decrease, overall increase [p < 0.01])

    View Slide

  134. Negative feedback mechanisms can
    (unintentionally) encourage trolling

    View Slide

  135. What we now know
    What we thought
    Trolls are a
    vocal minority
    Trolls can be
    ordinary people
    Trolling is innate
    Trolling can spiral from
    a single bad post
    Design mediates
    trolling behavior
    Design mediates
    a user’s experience
    ANTISOCIAL COMPUTING

    View Slide

  136. Justin Cheng / @jcccf / clr3.com
    with Jure Leskovec, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Tim Hsieh
    ANTISOCIAL COMPUTING
    Explaining, Predicting, and Mediating Online Negative Behavior
    v1.1

    View Slide