Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Antisocial Behavior in Online Discussion Communities

Antisocial Behavior in Online Discussion Communities

User contributions in the form of posts, comments, and votes are essential to the success of online communities. However, allowing user participation also invites undesirable behavior such as trolling. Here, we characterize antisocial behavior in three large online discussion communities by analyzing users who were banned from these communities. We find that such users tend to concentrate their efforts in a small number of threads, are more likely to post irrelevantly, and are more successful at garnering responses from other users. Studying the evolution of these users from the moment they join a community up to when they get banned, we find that not only do they write worse than other users over time, but they also become increasingly less tolerated by the community. Further, we discover that antisocial behavior is exacerbated when community feedback is overly harsh. Our analysis also reveals distinct groups of users with different levels of antisocial behavior that can change over time. We use these insights to identify antisocial users early on, a task of high practical importance to community maintainers.

Presented at ICWSM 2015.

Justin Cheng

May 28, 2015
Tweet

More Decks by Justin Cheng

Other Decks in Research

Transcript

  1. ANTISOCIAL BEHAVIOR IN
    ONLINE COMMUNITIES
    Justin Cheng STANFORD
    Cristian Danescu-Niculesu-Mizil CORNELL
    Jure Leskovec STANFORD

    View Slide

  2. View Slide

  3. Warning
    This talk contains user posts with strong writing

    (profanity, sexism, racism, and religious intolerance).
    !

    View Slide

  4. View Slide

  5. It also shows that Islam and Christianity teaching
    women to dress modest could be right afterall.

    View Slide

  6. Religious nut alert
    Clearly that is the only logical conclusion to
    this article. Now if you'll excuse me, I need to
    iron my tarp. I have work on Monday, and I
    want to appear 'modest'.
    fail at life. go bomb yourself.
    It also shows that Islam and Christianity teaching
    women to dress modest could be right afterall.

    View Slide

  7. Both of these little skanks are ugly to the bone.

    View Slide

  8. Both of these little skanks are ugly to the bone.
    You’re a troll.

    View Slide

  9. Trolls disrupt online discussions
    Baker, P. (2001); Donath, J. S. (1999); Herring, S., et al. (2011); Shachaf, P. and Hara, N. (2010)

    View Slide

  10. View Slide

  11. Characterizing trolls in online
    discussion communities

    View Slide

  12. Characterizing trolls in online
    discussion communities
    How do trolls
    differ from

    non-trolls?
    1
    How do trolls
    change over
    time?
    2
    How do we
    predict troll-like
    behavior?
    3

    View Slide

  13. Characterizing trolls in online
    discussion communities
    How do trolls
    differ from

    non-trolls?
    1
    How do trolls
    change over
    time?
    2
    How do we
    predict troll-like
    behavior?
    3
    Large-scale Analysis
    Baker, P. (2001); Donath, J. S. (1999); Herring, S., et al. (2011); Shachaf, P. and Hara, N. (2010)

    View Slide

  14. Characterizing trolls in online
    discussion communities
    How do trolls
    differ from
    non-trolls?
    1
    How do trolls
    change over
    time?
    2
    How do we
    predict troll-like
    behavior?
    3
    Large-scale Analysis Longitudinal Study

    View Slide

  15. Characterizing trolls in online
    discussion communities
    How do trolls
    differ from
    non-trolls?
    1
    How do trolls
    change over
    time?
    2
    How do we
    predict troll-like
    behavior?
    3
    Large-scale Analysis Longitudinal Study Predictive Modeling
    Adler, B. T., et al. (2011); Wang, W. Y. and McKeown, K. R. (2010)

    View Slide

  16. How do we define trolling?

    View Slide

  17. What data are we using?
    18 months ~1.7M users ~40M posts ~100M votes

    View Slide

  18. Post Deletions

    by moderators
    2.0%
    (>500k)
    2.3%
    (>180k)
    2.7%
    (>110k)
    User Bans

    by moderators
    3.3%
    (>37k)
    1.7%
    (>5k)
    2.2%
    (>5k)
    How common is trolling?

    View Slide

  19. How do we define trolling?
    Engaging in negatively marked online behavior
    Taking pleasure in upsetting others
    Not following the rules
    Disrupting a group while staying undercover
    Donath, J. S. (1999); Hardaker, C. (2010); Kirman, B., et al. (2012); Schwartz, M. (2008)

    View Slide

  20. How do we define trolling?
    Troll

    User banned in the future.

    View Slide

  21. How do we define trolling?
    Troll

    User banned in the future.
    Non-troll

    User who was never banned.

    View Slide

  22. How do we define trolling?
    Troll

    User banned in the future.
    Non-troll

    User who was never banned, but is similarly active.
    (matched)

    View Slide

  23. Characterizing trolls in online
    discussion communities
    How do trolls
    differ from

    non-trolls?
    1
    How do trolls
    change over
    time?
    2
    How do we
    predict troll-like
    behavior?
    3
    Large-scale Analysis Longitudinal Study Predictive Modeling

    View Slide

  24. Penny, once again you show why you are one of the
    best of the league. Always a class-act…
    Fantastic story. Kudos to Penny Hardaway. This is
    what you call giving back.
    *sniff* This had me tearing up in the office. Great job
    Penny! You were always a class act…
    Ex-NBA star returns to inner city, brings hoop dreams

    View Slide

  25. stop reading then. Just sayin.. CNN not making
    you read...you chose to.
    How many white NBA players grew up in the
    Why…do I not see any articles similar to this about
    white NBA basketball players?......every single touchy
    feely story is about a black ball player............YOU
    GUYS MAKE ME SICK AS A READER !
    what you call giving back.
    Less similar to previous posts

    9% less similar (cosine similarity), p<10-4
    Penny, once again you show why you are one of the
    best of the league. Always a class-act…

    View Slide

  26. charitable efforts by white NBA players, but
    this isn't a story about a Black NBA player...it's
    a story about someone who came back to their
    roots to contribute.
    b\c young black men need to see examples
    from their own race. They need to see that
    even minorities can succeed and give back
    instead. They connect with people of their own
    race. It allows for mentorship and guidance.
    A story about a millionaire helping kids in his
    poor neighborhood personally, and being a
    positive role model makes you sick as a reader.
    Gotta love conservatives.
    Get more replies from other users

    Twice as many replies, p<10-2

    View Slide

  27. If you claim you want less government but want to control the bedroom,you're a
    Republican; If you want to cut Education, you're a Republican; If you want to cut Social
    Security, you're a Republican; If you want to cut Medicare and Medicaid, you're a
    If you claim you want less government but want to control the bedroom,you're a
    Republican; If you want to cut Education, you're a Republican; If you want to cut Social
    Security, you're a Republican; If you want to cut Medicare and Medicaid, you're a
    If you claim you want less government but want to control the bedroom,you're a
    Republican; If you want to cut Education, you're a Republican; If you want to cut Social
    Security, you're a Republican; If you want to cut Medicare and Medicaid, you're a
    If you claim you want less government but want to control the
    bedroom,you're a Republican; If you want to cut Education, you're a
    If you claim you want less government but want to control the
    bedroom,you're a Republican; If you want to cut Education, you're a
    If you claim you want less government but want to control the
    bedroom,you're a Republican; If you want to cut Education, you're a
    If you claim you want less government but want to control the bedroom,you're a
    Republican; If you want to cut Education, you're a Republican; If you want to cut Social
    Security, you're a Republican; If you want to cut Medicare and Medicaid, you're a
    If you claim you want less government but want to control the bedroom,you're a
    Republican; If you want to cut Education, you're a Republican; If you want to cut Social
    Security, you're a Republican; If you want to cut Medicare and Medicaid, you're a
    If you claim you want less government but want to control the bedroom,you're a
    Post more per thread

    47% more posts per thread, p<10-2

    View Slide

  28. Characterizing trolls in online
    discussion communities
    How do trolls
    differ from
    non-trolls?
    1
    How do trolls
    change over
    time?
    2
    How do we
    predict troll-like
    behavior?
    3
    Large-scale Analysis Longitudinal Study Predictive Modeling

    View Slide

  29. A troll’s posts are deleted more
    Post Deletion Rate
    0
    0.2
    0.4
    0.6
    0.8
    1
    Time (Normalized)
    0.1 0.3 0.5 0.7 0.9
    Trolls
    Non-Trolls

    View Slide

  30. Why a troll’s posts are deleted more
    Trolls write worse posts over time.
    Trolls start out worse than non trolls, and worsen more over time.
    1 p<0.05

    View Slide

  31. Why a troll’s posts are deleted more
    Trolls write worse posts over time.
    Trolls start out worse than non trolls, and worsen more over time.
    Communities become less tolerant of trolls.
    A troll’s posts are more likely to be deleted later in their life.
    1
    2
    p<0.05
    p<10-4

    View Slide

  32. Why a troll’s posts are deleted more
    Trolls write worse posts over time.
    Trolls start out worse than non trolls, and worsen more over time.
    Communities become less tolerant of trolls.
    A troll’s posts are more likely to be deleted later in their life.
    Communities can exacerbate trolling.
    Unfairly deleting a user’s posts causes them to write worse later.
    1
    2
    3
    p<0.05
    p<10-4
    p<0.05

    View Slide

  33. Why a troll’s posts are deleted more
    Trolls write worse posts over time.
    Trolls start out worse than non trolls, and worsen more over time.
    Communities become less tolerant of trolls.
    A troll’s posts are more likely to be deleted later in their life.
    Communities can exacerbate trolling.
    Unfairly deleting a user’s posts causes them to write worse later.
    1
    2
    3
    ICWSM 2014
    >
    p<0.05
    p<10-4
    p<0.05

    View Slide

  34. Characterizing trolls in online
    discussion communities
    How do trolls
    differ from
    non-trolls?
    1
    How do trolls
    change over
    time?
    2
    How do we
    predict troll-like
    behavior?
    3
    Large-scale Analysis Longitudinal Study Predictive Modeling

    View Slide

  35. Can we predict whether a user
    will get banned in the future?
    First 10 Posts Balanced Dataset of Trolls and Non-Trolls

    View Slide

  36. Prediction results on CNN
    Bag of Words
    ROC AUC
    0.5 0.6 0.7 0.8 0.9
    0.70

    View Slide

  37. Prediction results on CNN
    Bag of Words
    Post Deletion Rate
    ROC AUC
    0.5 0.6 0.7 0.8 0.9
    0.74
    0.70
    (Manual)

    View Slide

  38. Prediction results on CNN
    Bag of Words
    Post Deletion Rate
    ROC AUC
    0.5 0.6 0.7 0.8 0.9
    0.83
    0.74
    0.70
    (Automatic)
    (Manual)
    Our Approach

    View Slide

  39. Our automatic approach
    generalizes across communities.
    Uses Interaction Patterns, Not Language
    Cross-Domain AUC = 0.68

    View Slide

  40. Conclusion
    Trolls differ from non-trolls in their language and
    behavior.
    Trolls change over time, and the community plays a
    role in exacerbating trolling.
    A user’s initial posting activity can effectively predict
    whether that user will be subsequently banned.

    View Slide

  41. Social Media

    View Slide

  42. Anti Social Media

    View Slide

  43. END OF PART 1

    View Slide

  44. END OF PART 1

    View Slide

  45. ANTISOCIAL BEHAVIOR IN
    ONLINE COMMUNITIES
    Justin Cheng STANFORD
    Cristian Danescu-Niculesu-Mizil CORNELL
    Jure Leskovec STANFORD http://bit.ly/trolls-paper

    View Slide