Antisocial Behavior in Online Discussion Communities

User contributions in the form of posts, comments, and votes are essential to the success of online communities. However, allowing user participation also invites undesirable behavior such as trolling. Here, we characterize antisocial behavior in three large online discussion communities by analyzing users who were banned from these communities. We find that such users tend to concentrate their efforts in a small number of threads, are more likely to post irrelevantly, and are more successful at garnering responses from other users. Studying the evolution of these users from the moment they join a community up to when they get banned, we find that not only do they write worse than other users over time, but they also become increasingly less tolerated by the community. Further, we discover that antisocial behavior is exacerbated when community feedback is overly harsh. Our analysis also reveals distinct groups of users with different levels of antisocial behavior that can change over time. We use these insights to identify antisocial users early on, a task of high practical importance to community maintainers.

Presented at ICWSM 2015.

Transcript

1. ANTISOCIAL BEHAVIOR IN ONLINE COMMUNITIES Justin Cheng STANFORD Cristian Danescu-Niculesu-Mizil CORNELL Jure Leskovec STANFORD
3. Warning This talk contains user posts with strong writing
 (profanity, sexism, racism, and religious intolerance). !
5. It also shows that Islam and Christianity teaching women to dress modest could be right afterall.
6. Religious nut alert Clearly that is the only logical conclusion to this article. Now if you'll excuse me, I need to iron my tarp. I have work on Monday, and I want to appear 'modest'. fail at life. go bomb yourself. It also shows that Islam and Christianity teaching women to dress modest could be right afterall.
7. Both of these little skanks are ugly to the bone.
8. Both of these little skanks are ugly to the bone. You’re a troll.
9. Trolls disrupt online discussions Baker, P. (2001); Donath, J. S. (1999); Herring, S., et al. (2011); Shachaf, P. and Hara, N. (2010)
11. Characterizing trolls in online discussion communities
12. Characterizing trolls in online discussion communities How do trolls differ from
 non-trolls? 1 How do trolls change over time? 2 How do we predict troll-like behavior? 3
13. Characterizing trolls in online discussion communities How do trolls differ from
 non-trolls? 1 How do trolls change over time? 2 How do we predict troll-like behavior? 3 Large-scale Analysis Baker, P. (2001); Donath, J. S. (1999); Herring, S., et al. (2011); Shachaf, P. and Hara, N. (2010)
14. Characterizing trolls in online discussion communities How do trolls differ from non-trolls? 1 How do trolls change over time? 2 How do we predict troll-like behavior? 3 Large-scale Analysis Longitudinal Study
15. Characterizing trolls in online discussion communities How do trolls differ from non-trolls? 1 How do trolls change over time? 2 How do we predict troll-like behavior? 3 Large-scale Analysis Longitudinal Study Predictive Modeling Adler, B. T., et al. (2011); Wang, W. Y. and McKeown, K. R. (2010)
16. How do we define trolling?
17. What data are we using? 18 months ~1.7M users ~40M posts ~100M votes
18. Post Deletions
 by moderators 2.0% (>500k) 2.3% (>180k) 2.7% (>110k) User Bans
 by moderators 3.3% (>37k) 1.7% (>5k) 2.2% (>5k) How common is trolling?
19. How do we define trolling? Engaging in negatively marked online behavior Taking pleasure in upsetting others Not following the rules Disrupting a group while staying undercover Donath, J. S. (1999); Hardaker, C. (2010); Kirman, B., et al. (2012); Schwartz, M. (2008)
20. How do we define trolling? Troll
 User banned in the future.
21. How do we define trolling? Troll
 User banned in the future. Non-troll
 User who was never banned.
22. How do we define trolling? Troll
 User banned in the future. Non-troll
 User who was never banned, but is similarly active. (matched)
23. Characterizing trolls in online discussion communities How do trolls differ from
 non-trolls? 1 How do trolls change over time? 2 How do we predict troll-like behavior? 3 Large-scale Analysis Longitudinal Study Predictive Modeling
24. Penny, once again you show why you are one of the best of the league. Always a class-act… Fantastic story. Kudos to Penny Hardaway. This is what you call giving back. *sniff* This had me tearing up in the office. Great job Penny! You were always a class act… Ex-NBA star returns to inner city, brings hoop dreams
25. stop reading then. Just sayin.. CNN not making you read...you chose to. How many white NBA players grew up in the Why…do I not see any articles similar to this about white NBA basketball players?......every single touchy feely story is about a black ball player............YOU GUYS MAKE ME SICK AS A READER ! what you call giving back. Less similar to previous posts
 9% less similar (cosine similarity), p<10-4 Penny, once again you show why you are one of the best of the league. Always a class-act…
26. charitable efforts by white NBA players, but this isn't a story about a Black NBA player...it's a story about someone who came back to their roots to contribute. b\c young black men need to see examples from their own race. They need to see that even minorities can succeed and give back instead. They connect with people of their own race. It allows for mentorship and guidance. A story about a millionaire helping kids in his poor neighborhood personally, and being a positive role model makes you sick as a reader. Gotta love conservatives. Get more replies from other users
 Twice as many replies, p<10-2
27. If you claim you want less government but want to control the bedroom,you're a Republican; If you want to cut Education, you're a Republican; If you want to cut Social Security, you're a Republican; If you want to cut Medicare and Medicaid, you're a If you claim you want less government but want to control the bedroom,you're a Republican; If you want to cut Education, you're a Republican; If you want to cut Social Security, you're a Republican; If you want to cut Medicare and Medicaid, you're a If you claim you want less government but want to control the bedroom,you're a Republican; If you want to cut Education, you're a Republican; If you want to cut Social Security, you're a Republican; If you want to cut Medicare and Medicaid, you're a If you claim you want less government but want to control the bedroom,you're a Republican; If you want to cut Education, you're a If you claim you want less government but want to control the bedroom,you're a Republican; If you want to cut Education, you're a If you claim you want less government but want to control the bedroom,you're a Republican; If you want to cut Education, you're a If you claim you want less government but want to control the bedroom,you're a Republican; If you want to cut Education, you're a Republican; If you want to cut Social Security, you're a Republican; If you want to cut Medicare and Medicaid, you're a If you claim you want less government but want to control the bedroom,you're a Republican; If you want to cut Education, you're a Republican; If you want to cut Social Security, you're a Republican; If you want to cut Medicare and Medicaid, you're a If you claim you want less government but want to control the bedroom,you're a Post more per thread
 47% more posts per thread, p<10-2
28. Characterizing trolls in online discussion communities How do trolls differ from non-trolls? 1 How do trolls change over time? 2 How do we predict troll-like behavior? 3 Large-scale Analysis Longitudinal Study Predictive Modeling
29. A troll’s posts are deleted more Post Deletion Rate 0 0.2 0.4 0.6 0.8 1 Time (Normalized) 0.1 0.3 0.5 0.7 0.9 Trolls Non-Trolls
30. Why a troll’s posts are deleted more Trolls write worse posts over time. Trolls start out worse than non trolls, and worsen more over time. 1 p<0.05
31. Why a troll’s posts are deleted more Trolls write worse posts over time. Trolls start out worse than non trolls, and worsen more over time. Communities become less tolerant of trolls. A troll’s posts are more likely to be deleted later in their life. 1 2 p<0.05 p<10-4
32. Why a troll’s posts are deleted more Trolls write worse posts over time. Trolls start out worse than non trolls, and worsen more over time. Communities become less tolerant of trolls. A troll’s posts are more likely to be deleted later in their life. Communities can exacerbate trolling. Unfairly deleting a user’s posts causes them to write worse later. 1 2 3 p<0.05 p<10-4 p<0.05
33. Why a troll’s posts are deleted more Trolls write worse posts over time. Trolls start out worse than non trolls, and worsen more over time. Communities become less tolerant of trolls. A troll’s posts are more likely to be deleted later in their life. Communities can exacerbate trolling. Unfairly deleting a user’s posts causes them to write worse later. 1 2 3 ICWSM 2014 > p<0.05 p<10-4 p<0.05
34. Characterizing trolls in online discussion communities How do trolls differ from non-trolls? 1 How do trolls change over time? 2 How do we predict troll-like behavior? 3 Large-scale Analysis Longitudinal Study Predictive Modeling
35. Can we predict whether a user will get banned in the future? First 10 Posts Balanced Dataset of Trolls and Non-Trolls
36. Prediction results on CNN Bag of Words ROC AUC 0.5 0.6 0.7 0.8 0.9 0.70
37. Prediction results on CNN Bag of Words Post Deletion Rate ROC AUC 0.5 0.6 0.7 0.8 0.9 0.74 0.70 (Manual)
38. Prediction results on CNN Bag of Words Post Deletion Rate ROC AUC 0.5 0.6 0.7 0.8 0.9 0.83 0.74 0.70 (Automatic) (Manual) Our Approach
39. Our automatic approach generalizes across communities. Uses Interaction Patterns, Not Language Cross-Domain AUC = 0.68
40. Conclusion Trolls differ from non-trolls in their language and behavior. Trolls change over time, and the community plays a role in exacerbating trolling. A user’s initial posting activity can effectively predict whether that user will be subsequently banned.
41. Social Media
42. Anti Social Media
43. END OF PART 1
44. END OF PART 1
45. ANTISOCIAL BEHAVIOR IN ONLINE COMMUNITIES Justin Cheng STANFORD Cristian Danescu-Niculesu-Mizil CORNELL Jure Leskovec STANFORD http://bit.ly/trolls-paper