Antisocial Computing - Speaker Deck

Slide 1

Slide 1 text

Justin Cheng Stanford University ANTISOCIAL COMPUTING Explaining and Predicting Negative Behavior Online

Slide 2

Slide 2 text

Vieweg, et al. (2010); Kittur, et al. (2013); Burke & Kraut (2016)

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Time (2016); The Atlantic (2016); Vanity Fair (2017)

Slide 5

Slide 5 text

47% of online users have been harassed Data & Society (2017)

Slide 6

Slide 6 text

Popular Science (2013); The Verge (2015); Chicago Sun-Times (2014)

Slide 7

Slide 7 text

Why is bad behavior so prevalent?  (›°□°)›ớ ᵲᴸᵲ Research Question

Slide 8

Slide 8 text

Understanding bad behavior helps us build healthier communities Implications Systems Guidelines Interventions

Slide 9

Slide 9 text

Antisocial behavior is largely due to sociopaths Prior Work Donath (1999); Hardaker (2010); Buckels, et al. (2014)

Slide 10

Slide 10 text

Antisocial behavior is largely due to ordinary people This Work

Slide 11

Slide 11 text

Antisocial Behavior & Its Spread Talk Outline What causes antisocial behavior? Can such cascades be predicted? Does it worsen over time? 1 2 3

Slide 12

Slide 12 text

Data Mining + Crowdsourcing Research Approach Large-scale Analysis + Experiments

Slide 13

Slide 13 text

Identifying principles of online behavior The Broader Picture Data + ML + Network Science + HCI

Slide 14

Slide 14 text

Antisocial Behavior & Its Spread Talk Outline 1 2 3 What causes antisocial behavior? Can such cascades be predicted? Does it worsen over time?

Slide 15

Slide 15 text

CSCW 2017 (Best Paper); ICWSM 2015 (Honorable Mention) with M. Bernstein, C. Danescu-Niculescu-Mizil, J. Leskovec CAN ANYONE BECOME A TROLL? Causes of Antisocial Behavior in Online Discussions

Slide 16

Slide 16 text

CONTENT WARNING!  This talk contains depictions of trolling that use strong language. !

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

It also shows that Islam and Christianity teaching women to dress modest could be right afterall.

Slide 19

Slide 19 text

It also shows that Islam and Christianity teaching women to dress modest could be right afterall. Clearly that is the only logical conclusion to this article. Now if you'll excuse me, I need to iron my tarp. I have work on Monday, and I want to appear 'modest'. fail at life. go bomb yourself. Religious nut alert

Slide 20

Slide 20 text

We studied multiple large comment- based news communities. 470M posts 831M votes 76M users

Slide 21

Slide 21 text

What is trolling?

Slide 22

Slide 22 text

What is trolling? Engaging in negatively marked online behavior Taking pleasure in upsetting others Not following the rules Disrupting a group while staying undercover Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008)

Slide 23

Slide 23 text

Trolling is behavior that occurs outside community norms. Deﬁned using community guidelines Our Deﬁnition e.g., name-calling, personal attacks, profanity, threats, hate speech, ethnically/racially offensive material

Slide 24

Slide 24 text

Are trolls just a vocal minority? Donath (1999); Hardaker (2010); Shachaf & Hara (2010); NYT (2008); Wired (2014); Vox (2014)

Slide 25

Slide 25 text

How much do trolls troll? Proportion of Banned Users 0 0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Slide 26

Slide 26 text

The distribution of trolls is bimodal Proportion of Banned Users 0 0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Slide 27

Slide 27 text

Are there two types of trolls? Proportion of Banned Users 0 0.1 0.2 0.3 0.4 Proportion of Deleted Posts 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Situational trolling? Lifelong trolling?

Slide 28

Slide 28 text

What if antisocial behavior is situational?

Slide 29

Slide 29 text

Challenge: how to show that antisocial behavior is situational? Observational data isn’t causal

Slide 30

Slide 30 text

Challenge: how to show that antisocial behavior is situational? Experiments hard to generalize

Slide 31

Slide 31 text

Simulated Discussion Experiment Large-Scale Analysis Solution: Experiment + Observational Study

Slide 32

Slide 32 text

Anyone can become a troll Our Hypothesis

Slide 33

Slide 33 text

“Broken windows” theory Zimbardo (1969); Wilson (1982)

Slide 34

Slide 34 text

Unpleasant stimuli increase aggression Jones & Bogat (1978); Rotton & Frey (1985)

Slide 35

Slide 35 text

Experiment: simulated discussion forum

Slide 36

Slide 36 text

N=667, 40% female Quiz Discussion Experimental method

Slide 37

Slide 37 text

Quiz Discussion Experimental method ×

Slide 38

Slide 38 text

Positive/Negative Mood Positive/Negative Context Experimental method × Quiz Discussion

Slide 39

Slide 39 text

Easy quiz (positive mood)

Slide 40

Slide 40 text

Difﬁcult quiz (negative mood)

Slide 41

Slide 41 text

Positive discussion context

Slide 42

Slide 42 text

Negative discussion context

Slide 43

Slide 43 text

People reported being in a worse mood after the difﬁcult quiz Easy: 12.2 Difﬁcult: 40.8 (POMS mood disturbance [higher scores = worse mood], p<0.01) Manipulation Check

Slide 44

Slide 44 text

Initial seed posts in the negative context condition perceived worse Positive: 90% upvoted Negative: 36% upvoted (p < 0.01) Manipulation Check

Slide 45

Slide 45 text

How did trolling differ across conditions? Two expert raters labeled posts independently

Slide 46

Slide 46 text

How did trolling differ across conditions? Positive Mood Negative Mood Positive Context Negative Context % Troll Posts

Slide 47

Slide 47 text

Trolling is lowest with positive conditions… Positive Mood Negative Mood Positive Context 35% Negative Context % Troll Posts

Slide 48

Slide 48 text

…increases with either negative condition… Positive Mood Negative Mood Positive Context 35% 49% Negative Context 47% % Troll Posts

Slide 49

Slide 49 text

…and almost doubles in the worst case Positive Mood Negative Mood Positive Context 35% 49% Negative Context 47% 68% % Troll Posts (p < 0.05 using a mixed effects logistic regression model)

Slide 50

Slide 50 text

Negative affect almost triples Positive Mood Negative Mood Positive Context 1.1% 1.4% Negative Context 2.3% 2.9% % Negative Affect Words (LIWC) (p < 0.05)

Slide 51

Slide 51 text

Hilary is a solid candidate. As a woman, I appreciate that she's a woman, but it's not the only reason I think she would do well in ofﬁce. Positive Mood + Context

Slide 52

Slide 52 text

Anyone who votes for her is a complete idiot. These supporters are why this country is in such bad shape now. Uneducated people. Negative Mood + Context

Slide 53

Slide 53 text

Bad mood and negative discussion context increase trolling

Slide 54

Slide 54 text

Simulated Discussion Experiment Large-Scale Analysis of CNN.com Online Experiment + Observational Study

Slide 55

Slide 55 text

Can trolling, like mood, vary with the time of day and day of week? Replicating Mood Golder & Macy (2011) Neg. Affect Time of Day ?

Slide 56

Slide 56 text

How does trolling vary with time of day? Proportion of Flagged Posts 0.03 0.033 0.036 0.039 0.042 Time of Day 0 6 12 18 24

Slide 57

Slide 57 text

Trolling peaks in the evening… Proportion of Flagged Posts 0.03 0.033 0.036 0.039 0.042 Time of Day 0 6 12 18 24 Negative Affect (Golder & Macy)

Slide 58

Slide 58 text

…and early in the work week. Proportion of Flagged Posts 0.03 0.032 0.034 0.036 0.038 0.04 Day of Week Mon Tue Wed Thu Fri Sat Sun

Slide 59

Slide 59 text

Trolling peaks when moods are worse Time of Day Proportion of Flagged Posts Negative Affect Proportion of Downvotes Day of Week

Slide 60

Slide 60 text

Mood spills over from prior discussions Replicating Mood

Slide 61

Slide 61 text

… … (Discussion) Mood spills over from prior discussions

Slide 62

Slide 62 text

… … … ? … (Discussion) (Unrelated Discussions) … ? … Mood spills over from prior discussions

Slide 63

Slide 63 text

… … … … (Discussion) … … Mood spills over from prior discussions

Slide 64

Slide 64 text

A user who trolled in a previous discussion is twice as likely to troll in a later, unrelated discussion (p < 0.01) Replicating Mood

Slide 65

Slide 65 text

The initial post affects subsequent trolling Replicating Context

Slide 66

Slide 66 text

(Separate discussions of same article) … ? ? ? … ? ? ? The initial post affects subsequent trolling

Slide 67

Slide 67 text

(Separate discussions of same article) … … The initial post affects subsequent trolling

Slide 68

Slide 68 text

An initial troll post increases the subsequent trolling by 63% (p < 0.01) Replicating Context

Slide 69

Slide 69 text

Can we predict trolling before it happens? Balanced dataset of 120K posts Logistic regression

Slide 70

Slide 70 text

Mood Context The User What factors affect trolling? Trolling is situational Trolling is innate

Slide 71

Slide 71 text

How predictable are troll posts? User-speciﬁc Mood Discussion Context Combined AUC 0.00 0.20 0.40 0.60 0.80 0.78 0.74 0.60 0.66

Slide 72

Slide 72 text

How predictable are troll posts? User-speciﬁc Mood Discussion Context Combined AUC 0.00 0.20 0.40 0.60 0.80 0.78 0.74 0.60 0.66

Slide 73

Slide 73 text

How predictable are troll posts? User-speciﬁc Mood Discussion Context Combined AUC 0.00 0.20 0.40 0.60 0.80 0.78 0.74 0.60 0.66

Slide 74

Slide 74 text

Troll or not? User

Slide 75

Slide 75 text

Troll or not? User Mood

Slide 76

Slide 76 text

Troll or not? User Mood Other users { }

Slide 77

Slide 77 text

Because trolling is situational, ordinary people can end up trolling

Slide 78

Slide 78 text

No content

Slide 79

Slide 79 text

Can voting mitigate bad behavior?

Slide 80

Slide 80 text

Downvoting causes negative behavior to worsen Our Hypothesis

Slide 81

Slide 81 text

Antisocial Behavior & Its Spread Talk Outline 1 2 3 What causes antisocial behavior? Can such cascades be predicted? Does it worsen over time?

Slide 82

Slide 82 text

How Antisocial Behavior Worsens ICWSM 2014 with C. Danescu-Niculescu-Mizil, J. Leskovec CAN ANTISOCIAL BEHAVIOR SPI R A L?

Slide 83

Slide 83 text

Downvoting causes negative behavior to worsen Our Hypothesis

Slide 84

Slide 84 text

What effects do evaluations have? Positively evaluated ? ? Negatively evaluated

Slide 85

Slide 85 text

What is a positive or negative evaluation?

Slide 86

Slide 86 text

Deﬁning positive and negative evaluations : 9 1 2 8 N↑  N↑ + N↓ 2  2+8 = = 0.2 p↓ ≤ : N↑  N↑ + N↓ 9  9+1 = = 0.9 p↑ ≥ Positive Evaluation Negative Evaluation (validated using a crowdsourcing experiment)

Slide 87

Slide 87 text

What effects do evaluations have? Positively evaluated ? ? Negatively evaluated

Slide 88

Slide 88 text

Does feedback encourage better behavior? Skinner (1938)

Slide 89

Slide 89 text

Or is bad stronger than good? Brinko (1993); Baumeister, et al. (2001)

Slide 90

Slide 90 text

Four large comment-based news communities

Slide 91

Slide 91 text

What effects do evaluations have? Positively evaluated Negatively evaluated

Slide 92

Slide 92 text

What effects do evaluations have? … … … …

Slide 93

Slide 93 text

What effects do evaluations have? … … … … Before After vs. Before After vs.

Slide 94

Slide 94 text

Challenge: how to compare different users and posts? Aren’t downvoted users/posts inherently worse?

Slide 95

Slide 95 text

Solution: propensity score matching PSM: Rosenbaum (1983); CEM: Iacus, et al. (2012) Positively evaluated Negatively evaluated

Slide 96

Slide 96 text

Match on text quality Similar text quality q } ≈

Slide 97

Slide 97 text

Computing text quality Learn p with bigrams  (binomial regression) 1 3 lorem ipsum… q = ? Lorem… ? ? 9 2 lorem ipsum… … Text quality q is predicted p

Slide 98

Slide 98 text

Validating text quality Manually label subset (n=171) using crowdsourcing lorem ipsum… Good Bad Good Good # Good  # Total q’=

Slide 99

Slide 99 text

Validating text quality Manually label subset (n=171) using crowdsourcing lorem ipsum… Good Bad Good Good # Good  # Total q’= ? ? corr(q’, p) = corr(q’, q) =

Slide 100

Slide 100 text

Validating text quality (n.s.) (p < 0.01) p: actual proportion of upvotes q: text quality (predicted proportion) q’: crowd guess corr(q’, q) = corr(q’, p) = 0.11 0.25

Slide 101

Slide 101 text

Validating text quality |Residuals| 0 0.2 0.4 0.6 0.8 1 0.6 0.7 0.8 0.9 1 |q − q’| (n.s. using a Breusch-Pagan test) Text Quality q

Slide 102

Slide 102 text

Match on text quality Similar text quality q(c↑ )=q(c↓ ) } ≈

Slide 103

Slide 103 text

…as well as other covariates Similar history (# posts, overall proportion of upvotes, etc.) { ≈ … … ≈

Slide 104

Slide 104 text

…as well as other covariates ≈ … … … … ≈

Slide 105

Slide 105 text

How are subsequent posts evaluated? ≈ … … … … ≈

Slide 106

Slide 106 text

How much are evaluations due to textual or community effects?

Slide 107

Slide 107 text

How much are evaluations due to textual effects (i.e., people writing worse)? f***ing a****** i.e., downvoting because of post content

Slide 108

Slide 108 text

How much are evaluations due to community effects (i.e., inherent bias)? We dislike you. i.e., downvoting because of community dislikes author We dislike you.

Slide 109

Slide 109 text

Do people write better/worse after a positive/negative evaluation? Textual Effects

Slide 110

Slide 110 text

≈ … … … … Better/Worse? Do people write better/worse after a positive/negative evaluation? ≈

Slide 111

Slide 111 text

Text quality drops signiﬁcantly after a negative evaluation… (p < 0.05, mean effect size r = 0.18) … … Negativity bias

Slide 112

Slide 112 text

…but doesn’t change after a positive evaluation … … (n.s.) Negativity bias

Slide 113

Slide 113 text

How does community bias change after an evaluation? Community Effects

Slide 114

Slide 114 text

… … … … How does community bias change after an evaluation?

Slide 115

Slide 115 text

Measuring community bias N↑ q N↓ N↑  N↑ + N↓ = 0.5 p(c) = = 0.8 q(c) p(c) q(c) = 0.3 Prop. Upvotes Text Quality Community Bias − −

Slide 116

Slide 116 text

Community bias increase more after a negative than positive evaluation (p < 0.01, mean effect size r = 0.13) … … Halo effect

Slide 117

Slide 117 text

More positive Negative Eval. Positive Eval.

Slide 118

Slide 118 text

More positive Negative Eval. Positive Eval. Similar text quality  q(c↑ )=q(c↓ )

Slide 119

Slide 119 text

More positive Before Negative Eval. Positive Eval. Similar history Similar text quality  q(c↑ )=q(c↓ )

Slide 120

Slide 120 text

More positive Before After Negative Eval. Positive Eval. Similar history Similar text quality  q(c↑ )=q(c↓ ) Worse text quality  q(c↑(1,3) ) > q(c↓(1,3) ) *

Slide 121

Slide 121 text

More positive Before After Negative Eval. Positive Eval. Similar history Similar text quality  q(c↑ )=q(c↓ ) Worse text quality  q(c↑(1,3) ) > q(c↓(1,3) ) Worse perception  q(c↓(1,3) ) - p(c↓(1,3) ) > q(c↑(1,3) ) - p(c↑(1,3) ) * *

Slide 122

Slide 122 text

What happens to negatively-evaluated users?

Slide 123

Slide 123 text

They post worse content Perceptions of them become worse They post more frequently* They evaluate others more negatively* * More details in our ICWSM 2014 paper (http://bit.ly/feedback-paper) What happens to negatively-evaluated users?

Slide 124

Slide 124 text

Troll or not? User Mood Other users { }

Slide 125

Slide 125 text

Troll or not? User Mood Other users { }

Slide 126

Slide 126 text

Troll or not? User Mood Other users { }

Slide 127

Slide 127 text

Trolls may start out normal, but tip into a spiral and never recover

Slide 128

Slide 128 text

Do communities worsen over time?

Slide 129

Slide 129 text

Communities may worsen over time (?) Proportion of Upvotes 0.6 0.65 0.7 0.75 0.8 Time December 2012 February 2013 April 2013 June 2013 August 2013

Slide 130

Slide 130 text

Antisocial Behavior & Its Spread Talk Outline 1 2 3 What causes antisocial behavior? Can such cascades be predicted? Does it worsen over time?

Slide 131

Slide 131 text

The Predictability of Information Cascades in Social Networks CAN C S BE PREDICTED? AS A C DE WWW 2014; WWW 2016 with L. Adamic, P. A. Dow, J. Kleinberg, J. Leskovec

Slide 132

Slide 132 text

Rumors on Facebook ICWSM 2014 (with A. Friggeri, L. Adamic, and D. Eckles)

Slide 133

Slide 133 text

Same rumor, different popularity

Slide 134

Slide 134 text

Are these cascades predictable?

Slide 135

Slide 135 text

No content

Slide 136

Slide 136 text

No content

Slide 137

Slide 137 text

No content

Slide 138

Slide 138 text

No content

Slide 139

Slide 139 text

No content

Slide 140

Slide 140 text

No content

Slide 141

Slide 141 text

Are cascades unpredictable?

Slide 142

Slide 142 text

Large cascades are rare Empirical CCDF 0.00 0.20 0.40 0.60 0.80 1.00 Cascade size 0 200 400 600 800 1000 0.09 100

Slide 143

Slide 143 text

“Increasing the strength of social inﬂuence increased both inequality and unpredictability of success.” Salganik, Dodds & Watts (2006)

Slide 144

Slide 144 text

Cascades can recur after long periods

Slide 145

Slide 145 text

Cascades are predictable Our Hypothesis size, structure, content even if they recur

Slide 146

Slide 146 text

How do we begin to predict cascade growth?

Slide 147

Slide 147 text

Challenge: how to predict cascade growth? ? k=5 reshares  observed

Slide 148

Slide 148 text

Challenge: how to predict cascade growth? ? Will a cascade get 100 reshares? Exactly how big will a small cascade get? Only consider the largest cascades?

Slide 149

Slide 149 text

Challenge: how to predict cascade growth? ? Will a cascade get 100 reshares? Exactly how big will a small cascade get? Only consider the largest cascades? class imbalance outliers skew results selection bias

Slide 150

Slide 150 text

Solution: will a cascade reach the median? ? ≤ the median f(k) ≥ the median f(k) k=5 reshares  observed

Slide 151

Slide 151 text

Solution: will a cascade double in size? ? ≤ the median f(k) ≥ the median f(k) k=5 reshares  observed

Slide 152

Slide 152 text

Given that a cascade has obtained k reshares, will it double in size? balanced track growth over time Cascade Growth Prediction Problem

Slide 153

Slide 153 text

Reshare cascades on Facebook 70M cascades 5B reshares Activity over 28 days

Slide 154

Slide 154 text

Content  has overlaid text captions … User  friend count  gender … Structural  tree depth outdegree … Temporal  time between shares change in time … What factors affect predictability?

Slide 155

Slide 155 text

How predictable is cascade doubling? All Temporal All but temporal Structural User Content AUC (k=5) 0.00 0.23 0.45 0.68 0.90 0.58 0.71 0.74 0.79 0.87 0.88 All but temporal

Slide 156

Slide 156 text

Given that a cascade has obtained k reshares, will it double in size? Cascade Growth Prediction Problem

Slide 157

Slide 157 text

Given that a cascade has obtained 5 reshares, will it double in size? Cascade Growth Prediction Problem

Slide 158

Slide 158 text

Given that a cascade has obtained 100 reshares, will it double in size? Cascade Growth Prediction Problem

Slide 159

Slide 159 text

How does performance change with k? k = 5 k > 10 k = 10 k > 20

Slide 160

Slide 160 text

How does performance change with k? k = 5 k > 10 k = 10 k > 20 Less data More data

Slide 161

Slide 161 text

How does performance change with k? k = 5 k > 10 k = 10 k > 20 Shorter-term Longer-term

Slide 162

Slide 162 text

Easier to predict larger cascades doubling Accuracy 0.78 0.79 0.8 0.81 0.82 Number of reshares observed, k 0 25 50 75 100

Slide 163

Slide 163 text

Cascade growth is predictable

Slide 164

Slide 164 text

Cascade structure is predictable AUC = 0.80 for predicting structural virality * More details in our WWW 2014 paper (http://bit.ly/memes-paper) vs.

Slide 165

Slide 165 text

Cascade recurrence is predictable AUC = 0.89 for predicting a subsequent burst * More details in our WWW 2016 paper (http://bit.ly/cascades-paper) vs.

Slide 166

Slide 166 text

Troll or not? User Mood Other users { }

Slide 167

Slide 167 text

No content

Slide 168

Slide 168 text

What we now know What we thought Trolls are a vocal minority Trolls can be ordinary people Trolling is innate Trolling can spiral from a single bad post Cascades can be predicted Cascades are unpredictable ANTISOCIAL COMPUTING

Slide 169

Slide 169 text

Predicting the demise of communities Proportion of Upvotes 0.6 0.65 0.7 0.75 0.8 Time December February April June August Future Directions

Slide 170

Slide 170 text

Designing prosocial discussion platforms Future Directions

Slide 171

Slide 171 text

Designing prosocial discussion platforms Future Directions

Slide 172

Slide 172 text

Munger (2016) Introducing conversation mediators Future Directions Don’t be a n****r. Hey man, just remember that there are real people who are hurt when you harass them with that kind of language. (e.g., bots)

Slide 173

Slide 173 text

WWW 2017 (with S. Kumar, J. Leskovec, and V.S. Subrahmanian) Identifying different types of trolling Future Directions Possibly the best blog I’ve ever read major props to you Thanks. I knew Marvel fans would try to ﬂame me, but they have nothing other than “oh that’s your opinion” Quit talking to yourself […] (e.g., sockpuppets)

Slide 174

Slide 174 text

Addressing polarization Future Directions Measuring algorithmic impact Tracking cascades at scale

Slide 175

Slide 175 text

Holistic approaches for analyzing and building social systems Research Approach

Slide 176

Slide 176 text

Holistic approaches for analyzing and building social systems Large-scale Analysis Experimentation + Macro-scale Micro-scale + Understand Build + Research Approach

Slide 177

Slide 177 text

Multi-methods analyses identify patterns in data, verify hypotheses, make predictions, and develop social systems. Multi-methods analyses identify patterns in data, verify hypotheses, make predictions, and inform the design of better social systems.

Slide 178

Slide 178 text

Jure Leskovec Michael Bernstein Jon Kleinberg Lada Adamic Thank you!

Slide 179

Slide 179 text

James Landay Jeff Hancock Cristian Danescu-Niculescu-Mizil Dan Cosley Thank you!

Slide 180

Slide 180 text

Thank you!

Slide 181

Slide 181 text

Thank you!

Slide 182

Slide 182 text

Thank you!

Slide 183

Slide 183 text

Thank you! Stanford HCI Group SNAP Group Stanford VPGE Microsoft Research Facebook Pinterest Disqus

Slide 184

Slide 184 text

Justin Cheng / @jcccf / clr3.com Stanford University More resources and credits: http://bit.ly/jobtalkcredits ANTISOCIAL COMPUTING Explaining and Predicting Negative Behavior Online