Antisocial Computing

In online social networks, large information cascades can develop as people share content with one another. However, as these cascades develop through complex processes, prior work has argued that their future trajectory may be inherently unpredictable. My research introduces methods for studying the mechanisms of these cascades and predicting their spread. Analyzing billions of interactions by hundreds of millions of users on Facebook, I show how the future growth and structure of these cascades can be predicted, how cascades may resurface after lying dormant for months, and how diverse social protocols can produce large information cascades. Through revealing the mechanisms in which information diffuses in social media, this work explores a future where systems can better promote sharing behavior online.

Presented at the Stanford Human-Computer Interaction Seminar.

Transcript

1. Justin Cheng
   Stanford University
   ANTISOCIAL COMPUTING
   Explaining and Predicting Negative Behavior Online
   

   View Slide

2. Vieweg, et al. (2010); Kittur, et al. (2013); Burke & Kraut (2016)
   

   View Slide

3. View Slide

4. Time (2016); The Atlantic (2016); Vanity Fair (2017)
   

   View Slide

5. 47% of online users
   have been harassed
   Data & Society (2017)
   

   View Slide

6. Popular Science (2013); The Verge (2015); Chicago Sun-Times (2014)
   

   View Slide

7. Why is bad behavior so prevalent?

   (›°□°)›ớ ᵲᴸᵲ
   Research Question
   

   View Slide

8. Understanding bad behavior helps
   us build healthier communities
   Implications
   Systems
   Guidelines Interventions
   

   View Slide

9. Antisocial behavior is largely
   due to sociopaths
   Prior Work
   Donath (1999); Hardaker (2010); Buckels, et al. (2014)
   

   View Slide

10. Antisocial behavior is largely
    due to ordinary people
    This Work
    

    View Slide

11. Antisocial Behavior & Its Spread
    Talk Outline
    What causes antisocial behavior?
    Can such cascades be predicted?
    Does it worsen over time?
    1
    2
    3
    

    View Slide

12. Data Mining + Crowdsourcing
    Research Approach
    Large-scale Analysis + Experiments
    

    View Slide

13. Identifying principles of
    online behavior
    The Broader Picture
    Data + ML + Network Science + HCI
    

    View Slide

14. Antisocial Behavior & Its Spread
    Talk Outline
    1
    2
    3
    What causes antisocial behavior?
    Can such cascades be predicted?
    Does it worsen over time?
    

    View Slide

15. CSCW 2017 (Best Paper); ICWSM 2015 (Honorable Mention)
    with M. Bernstein, C. Danescu-Niculescu-Mizil, J. Leskovec
    CAN ANYONE
    BECOME A TROLL?
    Causes of Antisocial Behavior in Online Discussions
    

    View Slide

16. CONTENT WARNING!

    This talk contains depictions of trolling that use strong language.
    !
    

    View Slide

17. View Slide

18. It also shows that Islam and
    Christianity teaching women to
    dress modest could be right
    afterall.
    

    View Slide

19. It also shows that Islam and
    Christianity teaching women to
    dress modest could be right
    afterall.
    Clearly that is the only logical
    conclusion to this article. Now if
    you'll excuse me, I need to iron
    my tarp. I have work on Monday,
    and I want to appear 'modest'.
    fail at life. go bomb yourself.
    Religious nut alert
    

    View Slide

20. We studied multiple large comment-
    based news communities.
    470M posts 831M votes
    76M users
    

    View Slide

21. What is trolling?
    

    View Slide

22. What is trolling?
    Engaging in negatively marked online
    behavior
    Taking pleasure in upsetting others
    Not following the rules
    Disrupting a group while staying
    undercover
    Donath (1999); Hardaker (2010); Kirman (2012); Schwartz (2008)
    

    View Slide

23. Trolling is behavior that occurs
    outside community norms.
    Defined using community guidelines
    Our Definition
    e.g., name-calling, personal attacks, profanity, threats, hate speech, ethnically/racially offensive material
    

    View Slide

24. Are trolls just a vocal minority?
    Donath (1999); Hardaker (2010); Shachaf & Hara (2010); NYT (2008); Wired (2014); Vox (2014)
    

    View Slide

25. How much do trolls troll?
    Proportion of Banned Users
    0
    0.1
    0.2
    0.3
    0.4
    Proportion of Deleted Posts
    0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
    

    View Slide

26. The distribution of trolls is bimodal
    Proportion of Banned Users
    0
    0.1
    0.2
    0.3
    0.4
    Proportion of Deleted Posts
    0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
    

    View Slide

27. Are there two types of trolls?
    Proportion of Banned Users
    0
    0.1
    0.2
    0.3
    0.4
    Proportion of Deleted Posts
    0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
    Situational trolling?
    Lifelong trolling?
    

    View Slide

28. What if antisocial behavior is
    situational?
    

    View Slide

29. Challenge: how to show that
    antisocial behavior is situational?
    Observational data isn’t causal
    

    View Slide

30. Challenge: how to show that
    antisocial behavior is situational?
    Experiments hard to generalize
    

    View Slide

31. Simulated Discussion Experiment Large-Scale Analysis
    Solution: Experiment + Observational Study
    

    View Slide

32. Anyone can become a troll
    Our Hypothesis
    

    View Slide

33. “Broken windows” theory
    Zimbardo (1969); Wilson (1982)
    

    View Slide

34. Unpleasant stimuli increase aggression
    Jones & Bogat (1978); Rotton & Frey (1985)
    

    View Slide

35. Experiment: simulated discussion forum
    

    View Slide

36. N=667, 40% female
    Quiz Discussion
    Experimental method
    

    View Slide

37. Quiz Discussion
    Experimental method
    ×
    

    View Slide

38. Positive/Negative Mood Positive/Negative Context
    Experimental method
    ×
    Quiz Discussion
    

    View Slide

39. Easy quiz (positive mood)
    

    View Slide

40. Difficult quiz (negative mood)
    

    View Slide

41. Positive discussion context
    

    View Slide

42. Negative discussion context
    

    View Slide

43. People reported being in a worse
    mood after the difficult quiz
    Easy: 12.2 Difficult: 40.8
    (POMS mood disturbance [higher scores = worse mood], p<0.01)
    Manipulation Check
    

    View Slide

44. Initial seed posts in the negative
    context condition perceived worse
    Positive: 90% upvoted Negative: 36% upvoted
    (p < 0.01)
    Manipulation Check
    

    View Slide

45. How did trolling differ across
    conditions?
    Two expert raters labeled posts independently
    

    View Slide

46. How did trolling differ across conditions?
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    Negative
    Context
    % Troll Posts
    

    View Slide

47. Trolling is lowest with positive conditions…
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    35%
    Negative
    Context
    % Troll Posts
    

    View Slide

48. …increases with either negative condition…
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    35% 49%
    Negative
    Context
    47%
    % Troll Posts
    

    View Slide

49. …and almost doubles in the worst case
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    35% 49%
    Negative
    Context
    47% 68%
    % Troll Posts
    (p < 0.05 using a mixed effects logistic regression model)
    

    View Slide

50. Negative affect almost triples
    Positive
    Mood
    Negative
    Mood
    Positive
    Context
    1.1% 1.4%
    Negative
    Context
    2.3% 2.9%
    % Negative Affect Words (LIWC)
    (p < 0.05)
    

    View Slide

51. Hilary is a solid candidate. As a woman, I
    appreciate that she's a woman, but it's not
    the only reason I think she would do well in
    office.
    Positive Mood + Context
    

    View Slide

52. Anyone who votes for her is a complete
    idiot. These supporters are why this country
    is in such bad shape now. Uneducated
    people.
    Negative Mood + Context
    

    View Slide

53. Bad mood and negative discussion
    context increase trolling
    

    View Slide

54. Simulated Discussion Experiment Large-Scale Analysis of CNN.com
    Online Experiment + Observational Study
    

    View Slide

55. Can trolling, like mood, vary with
    the time of day and day of week?
    Replicating Mood
    Golder & Macy (2011)
    Neg. Affect
    Time of Day
    ?
    

    View Slide

56. How does trolling vary with time of day?
    Proportion of Flagged Posts
    0.03
    0.033
    0.036
    0.039
    0.042
    Time of Day
    0 6 12 18 24
    

    View Slide

57. Trolling peaks in the evening…
    Proportion of Flagged Posts
    0.03
    0.033
    0.036
    0.039
    0.042
    Time of Day
    0 6 12 18 24
    Negative Affect
    (Golder & Macy)
    

    View Slide

58. …and early in the work week.
    Proportion of Flagged Posts
    0.03
    0.032
    0.034
    0.036
    0.038
    0.04
    Day of Week
    Mon Tue Wed Thu Fri Sat Sun
    

    View Slide

59. Trolling peaks when moods are worse
    Time of Day
    Proportion of
    Flagged Posts
    Negative Affect
    Proportion of
    Downvotes
    Day of Week
    

    View Slide

60. Mood spills over from prior
    discussions
    Replicating Mood
    

    View Slide

61. …
    …
    (Discussion)
    Mood spills over from prior
    discussions
    

    View Slide

62. …
    …
    …
    ?
    …
    (Discussion)
    (Unrelated Discussions)
    …
    ?
    …
    Mood spills over from prior
    discussions
    

    View Slide

63. …
    …
    …
    …
    (Discussion)
    …
    …
    Mood spills over from prior
    discussions
    

    View Slide

64. A user who trolled in a previous
    discussion is twice as likely to troll
    in a later, unrelated discussion
    (p < 0.01)
    Replicating Mood
    

    View Slide

65. The initial post affects subsequent
    trolling
    Replicating Context
    

    View Slide

66. (Separate discussions of same article)
    …
    ? ? ?
    …
    ? ? ?
    The initial post affects subsequent
    trolling
    

    View Slide

67. (Separate discussions of same article)
    …
    …
    The initial post affects subsequent
    trolling
    

    View Slide

68. An initial troll post increases the
    subsequent trolling by 63%
    (p < 0.01)
    Replicating Context
    

    View Slide

69. Can we predict trolling
    before it happens?
    Balanced dataset of 120K posts Logistic regression
    

    View Slide

70. Mood Context The User
    What factors affect trolling?
    Trolling is situational Trolling is innate
    

    View Slide

71. How predictable are troll posts?
    User-specific
    Mood
    Discussion Context
    Combined
    AUC
    0.00 0.20 0.40 0.60 0.80
    0.78
    0.74
    0.60
    0.66
    

    View Slide

72. How predictable are troll posts?
    User-specific
    Mood
    Discussion Context
    Combined
    AUC
    0.00 0.20 0.40 0.60 0.80
    0.78
    0.74
    0.60
    0.66
    

    View Slide

73. How predictable are troll posts?
    User-specific
    Mood
    Discussion Context
    Combined
    AUC
    0.00 0.20 0.40 0.60 0.80
    0.78
    0.74
    0.60
    0.66
    

    View Slide

74. Troll or not?
    User
    

    View Slide

75. Troll or not?
    User
    Mood
    

    View Slide

76. Troll or not?
    User
    Mood Other users
    { }
    

    View Slide

77. Because trolling is situational,
    ordinary people can end up trolling
    

    View Slide

78. View Slide

79. Can voting mitigate bad behavior?
    

    View Slide

80. Downvoting causes
    negative behavior to worsen
    Our Hypothesis
    

    View Slide

81. Antisocial Behavior & Its Spread
    Talk Outline
    1
    2
    3
    What causes antisocial behavior?
    Can such cascades be predicted?
    Does it worsen over time?
    

    View Slide

82. How Antisocial Behavior Worsens
    ICWSM 2014
    with C. Danescu-Niculescu-Mizil, J. Leskovec
    CAN ANTISOCIAL
    BEHAVIOR SPI
    R
    A
    L?
    

    View Slide

83. Downvoting causes
    negative behavior to worsen
    Our Hypothesis
    

    View Slide

84. What effects do evaluations have?
    Positively
    evaluated
    ?
    ?
    Negatively
    evaluated
    

    View Slide

85. What is a positive or negative evaluation?
    

    View Slide

86. Defining positive and negative evaluations
    :
    9 1
    2 8
    N↑

    N↑
    + N↓
    2

    2+8
    = = 0.2 p↓
    ≤
    : N↑

    N↑
    + N↓
    9

    9+1
    = = 0.9 p↑
    ≥
    Positive Evaluation
    Negative Evaluation
    (validated using a crowdsourcing experiment)
    

    View Slide

87. What effects do evaluations have?
    Positively
    evaluated
    ?
    ?
    Negatively
    evaluated
    

    View Slide

88. Does feedback encourage better behavior?
    Skinner (1938)
    

    View Slide

89. Or is bad stronger than good?
    Brinko (1993); Baumeister, et al. (2001)
    

    View Slide

90. Four large comment-based news
    communities
    

    View Slide

91. What effects do evaluations have?
    Positively
    evaluated
    Negatively
    evaluated
    

    View Slide

92. What effects do evaluations have?
    …
    …
    …
    …
    

    View Slide

93. What effects do evaluations have?
    …
    …
    …
    …
    Before After
    vs.
    Before After
    vs.
    

    View Slide

94. Challenge: how to compare
    different users and posts?
    Aren’t downvoted users/posts inherently worse?
    

    View Slide

95. Solution: propensity score matching
    PSM: Rosenbaum (1983); CEM: Iacus, et al. (2012)
    Positively
    evaluated
    Negatively
    evaluated
    

    View Slide

96. Match on text quality
    Similar text
    quality q
    }
    ≈
    

    View Slide

97. Computing text quality
    Learn p with bigrams

    (binomial regression)
    1 3
    lorem ipsum…
    q = ?
    Lorem…
    ? ?
    9 2
    lorem ipsum…
    …
    Text quality q is
    predicted p
    

    View Slide

98. Validating text quality
    Manually label subset (n=171) using crowdsourcing
    lorem ipsum…
    Good
    Bad
    Good
    Good
    # Good

    # Total
    q’=
    

    View Slide

99. Validating text quality
    Manually label subset (n=171) using crowdsourcing
    lorem ipsum…
    Good
    Bad
    Good
    Good
    # Good

    # Total
    q’=
    ? ?
    corr(q’, p) = corr(q’, q) =
    

    View Slide

100. Validating text quality
     (n.s.) (p < 0.01)
     p: actual proportion of upvotes
     q: text quality (predicted proportion)
     q’: crowd guess
     corr(q’, q) =
     corr(q’, p) = 0.11 0.25
     

     View Slide

101. Validating text quality
     |Residuals|
     0
     0.2
     0.4
     0.6
     0.8
     1
     0.6 0.7 0.8 0.9 1
     |q − q’|
     (n.s. using a Breusch-Pagan test)
     Text Quality q
     

     View Slide

102. Match on text quality
     Similar text
     quality
     q(c↑
     )=q(c↓
     )
     }
     ≈
     

     View Slide

103. …as well as other covariates
     Similar history
     (# posts, overall
     proportion of
     upvotes, etc.)
     { ≈
     …
     …
     ≈
     

     View Slide

104. …as well as other covariates
     ≈
     …
     …
     …
     …
     ≈
     

     View Slide

105. How are subsequent posts evaluated?
     ≈
     …
     …
     …
     …
     ≈
     

     View Slide

106. How much are evaluations due to
     textual or community effects?
     

     View Slide

107. How much are evaluations due to
     textual effects (i.e., people writing worse)?
     f***ing a******
     i.e., downvoting because of post content
     

     View Slide

108. How much are evaluations due to
     community effects (i.e., inherent bias)?
     We dislike you.
     i.e., downvoting because of community dislikes author
     We dislike you.
     

     View Slide

109. Do people write better/worse after
     a positive/negative evaluation?
     Textual Effects
     

     View Slide

110. ≈
     …
     …
     …
     …
     Better/Worse?
     Do people write better/worse after
     a positive/negative evaluation?
     ≈
     

     View Slide

111. Text quality drops significantly after
     a negative evaluation…
     (p < 0.05, mean effect size r = 0.18)
     … …
     Negativity bias
     

     View Slide

112. …but doesn’t change after
     a positive evaluation
     … …
     (n.s.)
     Negativity bias
     

     View Slide

113. How does community bias
     change after an evaluation?
     Community Effects
     

     View Slide

114. … …
     … …
     How does community bias
     change after an evaluation?
     

     View Slide

115. Measuring community bias
     N↑ q
     N↓
     N↑

     N↑
     + N↓
     = 0.5
     p(c) =
     = 0.8
     q(c)
     p(c) q(c) = 0.3
     Prop. Upvotes
     Text Quality
     Community Bias − −
     

     View Slide

116. Community bias increase more after
     a negative than positive evaluation
     (p < 0.01, mean effect size r = 0.13)
     … …
     Halo effect
     

     View Slide

117. More positive
     Negative Eval.
     Positive Eval.
     

     View Slide

118. More positive
     Negative Eval.
     Positive Eval.
     Similar text quality

     q(c↑
     )=q(c↓
     )
     

     View Slide

119. More positive
     Before Negative Eval.
     Positive Eval.
     Similar history Similar text quality

     q(c↑
     )=q(c↓
     )
     

     View Slide

120. More positive
     Before After
     Negative Eval.
     Positive Eval.
     Similar history Similar text quality

     q(c↑
     )=q(c↓
     )
     Worse text quality

     q(c↑(1,3)
     ) > q(c↓(1,3)
     )
     *
     

     View Slide

121. More positive
     Before After
     Negative Eval.
     Positive Eval.
     Similar history Similar text quality

     q(c↑
     )=q(c↓
     )
     Worse text quality

     q(c↑(1,3)
     ) > q(c↓(1,3)
     )
     Worse perception

     q(c↓(1,3)
     ) - p(c↓(1,3)
     ) >
     q(c↑(1,3)
     ) - p(c↑(1,3)
     )
     *
     *
     

     View Slide

122. What happens to
     negatively-evaluated users?
     

     View Slide

123. They post worse content
     Perceptions of them become worse
     They post more frequently*
     They evaluate others more negatively*
     * More details in our ICWSM 2014 paper (http://bit.ly/feedback-paper)
     What happens to
     negatively-evaluated users?
     

     View Slide

124. Troll or not?
     User
     Mood Other users
     { }
     

     View Slide

125. Troll or not?
     User
     Mood Other users
     { }
     

     View Slide

126. Troll or not?
     User
     Mood Other users
     { }
     

     View Slide

127. Trolls may start out normal, but tip
     into a spiral and never recover
     

     View Slide

128. Do communities worsen over time?
     

     View Slide

129. Communities may worsen over time (?)
     Proportion of Upvotes
     0.6
     0.65
     0.7
     0.75
     0.8
     Time
     December 2012 February 2013 April 2013 June 2013 August 2013
     

     View Slide

130. Antisocial Behavior & Its Spread
     Talk Outline
     1
     2
     3
     What causes antisocial behavior?
     Can such cascades be predicted?
     Does it worsen over time?
     

     View Slide

131. The Predictability of Information Cascades in Social Networks
     CAN C S
     BE PREDICTED?
     AS A
     C DE
     WWW 2014; WWW 2016
     with L. Adamic, P. A. Dow, J. Kleinberg, J. Leskovec
     

     View Slide

132. Rumors on Facebook
     ICWSM 2014 (with A. Friggeri, L. Adamic, and D. Eckles)
     

     View Slide

133. Same rumor, different popularity
     

     View Slide

134. Are these cascades predictable?
     

     View Slide

135. View Slide

136. View Slide

137. View Slide

138. View Slide

139. View Slide

140. View Slide

141. Are cascades unpredictable?
     

     View Slide

142. Large cascades are rare
     Empirical CCDF
     0.00
     0.20
     0.40
     0.60
     0.80
     1.00
     Cascade size
     0 200 400 600 800 1000
     0.09
     100
     

     View Slide

143. “Increasing the strength of social
     influence increased both inequality
     and unpredictability of success.”
     Salganik, Dodds & Watts (2006)
     

     View Slide

144. Cascades can recur after long periods
     

     View Slide

145. Cascades are predictable
     Our Hypothesis
     size, structure, content even if they recur
     

     View Slide

146. How do we begin to predict
     cascade growth?
     

     View Slide

147. Challenge: how to predict cascade growth?
     ?
     k=5 reshares

     observed
     

     View Slide

148. Challenge: how to predict cascade growth?
     ?
     Will a cascade get 100 reshares?
     Exactly how big will a small cascade get?
     Only consider the largest cascades?
     

     View Slide

149. Challenge: how to predict cascade growth?
     ?
     Will a cascade get 100 reshares?
     Exactly how big will a small cascade get?
     Only consider the largest cascades?
     class imbalance
     outliers skew results
     selection bias
     

     View Slide

150. Solution: will a cascade reach the median?
     ? ≤ the median f(k)
     ≥ the median f(k)
     k=5 reshares

     observed
     

     View Slide

151. Solution: will a cascade double in size?
     ? ≤ the median f(k)
     ≥ the median f(k)
     k=5 reshares

     observed
     

     View Slide

152. Given that a cascade has obtained
     k reshares, will it double in size?
     balanced track growth over time
     Cascade Growth Prediction Problem
     

     View Slide

153. Reshare cascades on Facebook
     70M cascades 5B reshares Activity over 28 days
     

     View Slide

154. Content

     has overlaid text
     captions
     …
     User

     friend count

     gender
     …
     Structural

     tree depth
     outdegree
     …
     Temporal

     time between shares
     change in time
     …
     What factors affect predictability?
     

     View Slide

155. How predictable is cascade doubling?
     All
     Temporal
     All but temporal
     Structural
     User
     Content
     AUC (k=5)
     0.00 0.23 0.45 0.68 0.90
     0.58
     0.71
     0.74
     0.79
     0.87
     0.88
     All but temporal
     

     View Slide

156. Given that a cascade has obtained
     k reshares, will it double in size?
     Cascade Growth Prediction Problem
     

     View Slide

157. Given that a cascade has obtained
     5 reshares, will it double in size?
     Cascade Growth Prediction Problem
     

     View Slide

158. Given that a cascade has obtained
     100 reshares, will it double in size?
     Cascade Growth Prediction Problem
     

     View Slide

159. How does performance change with k?
     k = 5 k > 10
     k = 10 k > 20
     

     View Slide

160. How does performance change with k?
     k = 5 k > 10
     k = 10 k > 20
     Less data
     More data
     

     View Slide

161. How does performance change with k?
     k = 5 k > 10
     k = 10 k > 20
     Shorter-term
     Longer-term
     

     View Slide

162. Easier to predict larger cascades doubling
     Accuracy
     0.78
     0.79
     0.8
     0.81
     0.82
     Number of reshares observed, k
     0 25 50 75 100
     

     View Slide

163. Cascade growth is predictable
     

     View Slide

164. Cascade structure is predictable
     AUC = 0.80 for predicting structural virality
     * More details in our WWW 2014 paper (http://bit.ly/memes-paper)
     vs.
     

     View Slide

165. Cascade recurrence is predictable
     AUC = 0.89 for predicting a subsequent burst
     * More details in our WWW 2016 paper (http://bit.ly/cascades-paper)
     vs.
     

     View Slide

166. Troll or not?
     User
     Mood Other users
     { }
     

     View Slide

167. View Slide

168. What we now know
     What we thought
     Trolls are a
     vocal minority
     Trolls can be
     ordinary people
     Trolling is innate
     Trolling can spiral from
     a single bad post
     Cascades can be
     predicted
     Cascades are
     unpredictable
     ANTISOCIAL COMPUTING
     

     View Slide

169. Predicting the demise of communities
     Proportion of Upvotes
     0.6
     0.65
     0.7
     0.75
     0.8
     Time
     December February April June August
     Future Directions
     

     View Slide

170. Designing prosocial discussion platforms
     Future Directions
     

     View Slide

171. Designing prosocial discussion platforms
     Future Directions
     

     View Slide

172. Munger (2016)
     Introducing conversation mediators
     Future Directions
     Don’t be a n****r.
     Hey man, just remember that there
     are real people who are hurt
     when you harass them with that
     kind of language.
     (e.g., bots)
     

     View Slide

173. WWW 2017 (with S. Kumar, J. Leskovec, and V.S. Subrahmanian)
     Identifying different types of trolling
     Future Directions
     Possibly the best blog I’ve ever
     read major props to you
     Thanks. I knew Marvel fans would try
     to flame me, but they have nothing
     other than “oh that’s your opinion”
     Quit talking to yourself […]
     (e.g., sockpuppets)
     

     View Slide

174. Addressing polarization
     Future Directions
     Measuring algorithmic impact
     Tracking cascades at scale
     

     View Slide

175. Holistic approaches for analyzing
     and building social systems
     Research Approach
     

     View Slide

176. Holistic approaches for analyzing
     and building social systems
     Large-scale Analysis Experimentation
     +
     Macro-scale Micro-scale
     +
     Understand Build
     +
     Research Approach
     

     View Slide

177. Multi-methods analyses identify patterns
     in data, verify hypotheses, make
     predictions, and develop social systems.
     Multi-methods analyses identify
     patterns in data, verify hypotheses,
     make predictions, and inform the
     design of better social systems.
     

     View Slide

178. Jure Leskovec Michael Bernstein Jon Kleinberg Lada Adamic
     Thank you!
     

     View Slide

179. James Landay Jeff Hancock
     Cristian
     Danescu-Niculescu-Mizil
     Dan Cosley
     Thank you!
     

     View Slide

180. Thank you!
     

     View Slide

181. Thank you!
     

     View Slide

182. Thank you!
     

     View Slide

183. Thank you!
     Stanford HCI Group
     SNAP Group
     Stanford VPGE
     Microsoft Research
     Facebook
     Pinterest
     Disqus
     

     View Slide

184. Justin Cheng / @jcccf / clr3.com
     Stanford University
     More resources and credits: http://bit.ly/jobtalkcredits
     ANTISOCIAL COMPUTING
     Explaining and Predicting Negative Behavior Online
     

     View Slide