Causal: Week 10

Transcript

1. C William Lowe Hertie School nd November
2. P → A policy problem in the US → e causal inference of the policy problem → What is the e ect of race? → What can be estimated? → What should be estimated? → A minimal approach (Knox et al., a) → Principal strati cation, assumptions, results
3. P
4. , - Racial disparities in force In NYPD Stop & Frisk data:
5. E - Evidence from causal inference is necessary for → realizing there is a problem at all (looking at you Wall Street Journal editorial page) → Doing something e ective about it (implicit-bias training, scenario training, etc.) Aside: America is an interesting place to study this → Policing is organized very locally, so lots of variation in techniques, training, etc. → Lots of variation in outcomes → Much interest in data collection (body and surveillance cameras, Stingrays, Shotspotter) and data-oriented solutions (‘predictive policing’) → Sometimes very revealing data, e.g. vehicle stop transcripts
6. G Too simply D Y V → D Race → Y Force → V Confounders Slightly more realistically D M Y V → M Police stop (Nearly) all the things that could go wrong D M Y V U W → U, W Confounders → Direct e ect D → Y
7. W ? How is there an arrow going into race (D)? → Our unit of analysis is the encounter or sighting involving a person of some race → Not the person → Scenario: O cer sees person, then decides whether to stop them is is re ected in police data which is organised by stop, not suspect → Sidestep issues about the manipulability of race → We can manipulate race by (experimentally, even) switching in a similarly situated person of a di erent race into the encounter Arrow into D means: factors that change the balance of race across encounters, e.g. neighbourhood indicators
8. N (Nearly) all the things that could go wrong D M Y V U W
9. W ? It’s o en argued that → race (gender etc.) are fundamentally not manipulable (Kohler-Hausmann, ) → non-manipulable variables cannot be causes, because they have no well-de ned counterfactuals (Holland, )
10. W ? It’s o en argued that → race (gender etc.) are fundamentally not manipulable (Kohler-Hausmann, ) → non-manipulable variables cannot be causes, because they have no well-de ned counterfactuals (Holland, ) And argued back either that → at doesn’t matter because the correlates of race, e.g. name, dress, accent, etc. are manipulable (Bertrand & Mullainathan, ; Greiner & Rubin, ) → there may not be much more to race than this anyway (Sen & Wasow, ) → it wouldn’t be a problem if there were (Pearl, ) → the problem is misidenti ed as ontological (VanderWeele & Hern´ an, ) (gender provides a natural comparison case for these responses)
11. A From (Sen & Wasow, )
12. W ? e opposite view is potentially di cult: → Strong essentialism (‘essence’ vs. ‘accident’) last popular in the medieval period → Switching race would be (e ectively) a ‘transformative treatment’ (Paul & Healy, ) (psychotherapy or alcoholism treatment provide natural comparison cases for Paul and Healy’s response) is apparently abstruse theoretical questions matters for policy
13. W ? e opposite view is potentially di cult: → Strong essentialism (‘essence’ vs. ‘accident’) last popular in the medieval period → Switching race would be (e ectively) a ‘transformative treatment’ (Paul & Healy, ) (psychotherapy or alcoholism treatment provide natural comparison cases for Paul and Healy’s response) is apparently abstruse theoretical questions matters for policy Aside: → If race, gender, etc. are protected characteristics, then so much for counterfactual theories of fairness as a way of dealing with them
14. N (Nearly) all the things that could go wrong D M Y V U W
15. E Assume we can measure and control for all these confounders D M Y U However, our data conditions on M D M Y U ρ? is is a mediation problem: → Direct e ect: Conditional on being stopped (M= ), race (D) a ects use of force (Y) → Indirect e ect: Force is only applied in stops (M= ) → Interaction: M= implies Y= (but not vice versa) is is a missing data problem: → M is a missing data indicator. If M = we get to see the case, otherwise not Lots of collider bias potential...
16. E It’s natural to ask → What is the causal e ect of race on use of force Turns out there are a lot of ways to answer this We’ll need to gure out → What are the possible answers → Which of them can be estimated from data → What kind of data we would need to estimate them Even before we start to ask what to do about the answer
17. E Consider a simple scenario with people (courtesy Macartan Humphreys [link]) → Assumption: Race (D) is unrelated to suspicious behaviour (U) P D = U = a A b B c C d D e E S M = if D + U ≥ D = U = a A b B c C d D e E F , = Y = if D + U ≥ D = U = a A b B c C d D e E O D = U = a A b B c C d D e E
18. E : P D = U = a A b B c C d D e E S D = U = a A b B c C d D e E F = D = U = a A b B c C d D e E O D = U = a A b B c C d D e E e causal e ect of D on Y → Proportion of people would (not) have had force applied if they had been the other race →
19. E : P D = U = a A b B c C d D e E S D = U = a A b B c C d D e E F = D = U = a A b B c C d D e E O D = U = a A b B c C d D e E → Proportion of stopped D= that experience force: → Proportion of stopped D= that experience force: Apparent e ect: −
20. E : ( ) P D = U = a A b B c C d D e E S D = U = a A b B c C d D e E F = D = U = a A b B c C d D e E O D = U = a A b B c C d D e E → → Proportion of the population for whom D a ects stopping
21. E : P D = U = a A b B c C d D e E S D = U = a A b B c C d D e E F = D = U = a A b B c C d D e E O D = U = a A b B c C d D e E e e ect of D on Y if everyone were stopped (M = ) → e proportion of people would (not) have had force applied if they had been the other race →
22. E : ( = ) P D = U = a A b B c C d D e E S D = U = a A b B c C d D e E F = D = U = a A b B c C d D e E O D = U = a A b B c C d D e E → → e e ect of D on those who actually were stopped M = → e B, C, D, and E are stopped → C and D have Y =
23. E : ( = ) P D = U = a A b B c C d D e E S D = U = a A b B c C d D e E F = D = U = a A b B c C d D e E O D = U = a A b B c C d D e E Note on ATE(M= ) → C would not have had Y = even if D = (from Table ) → D would would have had Y = even if D = , but then she wouldn’t have been stopped at all
24. E : ( = ) P D = U = a A b B c C d D e E S D = U = a A b B c C d D e E F = D = U = a A b B c C d D e E O D = U = a A b B c C d D e E → → Note the same as ATE(M= )! → Imagine changing D but with M xed to its observed value → D is now counted in the stopped crowd, regardless that they would not have been had they been the other race
25. E : P D = U = a A b B c C d D e E S D = U = a A b B c C d D e E F = D = U = a A b B c C d D e E O D = U = a A b B c C d D e E → Of all uses of force on minorities, how many were due to being a minority? → Subpopulation C, D, E → Only E’s race was irrelevant to the use of force →
26. W ? Which (if any) of these quantities is relevant → for public policy → for studying race → for studying bias → for causal inference
27. W ? ...without knowing D → M → Everything? (Fryer, ) → Everything sometimes? (Gaebler et al., n.d.) → Very little without extra assumptions (Knox et al., a, b)
28. P Divide units into principal strata → Would never have been stopped regardless of race → Would be stopped if D= but not if D= (anti-minority ‘racial stops’) → Would be stopped if D= but not if D= (anti-white ‘racial stops’) → Would be stopped regardless of race If we knew these we could condition on them as pre-treatment covariates (Rubin, ) All causal e ects are weighted averages of them
29. S : → If D → M, four types of police-civilian encounters: Mi( ) = Mi( ) = Mi( ) = “always stop” stop if black (serious crime) (jaywalking) Mi( ) = stop if white “never stop” ? (inconspicuous) What do we get to see in police data?
30. T - → If D → M, four types of police-civilian encounters: Mi( ) = Mi( ) = Mi( ) = “always stop” stop if black (serious crime) (jaywalking) Mi( ) = stop if white “never stop” ? (inconspicuous) What do we get to see in police data?
31. T - → If D → M, four types of police-civilian encounters: Mi( ) = Mi( ) = Mi( ) = “always stop” stop if black (serious crime) (jaywalking) Mi( ) = stop if white “never stop” ? (inconspicuous) What do we get to see in police data?
32. T - → If D → M, four types of police-civilian encounters: Mi( ) = Mi( ) = Mi( ) = “always stop” stop if black (serious crime) (jaywalking) Mi( ) = stop if white “never stop” ? (inconspicuous) What do we get to see in police data?
33. T - → If D → M, four types of police-civilian encounters: Mi( ) = Mi( ) = Mi( ) = “always stop” stop if black (serious crime) (jaywalking) Mi( ) = stop if white “never stop” ? (inconspicuous) What do we get to see in police data?
34. T - → If D → M, four types of police-civilian encounters: Mi( ) = Mi( ) = Mi( ) = “always stop” stop if black (serious crime) (jaywalking) Mi( ) = stop if white “never stop” ? (inconspicuous) What do we get to see in police data?
35. T - → If D → M, four types of police-civilian encounters: Mi( ) = Mi( ) = Mi( ) = “always stop” stop if black (serious crime) (jaywalking) Mi( ) = stop if white “never stop” ? (inconspicuous) For black civilians ...
36. T - → If D → M, four types of police-civilian encounters: Mi( ) = Mi( ) = Mi( ) = “always stop” stop if black (serious crime) (jaywalking) Mi( ) = stop if white “never stop” ? (inconspicuous) For white civilians ...
37. A → Mandatory reporting → Mediator monotonicity: No anti-white ‘racial stops’ → Relative non-severity of racial stops → Treatment ignorability Unsurprisingly we can’t get the ATE Naive estimator is biased for ATE(M= ) → even without unobserved U in the way → bias is always non-positive
38. B → Bias can be re-written in terms of all things that can be directly estimated from data except two: . ρ = Pr(Mi( ) = Di = , Mi = ): share of minority stops due to race (unknown) . θ = E[Y( , ) Di = , Mi( ) = , Mi( ) = ] violence rate among racially stopped minorities → If we knew the joint distribution Pr(Y( , ), Mi( ) = Di = , Mi( ) = ) = Pr(A, B), we could then back out θ → θ = P(A B) = Pr(A,B) Pr(B) = Pr(A,B) ρ → We don’t, but we can place Fr´ echet bounds on Pr(A, B)
39. B , . • • 0 500 1000 1500 0.00 0.25 0.50 0.75 1.00 Proportion of racially discriminatory stops Civilians subject to any racially discriminatory use of force (thousands)
40. W ρ? What is the share of minority stops that would not have happened if civilians had been white? → Can be anywhere in [ , ). If ρ = , bias disappears. → Two prior studies estimate this using data on “Stop, Question and Frisk” in → Gelman, Fagan & Kiss ( ) and Goel, Rao and Schro ( ) → Studies take totally di erent approaches → Results imply ρ is at least . or . , respective → We use ρ =. to be conservative
41. B , .
42. B , .
43. B , .
44. S is particular bit of applied causal inference opened up a lot of conceptually di cult and socially contentious issues: → How to think about race → How to think about fairness → How to think about e ective use of force → e limits of inference from data → What data should be collected When you get yelled at a er being written up in , you’re either doing something very wrong... Or very right...
45. R Bertrand, M. & Mullainathan, S. ( ). ‘Are {e}mily and {g}reg more employable than {l}akisha and {j}amal? {a} eld experiment on labor market discrimination’. e American Economic Review, ( ), – . Fryer, R. G. ( ). ‘An empirical analysis of racial di erences in police use of force’. Journal of Political Economy, ( ), – . Gaebler, J., Cai, W. & Basse, G. (n.d.). Deconstructing claims of post-treatment bias in observational studies of discrimination. Greiner, D. J. & Rubin, D. B. ( ). ‘Causal e ects of perceived immutable characteristics’. e Review of Economics and Statistics, ( ), – . Holland, P. ( ). ‘Causation and race’. ETS Research Report Series, . Knox, D., Lowe, W. & Mummolo, J. ( a). ‘Administrative records mask racially biased policing’. American Political Science Review, ( ), – .
46. R Knox, D., Lowe, W. & Mummolo, J. ( b, August ). Can racial bias in policing be credibly estimated using data contaminated by post-treatment selection? Kohler-Hausmann, I. ( , January ). Eddie murphy and the dangers of counterfactual causal thinking about detecting racial discrimination (SSRN Scholarly Paper No. ID ). Social Science Research Network. Rochester, NY. Paul, L. A. & Healy, K. ( ). ‘Transformative treatments’. Noˆ us, ( ), – . Pearl, J. ( ). ‘Does obesity shorten life? or is it the soda? on non-manipulable causes’. Journal of Causal Inference, ( ). Rubin, D. B. ( ). ‘Causal inference through potential outcomes and principal strati cation: Application to studies with “censoring” due to death’. Statistical Science, ( ), – . Sen, M. & Wasow, O. ( ). ‘Race as a bundle of sticks: Designs that estimate e ects of seemingly immutable characteristics’. Annual Review of Political Science, ( ), – .
47. R VanderWeele, T. J. & Hern´ an, M. A. ( , June ). Causal e ects and natural laws: Towards a conceptualization of causal counterfactuals for nonmanipulable exposures, with application to the e ects of race and sex. In C. Berzuini, P. Dawid & L. Bernardinelli (Eds.), Wiley series in probability and statistics (pp. – ). John Wiley & Sons, Ltd.