Fighting Gerrymandering with PyMC3

At the end of 2017, there were seven states with ongoing redistricting litigation. We will discuss a statistical model that is relevant to certain cases of racial gerrymandering and vote dilution under the Voting Rights Act, and show how it can be implemented and used with the library PyMC3. We will also discuss what the model tells us about racial gerrymandering in North Carolina.

Transcript

1. FIGHTING GERRYMANDERING WITH PYMC3 Dr. Karin Knudson and Dr. Colin Carroll
3. PLAN FOR THE DAY History of the Voting Rights Act What do we need to model? Ecological inference Modelling fake data Modelling real data
4. WHERE I’M COMING FROM
5. THE VOTING RIGHTS ACT OF 1965 “ I want you to write me the goddamndest, toughest, voting rights act that you can devise.” - LBJ to Attorney General Nicholas Katzenback
6. "So long as I do not firmly and irrevocably possess the right to vote I do not possess myself. I cannot make up my mind — it is made up for me. I cannot live as a democratic citizen, observing the laws I have helped to enact — I can only submit to the edict of others.” - Martin Luther King
7. VOTING RIGHTS ACT Section 2 - Prohibits racial discrimination in voting, vote denial and vote dilution Section 3 - Bail-in Section 4 - Coverage formula Section 5 - Pre-clearance http://www.nytimes.com/interactive/2013/06/23/us/voting-rights-act-map.html
8. SHELBY V. HOLDER (2013) Section 4 coverage formula unconstitutional Leaves section 5 preclearance intact but unenforceable Section 2 intact - challenges still possible and still slow and expensive Will congress write a new coverage formula? Credit Pablo Martinez Monsivais/Associated Press “the Act imposes current burdens and must be justified by current needs.”
9. THORNBURG V. GINGLES (1986) WHAT DOES A VRA CHALLENGE REQUIRE? AMONG OTHER GUIDANCE…. 1. The history of official discrimination in the jurisdiction that affects the right to vote; 2. The degree to which voting in the jurisdiction is racially polarized; 3. The extent of the jurisdiction's use of majority vote requirements, unusually large electoral districts, prohibitions on bullet voting, and other devices that tend to enhance the opportunity for voting discrimination; 4. Whether minority candidates are denied access to the jurisdiction's candidate slating processes, if any; 5. The extent to which the jurisdiction's minorities are discriminated against in socioeconomic areas, such as education, employment, and health; 6. Whether overt or subtle racial appeals in campaigns exist; 7. The extent to which minority candidates have won elections; 8. The degree that elected officials are unresponsive to the concerns of the minority group; and 9. Whether the policy justification for the challenged law is tenuous. 1. compactness - racial/language minority group is “sufficiently numerous and compact to form a majority in a single- member district” 2. the minority group is “politically cohesive” 3. “majority votes sufficiently as a block to enable it…usually to defeat the minority’s preferred candidate” SENATE FACTORS (1982)
10. THORNBURG V. GINGLES (1986) WHAT DOES A VRA CHALLENGE REQUIRE? AMONG OTHER GUIDANCE…. ….. 2. The degree to which voting in the jurisdiction is racially polarized; …. 1. compactness - racial/language minority group is “sufficiently numerous and compact to form a majority in a single-member district” 2. the minority group is “politically cohesive” 3. “majority votes sufficiently as a block to enable it…usually to defeat the minority’s preferred candidate” SENATE FACTORS (1982)
11. QUESTIONS OF INTEREST - ECOLOGICAL INFERENCE GROUP A GROUP B Group 1 bi11 = ?? bi12 = ?? X1,i Group 2 bi21 = ?? bi22 = ?? X2,i T1,i T2,i i = 1,…, p i
12. QUESTIONS OF INTEREST - ECOLOGICAL INFERENCE GROUP A GROUP B Group 1 bi11 = ?? bi12 = ?? X1,i Group 2 bi21 = ?? bi22 = ?? X2,i T1,i T2,i i = 1,…, p i Census Election
13. precinct 1: 90% Dem, 10% Rep, 85% Black, 15% White precinct 2: 40% Dem, 60% Rep, 35% Black, 65% White PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i T1,i T2,i QUESTIONS OF INTEREST- ECOLOGICAL INFERENCE i = 1,…, p i
14. precinct 1: 90% Dem, 10% Rep, 85% Black, 15% White precinct 2: 40% Dem, 60% Rep, 35% Black, 65% White PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i T1,i T2,i QUESTIONS OF INTEREST- ECOLOGICAL INFERENCE i = 1,…, p i Census Election
15. QUESTIONS OF INTEREST- ECOLOGICAL INFERENCE i = 1,…, p PRECINCT DEMOCRAT REPUBLICAN NO VOTE Black bi11 = ?? bi12 = ?? bi13 = ?? X1,i White bi21 = ?? bi22 = ?? bi23 = ?? X2,i Other bi31 = ?? bi32 = ?? bi33 = ?? X3,i T1,i T2,i T3,i i
16. WISHLIST Give accurate estimates when possible Give possible estimates (e.g. rates between 0% and 100%!) Quantify uncertainty Make assumptions explicit Results clearly communicable to courts (!)
17. GOODMAN’S ECOLOGICAL REGRESSION PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i i = 1,…, p Tc,i = b1c X1,i + b21 X2,i + … + brc Xr,i + eic i
18. GOODMAN’S ECOLOGICAL REGRESSION PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i i = 1,…, p Tc,i = b1c X1,i + b21 X2,i + … + brc Xr,i + eic i Election
19. GOODMAN’S ECOLOGICAL REGRESSION PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i i = 1,…, p Tc,i = b1c X1,i + b21 X2,i + … + brc Xr,i + eic i Election Census
20. "requires, in all but the smallest of jurisdictions, reliance on computers to perform the calculations." GOODMAN’S ECOLOGICAL REGRESSION
21. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* bi1 T’i bi2 *King, Rosen, and Tanner 1999; Rosen et al. 2001
22. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* bi1 T’i bi2 *King, Rosen, and Tanner 1999; Rosen et al. 2001 T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) θi=Xi bi1 + (1-Xi)b2i
23. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* bi1 T’i bi2 *King, Rosen, and Tanner 1999; Rosen et al. 2001 T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) θi=Xi bi1 + (1-Xi)b2i Election Census
24. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* bi1 T’i p bi2 *King, Rosen, and Tanner 1999; Rosen et al. 2001 T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) θi=Xi bi1 + (1-Xi)b2i Election Census
25. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* c1 d1 c2 d2 bi1 T’i p bi2 *King, Rosen, and Tanner 1999; Rosen et al. 2001 T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) bi1 | c1,d1 ~ Beta(c1,d1) i.i.d. bi2 | c2,d2 ~ Beta(c2,d2) i.i.d. θi=Xi bi1 + (1-Xi)b2i Election Census
26. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* λ c1 d1 c2 d2 bi1 T’i p bi2 *King, Rosen, and Tanner 1999; Rosen et al. 2001 T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) bi1 | c1,d1 ~ Beta(c1,d1) i.i.d. bi2 | c2,d2 ~ Beta(c2,d2) i.i.d. c1~ Exponential(λ) d1~ Exponential(λ) c2~ Exponential(λ) d2~ Exponential(λ) θi=Xi bi1 + (1-Xi)b2i Election Census
27. WITH COVARIATES p(α) = p(β) = p(γ) = p(δ)∝1 d1 ~ Exponential(λ) d2 ~ Exponential(λ) bi1 | Zi, d1, α, β ~ Beta(d1exp(α+βZi ), d1) bi2 | Zi, d2, γ, δ ~ Beta(d2exp(γ+δZi ), d2) θi=Xi b1i + (1-Xi) bi2 T’i ~ Binomial(Ni,θi) Note: log (b1i)/( 1-(b1,i)) =α+βZi
28. WITH MORE CATEGORIES (RXC) rc ~ Exponential(λ1) i.i.d. bir | r ~ Dirichlet(r) i.i.d. r = 1,…,R θic=Xir bi1c + XiRbiRc c = 1… C T’ic ~ Multinomial(Ni,θi) bi1 biR T’i p 1 R λ vectors of length C
29. WITH MORE CATEGORIES AND COVARIATES dr ~ Exponential (λ) i.i.d. r = 1….R r = (dr exp( r1 + r1 Zi),…,dr exp( rC-1 + rC-1 Zi), dr ) r = 1,…,R bri | r ~ Dirichlet(r) i.i.d. r = 1,…,R θic=Xi1 bi1c + XiRbiRc c = 1,…,C T’ic ~ Multinomial(Ni,θi) Note: log (birc)/( 1-(birc)) = rc + rc Zi λ d1 dR T’i p Zi R R bi1 biR 1 1
30. A HIERARCHICAL BAYESIAN APPROACH - SIMULATION STUDY
31. A HIERARCHICAL BAYESIAN APPROACH - SIMULATION STUDY PETE 2018 FOR PRESIDENT MORE HEAD PATS FEWER LOUD NOISES
32. SIMULATING DATA n_precincts = 14 # Hidden minority_pete = np.random.rand(n_precincts) * 0.3 + 0.7 majority_pete = np.random.rand(n_precincts) * 0.3
33. SIMULATING DATA # Observed pct_minority = np.random.rand(n_precincts) pct_for_pete = (pct_minority * minority_pete + (1 - pct_minority) * majority_pete)
34. SIMULATING DATA # Observed pct_minority = np.random.rand(n_precincts) pct_for_pete = (pct_minority * minority_pete + (1 - pct_minority) * majority_pete) Census Election
35. SIMULATING DATA # Observed voting_population = np.random.randint(100, 10000, size=n_precincts) num_voting_for_pete = pct_for_pete * voting_population
36. SIMULATING DATA # Observed voting_population = np.random.randint(100, 10000, size=n_precincts) num_voting_for_pete = pct_for_pete * voting_population Census Election
37. MODELLING!
38. VISUALIZING RESULTS
39. VISUALIZING RESULTS
40. VISUALIZING RESULTS
41. VISUALIZING RESULTS
42. NORTH CAROLINA
43. NORTH CAROLINA
45. NORTH CAROLINA • 6.8M out of 7.1M voting-age people • 3.1M out of 4.6M votes • 2,592 voting divisions
46. NORTH CAROLINA • 6.8M out of 7.1M voting-age people • 3.1M out of 4.6M votes • 2,592 voting divisions
47. NORTH CAROLINA • 6.8M out of 7.1M voting-age people • 3.1M out of 4.6M votes • 2,592 voting divisions
48. NORTH CAROLINA source: cnn.com
49. politico.com
50. politico.com
51. politico.com
53. OVERALL VOTE FOR DEMOCRATIC CANDIDATES
54. ESTIMATED NON-WHITE VOTE FOR DEMOCRATIC CANDIDATES
55. ESTIMATED WHITE VOTE FOR DEMOCRATIC CANDIDATES
56. IT’S A GREAT TIME… TO CARE ABOUT GERRYMANDERING 2010 2013 2020 2021 Census Census Major redistricting Shelby vs. Holder Supreme Court on gerrymandering in: WI, NC, MD…. Now-ish
57. THANK YOU! github.com/ColCarroll/redistricting-pymc3-pycon-2018
58. • King, Gary and Roberts, Molly.(2016). ei: Ecological Inference. R package version 1.3-3. • King, Gary, Ori Rosen and Martin A. Tanner. Ecological Inference: New Methodological Strategies • King, Gary, Ori Rosen, and Martin A. Tanner. "Binomial-beta hierarchical models for ecological inference." Sociological Methods & Research 28.1 (1999): 61-90. • King, Gary (1997). A Solution to the Ecological Inference Problem. Princeton, NJ: Princeton University Press. • Rosen, Ori, et al. "Bayesian and frequentist inference for ecological inference: The R× C case." Statistica Neerlandica 55.2 (2001): 134-156. • James Greiner, “Ecological Inference in Voting Rights Act Disputes: Where Are We Now, and Where Do We Want to Be?”, 47 Jurimetrics J. 115-167 (2007). • Bullock, Charles S. et al. The Rise and Fall of the Voting Rights Act. Oklahoma University Press: Norman, 2006. • Thornburg v. Gingles, 478 U.S. 30,49 (1986). • Shelby County v. Holder, No. 570 U.S 12-96, (2013) • Metric Geometry and Gerrymandering Group https://sites.tufts.edu/gerrymandr/ (particular acknowledgment to Dr. Mira Bernstein of the MGGG and Dr. Megan A. Gall of the Lawyers’ Committee for Civil Rights Under Law) • NC data from: OpenElections http://www.openelections.net/ and the US Census
59. KING’S ECOLOGICAL INFERENCE precinct 1: 70% Dem, 30% Rep, 65% Black, 35% White precinct 2: 50% Dem, 50% Rep, 25% Black, 75% White .70 = .65b11 + .35b12 .50 = .25b11 + .75b12 b11 b12
60. METHOD OF BOUNDS precinct 1: 90% Dem, 10% Rep, 85% Black, 15% White Note: 0.9 = .85b11 + .15b12 .33 < b12 ≤ 1 PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i
61. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* c1~ Exponential(λ) d1~ Exponential(λ) c2~ Exponential(λ) d2~ Exponential(λ) bi1 | c1,d1 ~ Beta(c1,d1) i.i.d. bi2 | c2,d2 ~ Beta(c2,d2) i.i.d. θi=Xi bi1 + (1-Xi)b2i T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) λ c1 d1 c2 d2 bi1 T’i p bi2 *King, Rosen, and Tanner