Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Colin Carroll, Karin C. Knudson - Fighting Gerrymandering with PyMC3

Colin Carroll, Karin C. Knudson - Fighting Gerrymandering with PyMC3

At the end of 2017, there were seven states with ongoing redistricting litigation. We will discuss a statistical model that the United States Supreme Court declared to be appropriate in cases of racial gerrymandering, and show how it can be implemented and used with the library `PyMC3`. We will also discuss what the model tells us about racial gerrymandering in North Carolina.

https://us.pycon.org/2018/schedule/presentation/110/

PyCon 2018

May 11, 2018
Tweet

More Decks by PyCon 2018

Other Decks in Programming

Transcript

  1. FIGHTING GERRYMANDERING WITH PYMC3 Dr. Karin Knudson and Dr. Colin

    Carroll
  2. PLAN FOR THE DAY History of gerrymandering What do we

    need to model? Ecological inference Modelling fake data Modelling real data
  3. WHERE I’M COMING FROM

  4. THE VOTING RIGHTS ACT OF 1965 “ I want you

    to write me the goddamndest, toughest, voting rights act that you can devise” - LBJ to Attorney General Nicholas Katzenback
  5. "So long as I do not firmly and irrevocably possess

    the right to vote I do not possess myself. I cannot make up my mind — it is made up for me. I cannot live as a democratic citizen, observing the laws I have helped to enact — I can only submit to the edict of others.” - Martin Luther King
  6. VOTING RIGHTS ACT Section 2 - Prohibits racial discrimination in

    voting, vote denial and vote dilution Section 3 - Bail-in Section 4 - Coverage formula Section 5 - Pre-clearance http://www.nytimes.com/interactive/2013/06/23/us/voting-rights-act-map.html
  7. SHELBY V. HOLDER (2013) Section 4 coverage formula unconstitutional Leaves

    section 5 preclearance intact but unenforceable Section 2 intact - challenges still possible and still slow and expensive Will congress write a new coverage formula? Credit Pablo Martinez Monsivais/Associated Press “the Act imposes current burdens and must be justified by current needs.”
  8. THORNBURG V. GINGLES (1986) WHAT DOES A VRA CHALLENGE REQUIRE?

    AMONG OTHER GUIDANCE…. 1. The history of official discrimination in the jurisdiction that affects the right to vote; 2. The degree to which voting in the jurisdiction is racially polarized; 3. The extent of the jurisdiction's use of majority vote requirements, unusually large electoral districts, prohibitions on bullet voting, and other devices that tend to enhance the opportunity for voting discrimination; 4. Whether minority candidates are denied access to the jurisdiction's candidate slating processes, if any; 5. The extent to which the jurisdiction's minorities are discriminated against in socioeconomic areas, such as education, employment, and health; 6. Whether overt or subtle racial appeals in campaigns exist; 7. The extent to which minority candidates have won elections; 8. The degree that elected officials are unresponsive to the concerns of the minority group; and 9. Whether the policy justification for the challenged law is tenuous. 1. compactness - racial/language minority group is “sufficiently numerous and compact to form a majority in a single- member district” 2. the minority group is “politically cohesive” 3. “majority votes sufficiently as a block to enable it…usually to defeat the minority’s preferred candidate” SENATE FACTORS (1982)
  9. QUESTIONS OF INTEREST - ECOLOGICAL INFERENCE GROUP A GROUP B

    Group 1 bi11 = ?? bi12 = ?? X1,i Group 2 bi21 = ?? bi22 = ?? X2,i T1,i T2,i i = 1,…, p i
  10. precinct 1: 90% Dem, 10% Rep, 85% Black, 15% White

    precinct 2: 40% Dem, 60% Rep, 35% Black, 65% White PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i T1,i T2,i QUESTIONS OF INTEREST i = 1,…, p i
  11. QUESTIONS OF INTEREST i = 1,…, p PRECINCT DEMOCRAT REPUBLICAN

    NO VOTE Black bi11 = ?? bi12 = ?? bi13 = ?? X1,i White bi21 = ?? bi22 = ?? bi23 = ?? X2,i Other bi31 = ?? bi32 = ?? bi33 = ?? X3,i T1,i T2,i T3,i i
  12. WISHLIST Give accurate estimates when possible Give possible estimates (e.g.

    rates between 0% and 100%!) Quantify uncertainty Make assumptions explicit Results clearly communicable to courts (!)
  13. GOODMAN’S ECOLOGICAL REGRESSION "requires, in all but the smallest of

    jurisdictions, reliance on computers to perform the calculations." PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i i = 1,…, p Tc,i = b1c X1,i + b21 X2,i + … + brc Xr,i + eic i
  14. METHOD OF BOUNDS precinct 1: 90% Dem, 10% Rep, 85%

    Black, 15% White Note: 0.9 = .85b11 + .15b12 .33 < b12 ≤ 1 PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i
  15. KING’S ECOLOGICAL INFERENCE precinct 1: 70% Dem, 30% Rep, 65%

    Black, 35% White precinct 2: 50% Dem, 50% Rep, 25% Black, 75% White .7 = .65b11 + .35b12 .5 = .25b11 + .75b12 b11 b12
  16. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL c1~ Exponential(λ) d1~

    Exponential(λ) c2~ Exponential(λ) d2~ Exponential(λ) bi1 | c1,d1 ~ Beta(c1,d1) i.i.d. bi2 | c2,d2 ~ Beta(c2,d2) i.i.d. θi=Xi bi1 + (1-Xi)b2i T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) λ c1 d1 c2 d2 bi1 T’i p bi2 King, Rosen, and Tanner
  17. WITH COVARIATES p(α) = p(β) = p(γ) = p(δ)∝1 d1

    ~ Exponential(λ) d2 ~ Exponential(λ) bi1 | Zi, d1, α, β ~ Beta(d1exp(α+βZi ), d1) bi2 | Zi, d2, γ, δ ~ Beta(d2exp(γ+δZi ), d2) θi=Xi b1i + (1-Xi) bi2 T’i ~ Binomial(Ni,θi) Note: log (b1i)/( 1-(b1,i)) =α+βZi
  18. WITH MORE CATEGORIES (RXC) rc ~ Exponential(λ1) i.i.d. bir |

    r ~ Dirichlet(r) i.i.d. r = 1,…,R θic=Xir bi1c + XiRbiRc c = 1… C T’ic ~ Multinomial(Ni,θi) bi1 biR T’i p 1 R λ vectors of length C
  19. WITH MORE CATEGORIES AND COVARIATES dr ~ Exponential (λ) i.i.d.

    r = 1….R r = (dr exp( r1 + r1 Zi),…,dr exp( rC-1 + rC-1 Zi), dr ) r = 1,…,R bri | r ~ Dirichlet(r) i.i.d. r = 1,…,R θic=Xi1 bi1c + XiRbiRc c = 1,…,C T’ic ~ Multinomial(Ni,θi) Note: log (birc)/( 1-(birc)) = rc + rc Zi λ d1 dR T’i p Zi R R bi1 biR 1 1
  20. A HIERARCHICAL BAYESIAN APPROACH - SIMULATION STUDY

  21. SIMULATING DATA # Hidden minority_pct_vote_for_pete = np.random.rand(n_precincts) * 0.3 +

    0.7 majority_pct_vote_for_pete = np.random.rand(n_precincts) * 0.3
  22. SIMULATING DATA # Observed pct_minority = np.random.rand(n_precincts) pct_voting_for_pete = (pct_minority

    * minority_pct_vote_for_pete + (1 - pct_minority) * majority_pct_vote_for_pete)
  23. SIMULATING DATA # Observed voting_population = np.random.randint(100, 10000, size=n_precincts) num_voting_for_pete

    = (pct_voting_for_pete * voting_population)
  24. MODELLING!

  25. VISUALIZING RESULTS

  26. NORTH CAROLINA

  27. NORTH CAROLINA

  28. None
  29. NORTH CAROLINA • 6.8M out of 7.1M voting-age people •

    3.1M out of 4.6M votes • 2,592 voting divisions
  30. NORTH CAROLINA source: cnn.com

  31. politico.com

  32. politico.com

  33. politico.com

  34. None
  35. OVERALL VOTE FOR DEMOCRATIC CANDIDATES

  36. ESTIMATED NON-WHITE VOTE FOR DEMOCRATIC CANDIDATES

  37. ESTIMATED WHITE VOTE FOR DEMOCRATIC CANDIDATES

  38. IT’S A GREAT TIME… TO CARE ABOUT GERRYMANDERING 2010 2013

    2020 2021 Census Census Major redistricting Shelby vs. Holder Supreme Court on gerrymandering in: WI, NC, …. Now-ish
  39. THANK YOU!

  40. • King, Gary and Roberts, Molly.(2016). ei: Ecological Inference. R

    package version 1.3-3. • King, Gary, Ori Rosen and Martin A. Tanner. Ecological Inference: New Methodological Strategies • King, Gary, Ori Rosen, and Martin A. Tanner. "Binomial-beta hierarchical models for ecological inference." Sociological Methods & Research 28.1 (1999): 61-90. • King, Gary (1997). A Solution to the Ecological Inference Problem. Princeton, NJ: Princeton University Press. • Rosen, Ori, et al. "Bayesian and frequentist inference for ecological inference: The R× C case." Statistica Neerlandica 55.2 (2001): 134-156. • James Greiner, “Ecological Inference in Voting Rights Act Disputes: Where Are We Now, and Where Do We Want to Be?”, 47 Jurimetrics J. 115-167 (2007). • Bullock, Charles S. et al. The Rise and Fall of the Voting Rights Act. Oklahoma University Press: Norman, 2006. • Thornburg v. Gingles, 478 U.S. 30,49 (1986). • Shelby County v. Holder, No. 570 U.S 12-96, (2013) • Metric Geometry and Gerrymandering Group https://sites.tufts.edu/gerrymandr/ (particular acknowledgment to Dr. Mira Bernstein of the MGGG and Dr. Megan A. Gall of the Lawyers’ Committee for Civil Rights Under Law) • NC data from: OpenElections http://www.openelections.net/ and the US Census