Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fighting Gerrymandering with PyMC3

23752e5d8b8ce3c8f0ade1aca0ee2408?s=47 Karin Knudson
May 12, 2018
190

Fighting Gerrymandering with PyMC3

At the end of 2017, there were seven states with ongoing redistricting litigation. We will discuss a statistical model that is relevant to certain cases of racial gerrymandering and vote dilution under the Voting Rights Act, and show how it can be implemented and used with the library PyMC3. We will also discuss what the model tells us about racial gerrymandering in North Carolina.

23752e5d8b8ce3c8f0ade1aca0ee2408?s=128

Karin Knudson

May 12, 2018
Tweet

Transcript

  1. FIGHTING GERRYMANDERING WITH PYMC3 Dr. Karin Knudson and Dr. Colin

    Carroll
  2. None
  3. PLAN FOR THE DAY History of the Voting Rights Act

    What do we need to model? Ecological inference Modelling fake data Modelling real data
  4. WHERE I’M COMING FROM

  5. THE VOTING RIGHTS ACT OF 1965 “ I want you

    to write me the goddamndest, toughest, voting rights act that you can devise.” - LBJ to Attorney General Nicholas Katzenback
  6. "So long as I do not firmly and irrevocably possess

    the right to vote I do not possess myself. I cannot make up my mind — it is made up for me. I cannot live as a democratic citizen, observing the laws I have helped to enact — I can only submit to the edict of others.” - Martin Luther King
  7. VOTING RIGHTS ACT Section 2 - Prohibits racial discrimination in

    voting, vote denial and vote dilution Section 3 - Bail-in Section 4 - Coverage formula Section 5 - Pre-clearance http://www.nytimes.com/interactive/2013/06/23/us/voting-rights-act-map.html
  8. SHELBY V. HOLDER (2013) Section 4 coverage formula unconstitutional Leaves

    section 5 preclearance intact but unenforceable Section 2 intact - challenges still possible and still slow and expensive Will congress write a new coverage formula? Credit Pablo Martinez Monsivais/Associated Press “the Act imposes current burdens and must be justified by current needs.”
  9. THORNBURG V. GINGLES (1986) WHAT DOES A VRA CHALLENGE REQUIRE?

    AMONG OTHER GUIDANCE…. 1. The history of official discrimination in the jurisdiction that affects the right to vote; 2. The degree to which voting in the jurisdiction is racially polarized; 3. The extent of the jurisdiction's use of majority vote requirements, unusually large electoral districts, prohibitions on bullet voting, and other devices that tend to enhance the opportunity for voting discrimination; 4. Whether minority candidates are denied access to the jurisdiction's candidate slating processes, if any; 5. The extent to which the jurisdiction's minorities are discriminated against in socioeconomic areas, such as education, employment, and health; 6. Whether overt or subtle racial appeals in campaigns exist; 7. The extent to which minority candidates have won elections; 8. The degree that elected officials are unresponsive to the concerns of the minority group; and 9. Whether the policy justification for the challenged law is tenuous. 1. compactness - racial/language minority group is “sufficiently numerous and compact to form a majority in a single- member district” 2. the minority group is “politically cohesive” 3. “majority votes sufficiently as a block to enable it…usually to defeat the minority’s preferred candidate” SENATE FACTORS (1982)
  10. THORNBURG V. GINGLES (1986) WHAT DOES A VRA CHALLENGE REQUIRE?

    AMONG OTHER GUIDANCE…. ….. 2. The degree to which voting in the jurisdiction is racially polarized; …. 1. compactness - racial/language minority group is “sufficiently numerous and compact to form a majority in a single-member district” 2. the minority group is “politically cohesive” 3. “majority votes sufficiently as a block to enable it…usually to defeat the minority’s preferred candidate” SENATE FACTORS (1982)
  11. QUESTIONS OF INTEREST - ECOLOGICAL INFERENCE GROUP A GROUP B

    Group 1 bi11 = ?? bi12 = ?? X1,i Group 2 bi21 = ?? bi22 = ?? X2,i T1,i T2,i i = 1,…, p i
  12. QUESTIONS OF INTEREST - ECOLOGICAL INFERENCE GROUP A GROUP B

    Group 1 bi11 = ?? bi12 = ?? X1,i Group 2 bi21 = ?? bi22 = ?? X2,i T1,i T2,i i = 1,…, p i Census Election
  13. precinct 1: 90% Dem, 10% Rep, 85% Black, 15% White

    precinct 2: 40% Dem, 60% Rep, 35% Black, 65% White PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i T1,i T2,i QUESTIONS OF INTEREST- ECOLOGICAL INFERENCE i = 1,…, p i
  14. precinct 1: 90% Dem, 10% Rep, 85% Black, 15% White

    precinct 2: 40% Dem, 60% Rep, 35% Black, 65% White PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i T1,i T2,i QUESTIONS OF INTEREST- ECOLOGICAL INFERENCE i = 1,…, p i Census Election
  15. QUESTIONS OF INTEREST- ECOLOGICAL INFERENCE i = 1,…, p PRECINCT

    DEMOCRAT REPUBLICAN NO VOTE Black bi11 = ?? bi12 = ?? bi13 = ?? X1,i White bi21 = ?? bi22 = ?? bi23 = ?? X2,i Other bi31 = ?? bi32 = ?? bi33 = ?? X3,i T1,i T2,i T3,i i
  16. WISHLIST Give accurate estimates when possible Give possible estimates (e.g.

    rates between 0% and 100%!) Quantify uncertainty Make assumptions explicit Results clearly communicable to courts (!)
  17. GOODMAN’S ECOLOGICAL REGRESSION PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ??

    bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i i = 1,…, p Tc,i = b1c X1,i + b21 X2,i + … + brc Xr,i + eic i
  18. GOODMAN’S ECOLOGICAL REGRESSION PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ??

    bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i i = 1,…, p Tc,i = b1c X1,i + b21 X2,i + … + brc Xr,i + eic i Election
  19. GOODMAN’S ECOLOGICAL REGRESSION PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ??

    bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i i = 1,…, p Tc,i = b1c X1,i + b21 X2,i + … + brc Xr,i + eic i Election Census
  20. "requires, in all but the smallest of jurisdictions, reliance on

    computers to perform the calculations." GOODMAN’S ECOLOGICAL REGRESSION
  21. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* bi1 T’i bi2

    *King, Rosen, and Tanner 1999; Rosen et al. 2001
  22. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* bi1 T’i bi2

    *King, Rosen, and Tanner 1999; Rosen et al. 2001 T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) θi=Xi bi1 + (1-Xi)b2i
  23. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* bi1 T’i bi2

    *King, Rosen, and Tanner 1999; Rosen et al. 2001 T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) θi=Xi bi1 + (1-Xi)b2i Election Census
  24. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* bi1 T’i p

    bi2 *King, Rosen, and Tanner 1999; Rosen et al. 2001 T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) θi=Xi bi1 + (1-Xi)b2i Election Census
  25. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* c1 d1 c2

    d2 bi1 T’i p bi2 *King, Rosen, and Tanner 1999; Rosen et al. 2001 T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) bi1 | c1,d1 ~ Beta(c1,d1) i.i.d. bi2 | c2,d2 ~ Beta(c2,d2) i.i.d. θi=Xi bi1 + (1-Xi)b2i Election Census
  26. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* λ c1 d1

    c2 d2 bi1 T’i p bi2 *King, Rosen, and Tanner 1999; Rosen et al. 2001 T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) bi1 | c1,d1 ~ Beta(c1,d1) i.i.d. bi2 | c2,d2 ~ Beta(c2,d2) i.i.d. c1~ Exponential(λ) d1~ Exponential(λ) c2~ Exponential(λ) d2~ Exponential(λ) θi=Xi bi1 + (1-Xi)b2i Election Census
  27. WITH COVARIATES p(α) = p(β) = p(γ) = p(δ)∝1 d1

    ~ Exponential(λ) d2 ~ Exponential(λ) bi1 | Zi, d1, α, β ~ Beta(d1exp(α+βZi ), d1) bi2 | Zi, d2, γ, δ ~ Beta(d2exp(γ+δZi ), d2) θi=Xi b1i + (1-Xi) bi2 T’i ~ Binomial(Ni,θi) Note: log (b1i)/( 1-(b1,i)) =α+βZi
  28. WITH MORE CATEGORIES (RXC) rc ~ Exponential(λ1) i.i.d. bir |

    r ~ Dirichlet(r) i.i.d. r = 1,…,R θic=Xir bi1c + XiRbiRc c = 1… C T’ic ~ Multinomial(Ni,θi) bi1 biR T’i p 1 R λ vectors of length C
  29. WITH MORE CATEGORIES AND COVARIATES dr ~ Exponential (λ) i.i.d.

    r = 1….R r = (dr exp( r1 + r1 Zi),…,dr exp( rC-1 + rC-1 Zi), dr ) r = 1,…,R bri | r ~ Dirichlet(r) i.i.d. r = 1,…,R θic=Xi1 bi1c + XiRbiRc c = 1,…,C T’ic ~ Multinomial(Ni,θi) Note: log (birc)/( 1-(birc)) = rc + rc Zi λ d1 dR T’i p Zi R R bi1 biR 1 1
  30. A HIERARCHICAL BAYESIAN APPROACH - SIMULATION STUDY

  31. A HIERARCHICAL BAYESIAN APPROACH - SIMULATION STUDY PETE 2018 FOR

    PRESIDENT MORE HEAD PATS FEWER LOUD NOISES
  32. SIMULATING DATA n_precincts = 14 # Hidden minority_pete = np.random.rand(n_precincts)

    * 0.3 + 0.7 majority_pete = np.random.rand(n_precincts) * 0.3
  33. SIMULATING DATA # Observed pct_minority = np.random.rand(n_precincts) pct_for_pete = (pct_minority

    * minority_pete + (1 - pct_minority) * majority_pete)
  34. SIMULATING DATA # Observed pct_minority = np.random.rand(n_precincts) pct_for_pete = (pct_minority

    * minority_pete + (1 - pct_minority) * majority_pete) Census Election
  35. SIMULATING DATA # Observed voting_population = np.random.randint(100, 10000, size=n_precincts) num_voting_for_pete

    = pct_for_pete * voting_population
  36. SIMULATING DATA # Observed voting_population = np.random.randint(100, 10000, size=n_precincts) num_voting_for_pete

    = pct_for_pete * voting_population Census Election
  37. MODELLING!

  38. VISUALIZING RESULTS

  39. VISUALIZING RESULTS

  40. VISUALIZING RESULTS

  41. VISUALIZING RESULTS

  42. NORTH CAROLINA

  43. NORTH CAROLINA

  44. None
  45. NORTH CAROLINA • 6.8M out of 7.1M voting-age people •

    3.1M out of 4.6M votes • 2,592 voting divisions
  46. NORTH CAROLINA • 6.8M out of 7.1M voting-age people •

    3.1M out of 4.6M votes • 2,592 voting divisions
  47. NORTH CAROLINA • 6.8M out of 7.1M voting-age people •

    3.1M out of 4.6M votes • 2,592 voting divisions
  48. NORTH CAROLINA source: cnn.com

  49. politico.com

  50. politico.com

  51. politico.com

  52. None
  53. OVERALL VOTE FOR DEMOCRATIC CANDIDATES

  54. ESTIMATED NON-WHITE VOTE FOR DEMOCRATIC CANDIDATES

  55. ESTIMATED WHITE VOTE FOR DEMOCRATIC CANDIDATES

  56. IT’S A GREAT TIME… TO CARE ABOUT GERRYMANDERING 2010 2013

    2020 2021 Census Census Major redistricting Shelby vs. Holder Supreme Court on gerrymandering in: WI, NC, MD…. Now-ish
  57. THANK YOU! github.com/ColCarroll/redistricting-pymc3-pycon-2018

  58. • King, Gary and Roberts, Molly.(2016). ei: Ecological Inference. R

    package version 1.3-3. • King, Gary, Ori Rosen and Martin A. Tanner. Ecological Inference: New Methodological Strategies • King, Gary, Ori Rosen, and Martin A. Tanner. "Binomial-beta hierarchical models for ecological inference." Sociological Methods & Research 28.1 (1999): 61-90. • King, Gary (1997). A Solution to the Ecological Inference Problem. Princeton, NJ: Princeton University Press. • Rosen, Ori, et al. "Bayesian and frequentist inference for ecological inference: The R× C case." Statistica Neerlandica 55.2 (2001): 134-156. • James Greiner, “Ecological Inference in Voting Rights Act Disputes: Where Are We Now, and Where Do We Want to Be?”, 47 Jurimetrics J. 115-167 (2007). • Bullock, Charles S. et al. The Rise and Fall of the Voting Rights Act. Oklahoma University Press: Norman, 2006. • Thornburg v. Gingles, 478 U.S. 30,49 (1986). • Shelby County v. Holder, No. 570 U.S 12-96, (2013) • Metric Geometry and Gerrymandering Group https://sites.tufts.edu/gerrymandr/ (particular acknowledgment to Dr. Mira Bernstein of the MGGG and Dr. Megan A. Gall of the Lawyers’ Committee for Civil Rights Under Law) • NC data from: OpenElections http://www.openelections.net/ and the US Census
  59. KING’S ECOLOGICAL INFERENCE precinct 1: 70% Dem, 30% Rep, 65%

    Black, 35% White precinct 2: 50% Dem, 50% Rep, 25% Black, 75% White .70 = .65b11 + .35b12 .50 = .25b11 + .75b12 b11 b12
  60. METHOD OF BOUNDS precinct 1: 90% Dem, 10% Rep, 85%

    Black, 15% White Note: 0.9 = .85b11 + .15b12 .33 < b12 ≤ 1 PRECINCT DEMOCRAT REPUBLICAN Black bi11 = ?? bi12 = ?? X1,i White bi21 = ?? bi22 = ?? X2,i =1 - X1,i T1,i T2,i = 1 - T1,i
  61. A HIERARCHICAL BAYESIAN APPROACH - THE MODEL* c1~ Exponential(λ) d1~

    Exponential(λ) c2~ Exponential(λ) d2~ Exponential(λ) bi1 | c1,d1 ~ Beta(c1,d1) i.i.d. bi2 | c2,d2 ~ Beta(c2,d2) i.i.d. θi=Xi bi1 + (1-Xi)b2i T’i | bi1 bi2, Xi ~ Binomial(Ni,θi) λ c1 d1 c2 d2 bi1 T’i p bi2 *King, Rosen, and Tanner