Fighting Gerrymandering with PyMC3

At the end of 2017, there were seven states with ongoing redistricting litigation. We will discuss a statistical model that is relevant to certain cases of racial gerrymandering and vote dilution under the Voting Rights Act, and show how it can be implemented and used with the library PyMC3. We will also discuss what the model tells us about racial gerrymandering in North Carolina.

Transcript

1. FIGHTING GERRYMANDERING
   WITH PYMC3
   Dr. Karin Knudson and Dr. Colin Carroll
   

   View Slide

2. View Slide

3. PLAN FOR THE DAY
   History of the Voting Rights Act
   What do we need to model?
   Ecological inference
   Modelling fake data
   Modelling real data
   

   View Slide

4. WHERE I’M COMING FROM
   

   View Slide

5. THE VOTING
   RIGHTS ACT OF
   1965
   “ I want you to write me the
   goddamndest, toughest, voting
   rights act that you can devise.”
   - LBJ to Attorney General
   Nicholas Katzenback
   

   View Slide

6. "So long as I do not firmly
   and irrevocably possess the
   right to vote I do not
   possess myself. I cannot
   make up my mind — it is
   made up for me. I cannot live
   as a democratic citizen,
   observing the laws I have
   helped to enact — I can only
   submit to the edict of
   others.”
   - Martin Luther King
   

   View Slide

7. VOTING RIGHTS ACT
   Section 2 - Prohibits racial
   discrimination in voting, vote
   denial and vote dilution
   Section 3 - Bail-in
   Section 4 - Coverage formula
   Section 5 - Pre-clearance
   http://www.nytimes.com/interactive/2013/06/23/us/voting-rights-act-map.html
   

   View Slide

8. SHELBY V. HOLDER (2013)
   Section 4 coverage formula
   unconstitutional
   Leaves section 5 preclearance
   intact but unenforceable
   Section 2 intact - challenges still
   possible and still slow and
   expensive
   Will congress write a new
   coverage formula?
   Credit
   Pablo Martinez Monsivais/Associated Press
   “the Act imposes current burdens
   and must be justified by current
   needs.”
   

   View Slide

9. THORNBURG V. GINGLES (1986)
   WHAT DOES A VRA CHALLENGE REQUIRE? AMONG OTHER GUIDANCE….
   1. The history of official discrimination in the jurisdiction that affects the right to vote;
   2. The degree to which voting in the jurisdiction is racially polarized;
   3. The extent of the jurisdiction's use of majority vote requirements, unusually large electoral districts,
   prohibitions on bullet voting, and other devices that tend to enhance the opportunity for voting discrimination;
   4. Whether minority candidates are denied access to the jurisdiction's candidate slating processes, if any;
   5. The extent to which the jurisdiction's minorities are discriminated against in socioeconomic areas, such as
   education, employment, and health;
   6. Whether overt or subtle racial appeals in campaigns exist;
   7. The extent to which minority candidates have won elections;
   8. The degree that elected officials are unresponsive to the concerns of the minority group; and
   9. Whether the policy justification for the challenged law is tenuous.
   1. compactness - racial/language minority group is “sufficiently numerous and compact to form a majority in a single-
   member district”
   2. the minority group is “politically cohesive”
   3. “majority votes sufficiently as a block to enable it…usually to defeat the minority’s
   preferred candidate”
   SENATE FACTORS (1982)
   

   View Slide

10. THORNBURG V. GINGLES (1986)
    WHAT DOES A VRA CHALLENGE REQUIRE? AMONG OTHER GUIDANCE….
    …..
    2. The degree to which voting in the jurisdiction is racially polarized;
    ….
    1. compactness - racial/language minority group is “sufficiently numerous and compact to
    form a majority in a single-member district”
    2. the minority group is “politically cohesive”
    3. “majority votes sufficiently as a block to enable it…usually to defeat
    the minority’s preferred candidate”
    SENATE FACTORS (1982)
    

    View Slide

11. QUESTIONS OF INTEREST -
    ECOLOGICAL INFERENCE
    GROUP A GROUP B
    Group 1 bi11 = ?? bi12 = ?? X1,i
    Group 2 bi21 = ?? bi22 = ?? X2,i
    T1,i T2,i
    i = 1,…, p
    i
    

    View Slide

12. QUESTIONS OF INTEREST -
    ECOLOGICAL INFERENCE
    GROUP A GROUP B
    Group 1 bi11 = ?? bi12 = ?? X1,i
    Group 2 bi21 = ?? bi22 = ?? X2,i
    T1,i T2,i
    i = 1,…, p
    i
    Census
    Election
    

    View Slide

13. precinct 1: 90% Dem, 10% Rep, 85% Black, 15% White
    precinct 2: 40% Dem, 60% Rep, 35% Black, 65% White
    PRECINCT DEMOCRAT REPUBLICAN
    Black bi11 = ?? bi12 = ?? X1,i
    White bi21 = ?? bi22 = ?? X2,i
    T1,i T2,i
    QUESTIONS OF INTEREST-
    ECOLOGICAL INFERENCE
    i = 1,…, p
    i
    

    View Slide

14. precinct 1: 90% Dem, 10% Rep, 85% Black, 15% White
    precinct 2: 40% Dem, 60% Rep, 35% Black, 65% White
    PRECINCT DEMOCRAT REPUBLICAN
    Black bi11 = ?? bi12 = ?? X1,i
    White bi21 = ?? bi22 = ?? X2,i
    T1,i T2,i
    QUESTIONS OF INTEREST-
    ECOLOGICAL INFERENCE
    i = 1,…, p
    i
    Census
    Election
    

    View Slide

15. QUESTIONS OF INTEREST-
    ECOLOGICAL INFERENCE
    i = 1,…, p
    PRECINCT DEMOCRAT REPUBLICAN NO VOTE
    Black bi11 = ?? bi12 = ?? bi13 = ?? X1,i
    White bi21 = ?? bi22 = ?? bi23 = ?? X2,i
    Other bi31 = ?? bi32 = ?? bi33 = ?? X3,i
    T1,i T2,i T3,i
    i
    

    View Slide

16. WISHLIST
    Give accurate estimates when possible
    Give possible estimates (e.g. rates between 0% and 100%!)
    Quantify uncertainty
    Make assumptions explicit
    Results clearly communicable to courts (!)
    

    View Slide

17. GOODMAN’S ECOLOGICAL
    REGRESSION
    PRECINCT DEMOCRAT REPUBLICAN
    Black bi11 = ?? bi12 = ?? X1,i
    White bi21 = ?? bi22 = ?? X2,i =1 - X1,i
    T1,i T2,i = 1 - T1,i
    i = 1,…, p
    Tc,i = b1c X1,i + b21 X2,i + … + brc Xr,i + eic
    i
    

    View Slide

18. GOODMAN’S ECOLOGICAL
    REGRESSION
    PRECINCT DEMOCRAT REPUBLICAN
    Black bi11 = ?? bi12 = ?? X1,i
    White bi21 = ?? bi22 = ?? X2,i =1 - X1,i
    T1,i T2,i = 1 - T1,i
    i = 1,…, p
    Tc,i = b1c X1,i + b21 X2,i + … + brc Xr,i + eic
    i
    Election
    

    View Slide

19. GOODMAN’S ECOLOGICAL
    REGRESSION
    PRECINCT DEMOCRAT REPUBLICAN
    Black bi11 = ?? bi12 = ?? X1,i
    White bi21 = ?? bi22 = ?? X2,i =1 - X1,i
    T1,i T2,i = 1 - T1,i
    i = 1,…, p
    Tc,i = b1c X1,i + b21 X2,i + … + brc Xr,i + eic
    i
    Election
    Census
    

    View Slide

20. "requires, in all but the smallest of jurisdictions,
    reliance on computers to perform the
    calculations."
    GOODMAN’S ECOLOGICAL
    REGRESSION
    

    View Slide

21. A HIERARCHICAL BAYESIAN
    APPROACH - THE MODEL*
    bi1
    T’i
    bi2
    *King, Rosen, and Tanner 1999; Rosen et al. 2001
    

    View Slide

22. A HIERARCHICAL BAYESIAN
    APPROACH - THE MODEL*
    bi1
    T’i
    bi2
    *King, Rosen, and Tanner 1999; Rosen et al. 2001
    T’i | bi1 bi2, Xi ~ Binomial(Ni,θi)
    θi=Xi bi1 + (1-Xi)b2i
    

    View Slide

23. A HIERARCHICAL BAYESIAN
    APPROACH - THE MODEL*
    bi1
    T’i
    bi2
    *King, Rosen, and Tanner 1999; Rosen et al. 2001
    T’i | bi1 bi2, Xi ~ Binomial(Ni,θi)
    θi=Xi bi1 + (1-Xi)b2i
    Election
    Census
    

    View Slide

24. A HIERARCHICAL BAYESIAN
    APPROACH - THE MODEL*
    bi1
    T’i
    p
    bi2
    *King, Rosen, and Tanner 1999; Rosen et al. 2001
    T’i | bi1 bi2, Xi ~ Binomial(Ni,θi)
    θi=Xi bi1 + (1-Xi)b2i
    Election
    Census
    

    View Slide

25. A HIERARCHICAL BAYESIAN
    APPROACH - THE MODEL*
    c1 d1 c2 d2
    bi1
    T’i
    p
    bi2
    *King, Rosen, and Tanner 1999; Rosen et al. 2001
    T’i | bi1 bi2, Xi ~ Binomial(Ni,θi)
    bi1 | c1,d1 ~ Beta(c1,d1) i.i.d.
    bi2 | c2,d2 ~ Beta(c2,d2) i.i.d.
    θi=Xi bi1 + (1-Xi)b2i
    Election
    Census
    

    View Slide

26. A HIERARCHICAL BAYESIAN
    APPROACH - THE MODEL*
    λ
    c1 d1 c2 d2
    bi1
    T’i
    p
    bi2
    *King, Rosen, and Tanner 1999; Rosen et al. 2001
    T’i | bi1 bi2, Xi ~ Binomial(Ni,θi)
    bi1 | c1,d1 ~ Beta(c1,d1) i.i.d.
    bi2 | c2,d2 ~ Beta(c2,d2) i.i.d.
    c1~ Exponential(λ)
    d1~ Exponential(λ)
    c2~ Exponential(λ)
    d2~ Exponential(λ)
    θi=Xi bi1 + (1-Xi)b2i
    Election
    Census
    

    View Slide

27. WITH COVARIATES
    p(α) = p(β) = p(γ) = p(δ)∝1
    d1 ~ Exponential(λ)
    d2 ~ Exponential(λ)
    bi1 | Zi, d1,
    α,
    β ~ Beta(d1exp(α+βZi ), d1)
    bi2 | Zi, d2,
    γ,
    δ ~ Beta(d2exp(γ+δZi ), d2)
    θi=Xi b1i + (1-Xi) bi2
    T’i ~ Binomial(Ni,θi)
    
    Note: log (b1i)/( 1-(b1,i)) =α+βZi
    

    View Slide

28. WITH MORE CATEGORIES (RXC)
    rc ~ Exponential(λ1) i.i.d.
    bir | r ~ Dirichlet(r) i.i.d.
    r = 1,…,R
    θic=Xir bi1c + XiRbiRc
    c = 1… C
    T’ic ~ Multinomial(Ni,θi)
    
    bi1 biR
    T’i p
    1 R
    λ
    vectors of length C
    

    View Slide

29. WITH MORE CATEGORIES AND
    COVARIATES
    dr ~ Exponential (λ) i.i.d. r = 1….R
    r = (dr exp( r1 + r1 Zi),…,dr exp( rC-1 + rC-1 Zi), dr )
    r = 1,…,R
    bri | r ~ Dirichlet(r) i.i.d.
    r = 1,…,R
    θic=Xi1 bi1c + XiRbiRc
    c = 1,…,C
    T’ic ~ Multinomial(Ni,θi)
    
    Note: log (birc)/( 1-(birc)) = rc + rc Zi
    λ
    d1 dR
    T’i
    p
    Zi
    R
    R
    bi1 biR
    1
    1
    

    View Slide

30. A HIERARCHICAL BAYESIAN
    APPROACH - SIMULATION STUDY
    

    View Slide

31. A HIERARCHICAL BAYESIAN
    APPROACH - SIMULATION STUDY
    PETE
    2018
    FOR PRESIDENT
    MORE HEAD PATS
    FEWER LOUD NOISES
    

    View Slide

32. SIMULATING DATA
    n_precincts = 14
    # Hidden
    minority_pete = np.random.rand(n_precincts) * 0.3 + 0.7
    majority_pete = np.random.rand(n_precincts) * 0.3
    

    View Slide

33. SIMULATING DATA
    # Observed
    pct_minority = np.random.rand(n_precincts)
    pct_for_pete = (pct_minority * minority_pete +
    (1 - pct_minority) * majority_pete)
    

    View Slide

34. SIMULATING DATA
    # Observed
    pct_minority = np.random.rand(n_precincts)
    pct_for_pete = (pct_minority * minority_pete +
    (1 - pct_minority) * majority_pete)
    Census
    Election
    

    View Slide

35. SIMULATING DATA
    # Observed
    voting_population = np.random.randint(100, 10000,
    size=n_precincts)
    num_voting_for_pete = pct_for_pete * voting_population
    

    View Slide

36. SIMULATING DATA
    # Observed
    voting_population = np.random.randint(100, 10000,
    size=n_precincts)
    num_voting_for_pete = pct_for_pete * voting_population
    Census
    Election
    

    View Slide

37. MODELLING!
    

    View Slide

38. VISUALIZING RESULTS
    

    View Slide

39. VISUALIZING RESULTS
    

    View Slide

40. VISUALIZING RESULTS
    

    View Slide

41. VISUALIZING RESULTS
    

    View Slide

42. NORTH CAROLINA
    

    View Slide

43. NORTH CAROLINA
    

    View Slide

44. View Slide

45. NORTH CAROLINA
    • 6.8M out of 7.1M voting-age people
    • 3.1M out of 4.6M votes
    • 2,592 voting divisions
    

    View Slide

46. NORTH CAROLINA
    • 6.8M out of 7.1M voting-age people
    • 3.1M out of 4.6M votes
    • 2,592 voting divisions
    

    View Slide

47. NORTH CAROLINA
    • 6.8M out of 7.1M voting-age people
    • 3.1M out of 4.6M votes
    • 2,592 voting divisions
    

    View Slide

48. NORTH CAROLINA
    source: cnn.com
    

    View Slide

49. politico.com
    

    View Slide

50. politico.com
    

    View Slide

51. politico.com
    

    View Slide

52. View Slide

53. OVERALL VOTE FOR
    DEMOCRATIC CANDIDATES
    

    View Slide

54. ESTIMATED NON-WHITE VOTE
    FOR DEMOCRATIC CANDIDATES
    

    View Slide

55. ESTIMATED WHITE VOTE FOR
    DEMOCRATIC CANDIDATES
    

    View Slide

56. IT’S A GREAT TIME…
    TO CARE ABOUT GERRYMANDERING
    2010 2013 2020 2021
    Census Census Major
    redistricting
    Shelby vs. Holder Supreme Court
    on gerrymandering in:
    WI, NC, MD….
    Now-ish
    

    View Slide

57. THANK YOU!
    github.com/ColCarroll/redistricting-pymc3-pycon-2018
    

    View Slide

58. • King, Gary and Roberts, Molly.(2016). ei: Ecological Inference. R package version
    1.3-3.
    • King, Gary, Ori Rosen and Martin A. Tanner. Ecological Inference: New
    Methodological Strategies
    • King, Gary, Ori Rosen, and Martin A. Tanner. "Binomial-beta hierarchical models for
    ecological inference." Sociological Methods & Research 28.1 (1999): 61-90.
    • King, Gary (1997). A Solution to the Ecological Inference Problem. Princeton, NJ:
    Princeton University Press.
    • Rosen, Ori, et al. "Bayesian and frequentist inference for ecological inference: The R×
    C case." Statistica Neerlandica 55.2 (2001): 134-156.
    • James Greiner, “Ecological Inference in Voting Rights Act Disputes: Where Are We
    Now, and Where Do We Want to Be?”, 47 Jurimetrics J. 115-167 (2007).
    • Bullock, Charles S. et al. The Rise and Fall of the Voting Rights Act. Oklahoma
    University Press: Norman, 2006.
    • Thornburg v. Gingles, 478 U.S. 30,49 (1986).
    • Shelby County v. Holder, No. 570 U.S 12-96, (2013)
    • Metric Geometry and Gerrymandering Group https://sites.tufts.edu/gerrymandr/
    (particular acknowledgment to Dr. Mira Bernstein of the MGGG and Dr. Megan A.
    Gall of the Lawyers’ Committee for Civil Rights Under Law)
    • NC data from: OpenElections http://www.openelections.net/ and the US Census
    

    View Slide

59. KING’S ECOLOGICAL INFERENCE
    precinct 1: 70% Dem, 30% Rep, 65% Black, 35% White
    precinct 2: 50% Dem, 50% Rep, 25% Black, 75% White
    .70 = .65b11 + .35b12
    .50 = .25b11 + .75b12
    b11
    b12
    

    View Slide

60. METHOD OF BOUNDS
    precinct 1: 90% Dem, 10% Rep, 85% Black, 15% White
    Note: 0.9 = .85b11 + .15b12
    .33 < b12 ≤ 1
    PRECINCT DEMOCRAT REPUBLICAN
    Black bi11 = ?? bi12 = ?? X1,i
    White bi21 = ?? bi22 = ?? X2,i =1 - X1,i
    T1,i T2,i = 1 - T1,i
    

    View Slide

61. A HIERARCHICAL BAYESIAN
    APPROACH - THE MODEL*
    c1~ Exponential(λ)
    d1~ Exponential(λ)
    c2~ Exponential(λ)
    d2~ Exponential(λ)
    bi1 | c1,d1 ~ Beta(c1,d1) i.i.d.
    bi2 | c2,d2 ~ Beta(c2,d2) i.i.d.
    θi=Xi bi1 + (1-Xi)b2i
    T’i | bi1 bi2, Xi ~ Binomial(Ni,θi)
    
    λ
    c1 d1 c2 d2
    bi1
    T’i p
    bi2
    *King, Rosen, and Tanner
    

    View Slide