Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Unnecessary

Carlisle Rainey
January 25, 2018
460

 Unnecessary

Carlisle Rainey

January 25, 2018
Tweet

Transcript

  1. Unnecessary Bias
    The Consequences of Averaging Simulated
    Quantities of Interest
    Carlisle Rainey
    Assistant Professor
    Texas A&M University
    [email protected]
    Holger Kern
    Assistant Professor
    Florida State University
    [email protected]
    paper, data, and code at
    carlislerainey.com/talk

    View Slide

  2. Unnecessary Bias
    The Consequences of Averaging Simulated
    Quantities of Interest
    Carlisle Rainey
    Assistant Professor
    Texas A&M University
    [email protected]
    Holger Kern
    Assistant Professor
    Florida State University
    [email protected]
    paper, data, and code at
    carlislerainey.com/talk

    View Slide

  3. Should you average simulated
    quantities of interest?

    View Slide

  4. Should you average simulated
    quantities of interest?

    View Slide

  5. …the way King, Tomz, and Wittenberg (2000) say to do.
    Should you average simulated
    quantities of interest?

    View Slide

  6. …the way King, Tomz, and Wittenberg (2000) say to do.
    …the way Clarify and Zelig do.
    Should you average simulated
    quantities of interest?

    View Slide

  7. …the way King, Tomz, and Wittenberg (2000) say to do.
    …the way Clarify and Zelig do.
    …the way that Matt Golder’s marginal effects plot code does.
    Should you average simulated
    quantities of interest?

    View Slide

  8. …the way King, Tomz, and Wittenberg (2000) say to do.
    …the way Clarify and Zelig do.
    …the way that Matt Golder’s marginal effects plot code does.
    …the way that Hanmer and Kalkan’s (2013) code does.
    Should you average simulated
    quantities of interest?

    View Slide

  9. …the way King, Tomz, and Wittenberg (2000) say to do.
    …the way Clarify and Zelig do.
    …the way that Matt Golder’s marginal effects plot code does.
    …the way that Hanmer and Kalkan’s (2013) code does.
    …the way that my own code does.
    Should you average simulated
    quantities of interest?

    View Slide

  10. …the way King, Tomz, and Wittenberg (2000) say to do.
    …the way Clarify and Zelig do.
    …the way that Matt Golder’s marginal effects plot code does.
    …the way that Hanmer and Kalkan’s (2013) code does.
    …the way that my own code does.
    …the way that Holger’s code does.
    Should you average simulated
    quantities of interest?

    View Slide

  11. …the way King, Tomz, and Wittenberg (2000) say to do.
    …the way Clarify and Zelig do.
    …the way that Matt Golder’s marginal effects plot code does.
    …the way that Hanmer and Kalkan’s (2013) code does.
    …the way that my own code does.
    …the way that Holger’s code does.
    …the way that your own code might.
    Should you average simulated
    quantities of interest?

    View Slide

  12. No.

    View Slide

  13. View Slide

  14. TABLE 2. Models of Involvement in Militarized Disputes, 1950–1985:
    Assessing the Liberal Peace
    Variable Eqn 1 Eqn 2 Eqn 3 Eqn 4
    Democracy scoreL
    ß -0.0497 -0.0554 -0.0413 -0.0457
    SEß
    0.0074 0.0077 0.0083 0.0076
    p < .001 < .001 < .001 < .001
    Economic growth rateL
    -0.0223 -0.0318 -0.0297 -0.0220
    0.0085 0.0093 0.0094 0.0090
    .009 < .001 < .001 .02
    Allies -0.821 -0.868 -0.864 -0.815
    0.080 0.092 0.092 0.083
    < .001 < .001 < .001 < .001
    Contiguity 1.31 1.26 1.32 1.39
    0.08 0.09 0.09 0.08
    < .001 < .001 < .001 < .001
    Capability ratio -0.00307 -0.00345 -0.00356 -0.00279
    0.00042 0.00050 0.00051 0.00041
    < .001 < .001 < .001 < .001
    Dyadic trade-to-GDP ratioL
    -66.1 -43.8 -81.0
    13.4 12.5 15.2
    < .001 < .001 < .001
    Total trade-to-GDP ratioL
    -0.706 -0.512
    0.192 0.193
    < .001 .008
    Trend, dyadic trade-to-GDP -8.89
    ratioH
    2.93
    < .001
    Constant -3.29 -3.28 -3.20 -3.32
    0.08 0.10 0.10 0.08
    < .001 < .001 < .001 < .001
    Chi2 764.04 593.65 611.74 728.56
    P of chi2 < .0001 < .0001 < .0001 < .0001
    Log likelihood -3477.57 -2795.89 -2786.85 -3231.42
    N 20,990 17,709 17,709 19,772

    View Slide

  15. 0.08
    < .001
    Capability ratio -0.00307
    0.00042
    < .001
    Dyadic trade-to-GDP ratioL
    -66.1
    13.4
    < .001
    Total trade-to-GDP ratioL
    Trend, dyadic trade-to-GDP

    View Slide

  16. TABLE 3. Annual Probabilities of Involvement in a Militarized Dispute, 1950–1985,
    Based on the Estimated Coefficients for Equations 1 and 2 in Table 2
    Equation 1 Equation 2
    1. Democracy scoreL
    , economic growth rateL
    , dyadic .078 .070
    trade-to-GDP ratioL
    , and capability ratio at mean
    values; allies equals 0; contiguity equals 1
    2. Increase in democracyL
    of 1 std. dev.; other variables .057 .049
    at baseline values
    3. Increase in dyadic trade-to-GDP ratioL
    of 1 std. dev.; .047 —
    other variables at baseline values
    4. Increase in total trade-to-GDP ratioL
    of 1 std. dev.; — .061
    other variables at baseline values
    5. Increase in economic growth rateL
    of 1 std. dev.; other .073 .064
    variables at baseline values
    6. Allies equals 1; other variables at baseline values .036 .031
    7. Contiguity equals 0; other variables at baseline values .022 .021
    8. Increase in capability ratio of 1 std. dev.; other variables .021 .016
    at baseline values
    9. Capability ratio equals 1; other variables at baseline values .122 .117
    10. Capability ratio equals 10; other variables at baseline values .119 .114

    View Slide

  17. TABLE 3. Annual Probabilities of Involvement in a Militarized Dispute, 1950–1985,
    Based on the Estimated Coefficients for Equations 1 and 2 in Table 2
    Equation 1 Equation 2
    1. Democracy scoreL
    , economic growth rateL
    , dyadic .078 .070
    trade-to-GDP ratioL
    , and capability ratio at mean
    values; allies equals 0; contiguity equals 1
    2. Increase in democracyL
    of 1 std. dev.; other variables .057 .049
    at baseline values
    3. Increase in dyadic trade-to-GDP ratioL
    of 1 std. dev.; .047 —
    other variables at baseline values
    4. Increase in total trade-to-GDP ratioL
    of 1 std. dev.; — .061
    other variables at baseline values
    5. Increase in economic growth rateL
    of 1 std. dev.; other .073 .064
    variables at baseline values
    6. Allies equals 1; other variables at baseline values .036 .031
    7. Contiguity equals 0; other variables at baseline values .022 .021
    8. Increase in capability ratio of 1 std. dev.; other variables .021 .016
    at baseline values
    9. Capability ratio equals 1; other variables at baseline values .122 .117
    10. Capability ratio equals 10; other variables at baseline values .119 .114

    View Slide

  18. TABLE 3. Annual Probabilities of Involvement in a Militarized Dispute, 1950–1985,
    Based on the Estimated Coefficients for Equations 1 and 2 in Table 2
    Equation 1 Equation 2
    1. Democracy scoreL
    , economic growth rateL
    , dyadic .078 .070
    trade-to-GDP ratioL
    , and capability ratio at mean
    values; allies equals 0; contiguity equals 1
    2. Increase in democracyL
    of 1 std. dev.; other variables .057 .049
    at baseline values
    3. Increase in dyadic trade-to-GDP ratioL
    of 1 std. dev.; .047 —
    other variables at baseline values
    4. Increase in total trade-to-GDP ratioL
    of 1 std. dev.; — .061
    other variables at baseline values
    5. Increase in economic growth rateL
    of 1 std. dev.; other .073 .064
    variables at baseline values
    6. Allies equals 1; other variables at baseline values .036 .031
    7. Contiguity equals 0; other variables at baseline values .022 .021
    8. Increase in capability ratio of 1 std. dev.; other variables .021 .016
    at baseline values
    9. Capability ratio equals 1; other variables at baseline values .122 .117
    10. Capability ratio equals 10; other variables at baseline values .119 .114

    View Slide

  19. TABLE 3. Annual Probabilities of Involvement in a Militarized Dispute, 1950–1985,
    Based on the Estimated Coefficients for Equations 1 and 2 in Table 2
    Equation 1 Equation 2
    1. Democracy scoreL
    , economic growth rateL
    , dyadic .078 .070
    trade-to-GDP ratioL
    , and capability ratio at mean
    values; allies equals 0; contiguity equals 1
    2. Increase in democracyL
    of 1 std. dev.; other variables .057 .049
    at baseline values
    3. Increase in dyadic trade-to-GDP ratioL
    of 1 std. dev.; .047 —
    other variables at baseline values
    4. Increase in total trade-to-GDP ratioL
    of 1 std. dev.; — .061
    other variables at baseline values
    5. Increase in economic growth rateL
    of 1 std. dev.; other .073 .064
    variables at baseline values
    6. Allies equals 1; other variables at baseline values .036 .031
    7. Contiguity equals 0; other variables at baseline values .022 .021
    8. Increase in capability ratio of 1 std. dev.; other variables .021 .016
    at baseline values
    9. Capability ratio equals 1; other variables at baseline values .122 .117
    10. Capability ratio equals 10; other variables at baseline values .119 .114

    View Slide

  20. TABLE 3. Annual Probabilities of Involvement in a Militarized Dispute, 1950–1985,
    Based on the Estimated Coefficients for Equations 1 and 2 in Table 2
    Equation 1 Equation 2
    1. Democracy scoreL
    , economic growth rateL
    , dyadic .078 .070
    trade-to-GDP ratioL
    , and capability ratio at mean
    values; allies equals 0; contiguity equals 1
    2. Increase in democracyL
    of 1 std. dev.; other variables .057 .049
    at baseline values
    3. Increase in dyadic trade-to-GDP ratioL
    of 1 std. dev.; .047 —
    other variables at baseline values
    4. Increase in total trade-to-GDP ratioL
    of 1 std. dev.; — .061
    other variables at baseline values
    5. Increase in economic growth rateL
    of 1 std. dev.; other .073 .064
    variables at baseline values
    6. Allies equals 1; other variables at baseline values .036 .031
    7. Contiguity equals 0; other variables at baseline values .022 .021
    8. Increase in capability ratio of 1 std. dev.; other variables .021 .016
    at baseline values
    9. Capability ratio equals 1; other variables at baseline values .122 .117
    10. Capability ratio equals 10; other variables at baseline values .119 .114
    difference of -0.031

    View Slide

  21. two options

    View Slide

  22. two options
    use the invariance property

    View Slide

  23. two options
    use the invariance property
    Long (1997)

    View Slide

  24. two options
    use the invariance property
    average simulated QIs

    View Slide

  25. two options
    use the invariance property
    average simulated QIs
    Clarify in Stata

    View Slide

  26. two options
    use the invariance property
    average simulated QIs

    View Slide

  27. two options
    use the invariance property
    average simulated QIs
    t.i. bias

    View Slide

  28. two options
    use the invariance property
    average simulated QIs
    t.i. bias
    2x t.i. bias

    View Slide

  29. two options
    use the invariance property
    average simulated QIs
    t.i. bias
    2x t.i. bias
    2017 PA
    (1)

    View Slide

  30. two options
    use the invariance property
    average simulated QIs
    t.i. bias
    2x t.i. bias
    2017 PA
    What happens when you
    average simulated QIs?
    (2)
    (1)

    View Slide

  31. two options
    use the invariance property
    average simulated QIs
    t.i. bias
    2x t.i. bias
    2017 PA
    What happens when you
    average simulated QIs?
    Does any of this matter?
    (2)
    (3)
    (1)

    View Slide

  32. Rainey (2017)

    View Slide

  33. Long (1997)

    View Slide

  34. King (1998)

    View Slide

  35. margins

    View Slide

  36. predict()

    View Slide

  37. bias

    View Slide

  38. bias
    expected estimate - true value

    View Slide

  39. bias
    expected estimate - true value
    under repeated sampling

    View Slide

  40. bias
    expected estimate - true value
    under repeated sampling
    *IMPORTANT*
    property of procedure,
    not estimate

    View Slide

  41. View Slide

  42. View Slide

  43. View Slide

  44. e.g., small-sample bias in logit

    View Slide

  45. Jensen’s inequality: E[g(X)] > g(E[X]) for convex g.
    e.g., small-sample bias in logit

    View Slide

  46. A Stark Illustration

    View Slide

  47. yi
    ⇠ N(µ, 1), for i = 1, 2, ..., 100
    ⌧(µ) = µ2
    µ = 0

    View Slide

  48. yi
    ⇠ N(µ, 1), for i = 1, 2, ..., 100
    ⌧(µ) = µ2
    µ = 0
    DGP or model

    View Slide

  49. yi
    ⇠ N(µ, 1), for i = 1, 2, ..., 100
    ⌧(µ) = µ2
    µ = 0
    DGP or model
    QI

    View Slide

  50. yi
    ⇠ N(µ, 1), for i = 1, 2, ..., 100
    ⌧(µ) = µ2
    µ = 0
    DGP or model
    QI
    truth

    View Slide

  51. sample ave. is a BUE of u.
    (…and MLE!)
    yi
    ⇠ N(µ, 1), for i = 1, 2, ..., 100
    ⌧(µ) = µ2
    µ = 0
    DGP or model
    QI
    truth

    View Slide

  52. 0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 1

    View Slide


  53. τ
    ^mle
    µ
    ^mle
    0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 1

    View Slide



  54. 0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 2

    View Slide


  55. ● ●
    0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 3

    View Slide


  56. ● ●

    0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 4

    View Slide


  57. ● ●


    0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 5

    View Slide


  58. ● ●



    0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 6

    View Slide


  59. ● ●




    0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 7

    View Slide


  60. ● ●





    0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 8

    View Slide


  61. ● ●






    0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 9

    View Slide


  62. ● ●







    0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 10

    View Slide

  63. 0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 10000

    View Slide

  64. µ
    ^
    τ
    ^
    −0.4 −0.2 0.0 0.2 0.4 0.00 0.05 0.10 0.15
    0
    30
    60
    90
    0
    1
    2
    3
    4
    Estimate
    Density
    Method Maximum Likelihood Average of Simulations

    View Slide


  65. µ
    ^
    τ
    ^
    −0.4 −0.2 0.0 0.2 0.4 0.00 0.05 0.10 0.15
    0
    30
    60
    90
    0
    1
    2
    3
    4
    Estimate
    Density
    Method Maximum Likelihood Average of Simulations

    View Slide

  66. ● ●
    µ
    ^
    τ
    ^
    −0.4 −0.2 0.0 0.2 0.4 0.00 0.05 0.10 0.15
    0
    30
    60
    90
    0
    1
    2
    3
    4
    Estimate
    Density
    Method Maximum Likelihood Average of Simulations

    View Slide

  67. One Theoretical Result

    View Slide

  68. One Theoretical Result
    Convex transformations lead to positive
    transformation-induced bias.

    View Slide

  69. two options
    use the invariance property
    average simulated QIs
    t.i. bias
    2x t.i. bias
    2017 PA
    What happens when you
    average simulated QIs?
    Does any of this matter?
    (2)
    (3)
    (1)

    View Slide

  70. Simulating Quantities of Interest
    King, Tomz, Wittenberg (2000)

    View Slide

  71. Simulating Quantities of Interest
    King, Tomz, Wittenberg (2000)
    1. Fit the model.

    View Slide

  72. Simulating Quantities of Interest
    King, Tomz, Wittenberg (2000)
    1. Fit the model.
    2. Simulate the coefficients.

    View Slide

  73. Simulating Quantities of Interest
    King, Tomz, Wittenberg (2000)
    1. Fit the model.
    2. Simulate the coefficients.
    3. Transform simulated coefficients into the quantities of
    interest.

    View Slide

  74. Simulating Quantities of Interest
    King, Tomz, Wittenberg (2000)
    1. Fit the model.
    2. Simulate the coefficients.
    3. Transform simulated coefficients into the quantities of
    interest.
    4. Average the simulations of the quantity of interest to
    obtain a point estimate.

    View Slide

  75. King, Tomz, Wittenberg (2000)

    View Slide

  76. Gelman and Hill (2006)

    View Slide

  77. mattgolder.com/interactions

    View Slide

  78. A Stark Illustration
    continued…

    View Slide

  79. sample ave. is a BUE of u.
    (…and MLE!)
    yi
    ⇠ N(µ, 1), for i = 1, 2, ..., 100
    ⌧(µ) = µ2
    µ = 0
    DGP or model
    QI
    truth

    View Slide

  80. sample ave. is a BUE of u.
    (…and MLE!)
    yi
    ⇠ N(µ, 1), for i = 1, 2, ..., 100
    ⌧(µ) = µ2
    µ = 0
    DGP or model
    QI
    truth
    SE is 1/sqrt(100)

    View Slide

  81. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1

    View Slide

  82. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    normal w/ mean of 0.2
    and SD of 0.1

    View Slide

  83. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1

    View Slide

  84. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    sim. coef.

    View Slide

  85. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    sim. coef.
    translated into sim. QI

    View Slide

  86. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    sim. coef.
    translated into sim. QI

    View Slide

  87. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1

    View Slide

  88. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2

    View Slide

  89. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2

    View Slide

  90. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2

    View Slide

  91. 0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2

    View Slide

  92. τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2

    View Slide

  93. τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 3

    View Slide

  94. τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 3
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 4

    View Slide

  95. τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 3
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 4
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 5

    View Slide

  96. τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 3
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 4
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 5
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 6

    View Slide

  97. τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 3
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 4
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 5
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 6
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 7

    View Slide

  98. τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 3
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 4
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 5
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 6
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 7
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 8

    View Slide

  99. τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 3
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 4
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 5
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 6
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 7
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 8
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 9

    View Slide

  100. ● ●
    µ
    ^
    τ
    ^
    −0.4 −0.2 0.0 0.2 0.4 0.00 0.05 0.10 0.15
    0
    30
    60
    90
    0
    1
    2
    3
    4
    Estimate
    Density
    Method Maximum Likelihood Average of Simulations

    View Slide

  101. ● ●
    µ
    ^
    τ
    ^
    −0.4 −0.2 0.0 0.2 0.4 0.00 0.05 0.10 0.15
    0
    30
    60
    90
    0
    1
    2
    3
    4
    Estimate
    Density
    Method Maximum Likelihood Average of Simulations

    View Slide

  102. ● ●

    µ
    ^
    τ
    ^
    −0.4 −0.2 0.0 0.2 0.4 0.00 0.05 0.10 0.15
    0
    30
    60
    90
    0
    1
    2
    3
    4
    Estimate
    Density
    Method Maximum Likelihood Average of Simulations

    View Slide

  103. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1

    View Slide

  104. 1
    2
    3

    View Slide

  105. For a convex transformation, the average
    of simulations is always greater than the
    (upwardly biased) ML estimate.
    1
    2
    3

    View Slide

  106. For a convex transformation, the average
    of simulations is always greater than the
    (upwardly biased) ML estimate.
    So, of course the average of simulations is
    more biased than the ML estimate.
    1
    2
    3

    View Slide

  107. For a convex transformation, the average
    of simulations is always greater than the
    (upwardly biased) ML estimate.
    So, of course the average of simulations is
    more biased than the ML estimate.
    This additional bias is roughly equal to the
    transformation-induced bias.
    1
    2
    3

    View Slide

  108. two options
    use the invariance property
    average simulated QIs
    t.i. bias
    2x t.i. bias
    2017 PA
    What happens when you
    average simulated QIs?
    Does any of this matter?
    (2)
    (3)
    (1)

    View Slide

  109. additional bias from simulation
    is similar to
    small-sample bias in logit

    View Slide

  110. The additional bias grows larger…

    View Slide

  111. The additional bias grows larger…
    as the standard errors increase and

    View Slide

  112. The additional bias grows larger…
    as the standard errors increase and
    as the transformations become more nonlinear.

    View Slide

  113. Three Examples

    View Slide

  114. study George and Epstein (1992)
    key claim Supreme Ct. decisions depend on legal and
    extralegal factors.
    data 64 observations, 11 explanatory variables
    quantities of
    interest
    first difference: effect of a Solicitor General brief
    on the probability of a conservative decision for all
    64 combinations of the other covariates

    View Slide

































































  115. 0.0
    0.1
    0.2
    0.3
    0.4
    0.0 0.2 0.4 0.6
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Effect of a Solicitor General Brief

    View Slide

































































  116. 0.0
    0.1
    0.2
    0.3
    0.4
    0.0 0.2 0.4 0.6
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Effect of a Solicitor General Brief
    sim. avg. about 20%
    smaller than MLE

    View Slide

































































  117. 0.0
    0.1
    0.2
    0.3
    0.4
    0.0 0.2 0.4 0.6
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Effect of a Solicitor General Brief
    sim. avg. about 20%
    smaller than MLE
    sim. avg. about
    2x the MLE

    View Slide

































































  118. 0.0
    0.1
    0.2
    0.3
    0.4
    0.0 0.2 0.4 0.6
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Effect of a Solicitor General Brief
    sim. avg. about 20%
    smaller than MLE
    sim. avg. about
    2x the MLE
    7 cases where sim. avg. >10x MLE

    View Slide

  119. study Fearon and Laitin (2003)
    key claim Lots of factors lead to civil wars,
    but ethnic and religious diversity do not.
    data 6,327 observations (106 events),
    11 explanatory variables
    quantities of
    interest
    risk ratio: the effect of a 10% decrease in GDP per
    capita on the probability of civil war for all 6,327
    combinations of explanatory variables

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  120. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  121. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  122. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72
    year MLE s.a. ratio
    1968 101 743 7

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  123. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72
    year MLE s.a. ratio
    1968 101 743 7
    1969 99 716 7

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  124. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72
    year MLE s.a. ratio
    1968 101 743 7
    1969 99 716 7
    1970 65 311 5

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  125. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72
    year MLE s.a. ratio
    1968 101 743 7
    1969 99 716 7
    1970 65 311 5
    1971 48 177 4

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  126. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72
    year MLE s.a. ratio
    1968 101 743 7
    1969 99 716 7
    1970 65 311 5
    1971 48 177 4
    1972 50 189 4

    View Slide

  127. study Holland (2017)
    key claim Politicians enforce laws less diligently when it
    helps them keep office
    data 19, 36, and 34 observations
    3 explanatory variables
    quantities of
    interest
    expected count: the expected number of
    enforcement operations for each observation.

    View Slide




















  128. ● ●




































    ●●










    ● ●











    ● ●





    Bogota Lima Santiago
    10 20 30 10 20 30 40 50 2 4 6 8
    5
    10
    20
    40
    60
    10
    20
    30
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Expected Number of Enforcement Operations

    View Slide




















  129. ● ●




































    ●●










    ● ●











    ● ●





    Bogota Lima Santiago
    10 20 30 10 20 30 40 50 2 4 6 8
    5
    10
    20
    40
    60
    10
    20
    30
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Expected Number of Enforcement Operations
    MLE is 24
    sim. avg. is 46
    sim. avg. is 90% bigger

    View Slide




















  130. ● ●




































    ●●










    ● ●











    ● ●





    Bogota Lima Santiago
    10 20 30 10 20 30 40 50 2 4 6 8
    5
    10
    20
    40
    60
    10
    20
    30
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Expected Number of Enforcement Operations
    MLE is 24
    sim. avg. is 46
    sim. avg. is 90% bigger
    MLE is 4.6
    sim. avg. is 5.8
    sim. avg. is 25% bigger

    View Slide

  131. two options
    use the invariance property
    average simulated QIs
    t.i. bias
    2x t.i. bias
    2017 PA
    What happens when you
    average simulated QIs?
    Does any of this matter?
    (2)
    (3)
    (1)

    View Slide

  132. Questions?

    View Slide

  133. Appendix
    Stark Illustration
    • Illustration of Transformation-
    Induced Bias.

    Iteration 1

    Iteration 10

    Iteration 10,000

    • Illustration of Additional Bias

    Iteration 1

    Iteration 2

    Iterations 1-9

    Sampling Distributions
    Examples
    George and Epstein (1992)

    Fearon and Laitin (2003)

    Holland (2017)

    Logit Quantities of Interest

    Technical Details
    Theorem 1 from Rainey 2017

    Lemma 1

    Theorem 1

    Law of Iterated Expectations

    Blank

    View Slide


  134. τ
    ^mle
    µ
    ^mle
    0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 1
    toc

    View Slide


  135. ● ●







    0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 10
    toc

    View Slide

  136. 0.00
    0.05
    0.10
    0.15
    −0.4 −0.2 0.0 0.2 0.4
    µ
    ^
    τ
    ^
    Iteration 10000
    toc

    View Slide

  137. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    toc

    View Slide

  138. τ
    ^mle
    µ
    ^mle
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2
    toc

    View Slide

  139. τ
    ^avg.
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 1
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 2
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 3
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 4
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 5
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 6
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 7
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 8
    0.0
    0.1
    0.2
    0.3
    0.4
    −0.3 0.0 0.3 0.6
    µ
    τ
    Iteration 9
    toc

    View Slide

  140. ● ●

    µ
    ^
    τ
    ^
    −0.4 −0.2 0.0 0.2 0.4 0.00 0.05 0.10 0.15
    0
    30
    60
    90
    0
    1
    2
    3
    4
    Estimate
    Density
    Method Maximum Likelihood Average of Simulations
    toc

    View Slide

































































  141. 0.0
    0.1
    0.2
    0.3
    0.4
    0.0 0.2 0.4 0.6
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Effect of a Solicitor General Brief
    toc

    View Slide

































































  142. 0.0
    0.1
    0.2
    0.3
    0.4
    0.0 0.2 0.4 0.6
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Effect of a Solicitor General Brief
    sim. avg. about 20%
    smaller than MLE
    toc

    View Slide

































































  143. 0.0
    0.1
    0.2
    0.3
    0.4
    0.0 0.2 0.4 0.6
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Effect of a Solicitor General Brief
    sim. avg. about 20%
    smaller than MLE
    sim. avg. about
    2x the MLE
    toc

    View Slide

































































  144. 0.0
    0.1
    0.2
    0.3
    0.4
    0.0 0.2 0.4 0.6
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Effect of a Solicitor General Brief
    sim. avg. about 20%
    smaller than MLE
    sim. avg. about
    2x the MLE
    7 cases where sim. avg. >10x MLE
    toc

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  145. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    toc

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  146. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72
    toc

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  147. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72
    year MLE s.a. ratio
    1968 101 743 7
    toc

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  148. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72
    year MLE s.a. ratio
    1968 101 743 7
    1969 99 716 7
    toc

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  149. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72
    year MLE s.a. ratio
    1968 101 743 7
    1969 99 716 7
    1970 65 311 5
    toc

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  150. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72
    year MLE s.a. ratio
    1968 101 743 7
    1969 99 716 7
    1970 65 311 5
    1971 48 177 4
    toc

    View Slide






























































































































































































































































































































































































































































































































































































































































































































































































  151. ●●



























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●






























































































































































































































    ●●











































































































































































































































































































































































































    ● ●
















    ●●








































    ●●●











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































    ●●














































































    ●●




















































































    100
    101
    102
    103
    104
    105
    100 100.5 101 101.5 102 102.5 103
    Maximum Likelihood Estimate
    Simulation Average Estimate
    100 100.5 101 101.5 102
    Ratio: Avg./MLE
    The Effect of a 10% Decrease in GDP on the Probability of Civil War
    Kuwait from 1968-72
    year MLE s.a. ratio
    1968 101 743 7
    1969 99 716 7
    1970 65 311 5
    1971 48 177 4
    1972 50 189 4
    toc

    View Slide




















  152. ● ●




































    ●●










    ● ●











    ● ●





    Bogota Lima Santiago
    10 20 30 10 20 30 40 50 2 4 6 8
    5
    10
    20
    40
    60
    10
    20
    30
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Expected Number of Enforcement Operations
    toc

    View Slide




















  153. ● ●




































    ●●










    ● ●











    ● ●





    Bogota Lima Santiago
    10 20 30 10 20 30 40 50 2 4 6 8
    5
    10
    20
    40
    60
    10
    20
    30
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Expected Number of Enforcement Operations
    MLE is 24
    sim. avg. is 46
    sim. avg. is 90% bigger
    toc

    View Slide




















  154. ● ●




































    ●●










    ● ●











    ● ●





    Bogota Lima Santiago
    10 20 30 10 20 30 40 50 2 4 6 8
    5
    10
    20
    40
    60
    10
    20
    30
    Maximum Likelihood Estimate
    Simulation Average Estimate
    Expected Number of Enforcement Operations
    MLE is 24
    sim. avg. is 46
    sim. avg. is 90% bigger
    MLE is 4.6
    sim. avg. is 5.8
    sim. avg. is 25% bigger
    toc

    View Slide

  155. yi
    ⇠ Bernoulli(⇡i), for i = 1, 2, ..., 100
    logit(⇡i) = 1.5 + xi + 0.2wi + 0.5zi
    toc

    View Slide

  156. probability marginal effect
    first difference risk ratio
    coefficient−induced bias transformation−induced bias additional bias coefficient−induced bias transformation−induced bias additional bias
    coefficient−induced bias transformation−induced bias additional bias coefficient−induced bias transformation−induced bias additional bias
    0.1 0.2 0.3 0.4 0.1 0.2 0.3 0.4 0.1 0.2 0.3 0.4 0.05 0.10 0.15 0.20 0.25
    0.05 0.10 0.15 0.20 0.25
    0.05 0.10 0.15 0.20 0.25
    0.050 0.075 0.100 0.125 0.050 0.075 0.100 0.125 0.050 0.075 0.100 0.125 1.3 1.4 1.5 1.6 1.3 1.4 1.5 1.6 1.3 1.4 1.5 1.6
    True Quantity of Interest True Quantity of Interest
    True Quantity of Interest True Quantity of Interest
    Bias
    Bias
    Bias
    Bias
    −0.02
    −0.01
    0.00
    0.01
    0.02
    −0.04
    −0.02
    0.00
    −0.02
    −0.01
    0.00
    −0.04
    0.00
    0.04





























    ● ●●






    ● ●



    ● ●










    ● ●





    ● ●


    ● ●





    ● ●


    ● ●







    ● ●





    ● ●




































    ● ●

    ● ●



















    ● ●





    ● ●
    ● ●
    ● ●





    ● ●



    ● ●



    ● ●



    ● ●





    ● ●















    ● ●












    ● ●
























































    ● ●









    ● ● ●







    ● ●























    ● ●
























    ● ●




    ● ●









    ● ●
















    ● ●

















    ● ●

















    ● ● ●






    ● ●
























    ● ●











    ● ●




    ● ●

    ● ●































    ● ●









    ● ●






    ● ●




























    ● ●



    ● ●











    ● ●



    ● ●
    ● ●










    ● ●








    ● ●













    ● ●
























    ● ●




    ● ●









    ● ●
















    ● ●

















    ● ●











    ● ●




    ● ● ●






























    ● ●









    ● ●








    ● ●

    ● ●











    ● ●





























    ● ● ●








    ● ●
























    ● ●




    ● ●
    ● ●






    ● ●






    ● ●
































    ● ●
    ● ●




























    ● ●































    ● ●
    ● ●
    ● ●






















    ● ●






















    ● ●







    ● ●


















    ● ●








    ● ●











    ● ●









































































    ● ●























    ● ●




    toc

    View Slide

  157. Theorem 1 (t.i. bias, Rainey 2017) Suppose a non-degenerate estimator ˆ.
    Then any strictly convex (concave) ⌧ creates upward (downward) transformation-
    induced ⌧-bias.
    Proof The proof follows directly from Jensen’s inequality. Suppose that the
    non-degenerate sampling distribution of ˆ is given by S (b) so that ˆ ⇠ S (b).
    Then E(ˆ) =
    R
    B
    bS (b)db and E[⌧(ˆ)] =
    R
    B
    ⌧(b)S (b)db. Suppose first that ⌧ is
    convex. By Jensen’s inequality,
    R
    B
    ⌧(b)S (b)db > ⌧
    ⇥R
    B
    bS (b)db

    , which implies
    that E[⌧(ˆ)] > ⌧[E(ˆ)]. Because E[⌧(ˆ)] ⌧[E(ˆ)] > 0, the transformation-
    induced ⌧-bias is upward. By similar argument, one can show that for any
    strictly concave ⌧, E[⌧(ˆ)] ⌧[E(ˆ)] > 0 and that the transformation-induced
    ⌧-bias is downward. ⌅
    toc

    View Slide

  158. toc
    Lemma 1 Suppose a maximum likelihood estimator ˆmle. Then any strictly
    convex (concave) ⌧ guarantees that ˆ
    ⌧avg. is strictly greater [less] than ˆ
    ⌧mle.
    Proof By definition,
    ˆ
    ⌧avg. = E
    h


    ˜
    ⌘i
    .
    Using Jensen’s inequality, we know that E
    h


    ˜
    ⌘i
    > ⌧
    h
    E

    ˜
    ⌘i
    , so that
    ˆ
    ⌧avg. > ⌧
    h
    E

    ˜
    ⌘i
    .
    However, because ˜ ⇠ N
    h
    ˆmle, ˆ
    V

    ˆmle
    ⌘i
    , E

    ˜

    = ˆmle, so that
    ˆ
    ⌧avg. > ⌧

    ˆmle

    .
    Of course, ˆ
    ⌧mle = ⌧

    ˆmle

    by definition, so that
    ˆ
    ⌧avg. > ˆ
    ⌧mle.
    The proof for concave ⌧ follows similarly. ⌅

    View Slide

  159. Theorem 1 Suppose a maximum likelihood estimator ˆmle. Then for any strictly
    convex or concave ⌧, the transformation-induced ⌧-bias for ˆ
    ⌧avg. is strictly
    greater in magnitude than the transformation-induced ⌧-bias for ˆ
    ⌧mle.
    Proof According to Theorem 1 of Rainey (2017), E ˆ
    ⌧mle ⌧
    h
    E

    ˆmle
    ⌘i
    >
    0. Lemma 1 shows that for any convex ⌧, ˆ
    ⌧avg. > ˆ
    ⌧mle. It follows that
    E (ˆ
    ⌧avg.) ⌧
    h
    E

    ˆmle
    ⌘i
    | {z }
    t.i. ⌧-bias in ˆ
    ⌧avg.
    > E ˆ
    ⌧mle ⌧
    h
    E

    ˆmle
    ⌘i
    | {z }
    t.i. ⌧-bias in ˆ
    ⌧mle
    > 0. For the concave case,
    it follows similarly that E (ˆ
    ⌧avg.) ⌧
    h
    E

    ˆmle
    ⌘i
    | {z }
    t.i. ⌧-bias in ˆ
    ⌧avg.
    < E ˆ
    ⌧mle ⌧
    h
    E

    ˆmle
    ⌘i
    | {z }
    t.i. ⌧-bias in ˆ
    ⌧mle
    < 0

    toc

    View Slide

  160. toc

    View Slide

  161. toc

    View Slide