Slide 1

Slide 1 text

with Python
 Synthetic Difference in Differences
 twitter @asas_mimi 1

Slide 2

Slide 2 text

Table of Contents 1. Difference in Differences 2. Synthetic Control Method 3. Synthetic Difference in Differences 4. Python pkg : pysynthdid 2

Slide 3

Slide 3 text

Original Paper: Arkhangelsky, Dmitry, et al. Synthetic difference in differences. No. w25532. National Bureau of Economic Research, 2019. https://www.nber.org/system/files/working_papers/w25532/w25532.pdf 3

Slide 4

Slide 4 text

Difference in Differences (DID)
 4 pre post outcome If there had been no intervention, the outcome of the treatment group would probably have been here counterfactual treatment group control group pros Excellent interpretability cons Parallel trend assumption ● DID requires the selection of a control group that has parallel trends in the treatment group and the pre-intervention period. ● This selection may be arbitrary in some cases. e.t.c.

Slide 5

Slide 5 text

setup
 binary treatment observable outcomes T pre
 T post
 N tr
 N co
 5

Slide 6

Slide 6 text

DID as a two-way fixed effect regression
 Arkhangelsky, Dmitry, et al. Synthetic difference in differences. No. w25532. National Bureau of Economic Research, 2019. unit fixed effect time fixed effect 6

Slide 7

Slide 7 text

Synthetic Control Method (SC)
 ωSC vertical regression Estimating the ω that regresses the treatment group from the control group in the pre-term ωSC Synthetic Control counterfactual observable outcome ADH restrictions (Abadie, Diamond and Hainmueller (2010)) SUM(ω) = 1 , non-negativity, no intercept Arkhangelsky, Dmitry, et al. Synthetic difference in differences. No. w25532. National Bureau of Economic Research, 2019. 7

Slide 8

Slide 8 text

Synthetic Control Method (SC)
 SC regression formula : regression with time fixed effects and sc weights omit New!! Arkhangelsky, Dmitry, et al. Synthetic difference in differences. No. w25532. National Bureau of Economic Research, 2019. 8

Slide 9

Slide 9 text

Synthetic Difference in Differences (SDID)
 ωSDID vertical regression + horizontal regression - Estimating the ω that regresses the treatment group from the control group in the pre-term - The basic idea is the same as ADH SC. However, an intercept term and an L2 penalty term are introduced. λSDID non-negativity sum(λ)=1 with intercept L2 penalty Arkhangelsky, Dmitry, et al. Synthetic difference in differences. No. w25532. National Bureau of Economic Research, 2019. 9

Slide 10

Slide 10 text

Synthetic Difference in Differences (SDID)
 The method of estimating ω is slightly different from the classical one(ADH). non-negativity sum(λ)=1 with intercept L2 penalty Arkhangelsky, Dmitry, et al. Synthetic difference in differences. No. w25532. National Bureau of Economic Research, 2019. intercept “unexposed pre-trends perfectly match the exposed ones; rather, it is sufficient that the weights make the trends paralle” L2 penalty “to increase the dispersion, and ensure the uniqueness, of the weight” Even if we don't get a perfect match, we can correct it later with unit fixed effect (DID). 10

Slide 11

Slide 11 text

Synthetic Difference in Differences (SDID)
 ωSDID vertical regression + horizontal regression - Estimating the λ that regresses the post-term from the pre-term λSDID Objective variable: average outcome of post term non-negativity sum(λ)=1 with intercept Arkhangelsky, Dmitry, et al. Synthetic difference in differences. No. w25532. National Bureau of Economic Research, 2019. 11

Slide 12

Slide 12 text

Synthetic Difference in Differences (SDID)
 Arkhangelsky, Dmitry, et al. Synthetic difference in differences. No. w25532. National Bureau of Economic Research, 2019. 12

Slide 13

Slide 13 text

Synthetic Difference in Differences (SDID)
 pre post outcome treatment group synthetic control Synthetic Control Method ATTSC pre post outcome treatment group synthetic control Synthetic Difference in Differences   λ intercept ATTSDID Adjusting the reference point by λ 13

Slide 14

Slide 14 text

Reproduction experiment in Python
 My notebooks are here:
 https://github.com/MasaAsami/pysynthdid
 
 These implementations were based on the following R package:
 https://github.com/synth-inference/synthdid 
 14

Slide 15

Slide 15 text

Data
 The following section examines the Tobacco Tax and Health Protection Act of 1989 (California), famous dataset for Synthetic Control Method. see: 1988 California Proposition 99 ( https://en.wikipedia.org/wiki/1988_California_Proposition_99 ) 15

Slide 16

Slide 16 text

setup
 16

Slide 17

Slide 17 text

Setup & fitting
 1989 2000 1970 1988 California = treated Control (donor pool) 17

Slide 18

Slide 18 text

plot : sdid.plot(model=“hogehoge”)
 Difference in Differences Synthetic Control Method 18

Slide 19

Slide 19 text

plot : sdid.plot(model=“sdid”)
 19

Slide 20

Slide 20 text

Comparison of results
 pysynthdid result Arkhangelsky, Dmitry, et al. (2019) table 1: Arkhangelsky, Dmitry, et al. Synthetic difference in differences. No. w25532. National Bureau of Economic Research, 2019. 20

Slide 21

Slide 21 text

Comparison of ω
 Arkhangelsky, Dmitry, et al. Synthetic difference in differences. No. w25532. National Bureau of Economic Research, 2019. ω is able to confirm that the results were mostly consistent with the original paper. Also, it matches the results of the R package perfectly. typo?? 21

Slide 22

Slide 22 text

Comparison of λ
 Arkhangelsky, Dmitry, et al. Synthetic difference in differences. No. w25532. National Bureau of Economic Research, 2019. λ is almost identical to the original paper 22

Slide 23

Slide 23 text

Robust check for scale differences in donor pools
 In following note, we checked how the estimation results change with changes in the scale of the donor pool features. see: https://github.com/MasaAsami/pysynthdid/blob/main/notebook/ScaleTesting_of _DonorPools.ipynb Due to the L2 penalty term, mixing donor pools with extremely different orders may influence the results. In such a case, it may be better to consider logarithmic transformation, etc., as in a normal regression model.
 23

Slide 24

Slide 24 text

Thank you
 24