Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Testing Hypotheses of No Meaningful Effect
Carlisle Rainey
January 04, 2013
Research
1
350
Testing Hypotheses of No Meaningful Effect
Carlisle Rainey
January 04, 2013
Tweet
Share
More Decks by Carlisle Rainey
See All by Carlisle Rainey
carlislerainey
0
43
carlislerainey
0
290
carlislerainey
0
130
carlislerainey
0
110
carlislerainey
0
80
carlislerainey
0
360
carlislerainey
0
400
carlislerainey
0
340
carlislerainey
0
340
Other Decks in Research
See All in Research
matsumoto_r
PRO
0
410
takuyaa
0
130
masadooon
3
2.1k
hiroki13
0
140
sei88888
0
220
scatterlab
0
1.3k
shunk031
0
760
sshimizu2006
1
200
oshinko90yen
1
510
ksnt
0
560
kayceesrk
2
3.4k
kikuzo
0
220
Featured
See All Featured
phodgson
87
3.9k
morganepeng
17
1.1k
davidbonilla
69
3.5k
vanstee
116
4.8k
michaelherold
225
8.5k
kastner
54
1.9k
orderedlist
PRO
329
35k
nonsquared
81
3.3k
addyosmani
310
21k
cassininazir
347
20k
destraynor
146
19k
jponch
103
5k
Transcript
Testing Hypotheses of No Meaningful Effect Carlisle Rainey
I'd like to convince you of three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
Important Step 1:
Hypotheses of no meaningful effect are crucial to complete evaluation
of theories
Interaction
Social heterogeneity increases the number of parties, but only when
electoral institutions are sufficiently permissive. “ ” Clark and Golder (2006)
Adjudication
None
None
How often do these examples occur?
30%
Our arguments are not compelling Step 2:
No American president since FDR has won a second term
when the unemployment rate topped 7.2 percent... Obama must defy that trend to keep his job. “ ” New York Times June 1, 2011 “ ”
None
None
The nation's unemployment status by itself is not going to
affect Obama's. “ ” Seth Masket June 2, 2011
My critique
Rule out implausible relationships
None
None
None
None
None
None
Insignificance can't be used to argue for “no effect.”
But doesn't everyone already know that? “ ” A skeptic
Political scientists draw strong conclusions from insignificance.
Recessions have no effect on whether a democracy is consolidated.
“ ” Svolik (2008)
Our arguments can be compelling Step 3:
just like any other hypothesis
Argue against relationships inconsistent with the hypothesis
Define substantively meaningful Step 1:
Check the 90% CI Step 2:
Only negativity about the respondent’s preferred candidate should have a
significant demobilizing effect on voter turnout. “ ” Krupnikov (2011)
What's a meaningful effect? 1% 3% 5% 7% 9%
The demobilizing effect could be as large as 9%
Those are the three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
But what should I do about this? “ ” You
Go read the details crain.co/nme 1
Think about your own work. 2
Keep in mind when reviewing others' work 3
None