Testing Hypotheses of
No Meaningful Effect
Carlisle Rainey
Slide 2
Slide 2 text
I'd like to convince you of
three things
Slide 3
Slide 3 text
Important
1
Slide 4
Slide 4 text
Our arguments are
not compelling
2
Slide 5
Slide 5 text
Our arguments can
be compelling
3
Slide 6
Slide 6 text
Important
Step 1:
Slide 7
Slide 7 text
Hypotheses of no meaningful effect are
crucial
to complete evaluation of theories
Slide 8
Slide 8 text
Interaction
Slide 9
Slide 9 text
Social heterogeneity increases the
number of parties, but only when electoral
institutions are sufficiently permissive.
“
”
Clark and Golder (2006)
Slide 10
Slide 10 text
Adjudication
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
How often do these
examples occur?
Slide 14
Slide 14 text
30%
Slide 15
Slide 15 text
Our arguments are
not compelling
Step 2:
Slide 16
Slide 16 text
No American president since FDR has won
a second term when the unemployment
rate topped 7.2 percent...
Obama must defy that
trend to keep his job.
“
”
New York Times
June 1, 2011
“
”
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
The nation's unemployment
status by itself is not going to
affect Obama's.
“
”
Seth Masket
June 2, 2011
Slide 20
Slide 20 text
My critique
Slide 21
Slide 21 text
Rule out implausible
relationships
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
No content
Slide 24
Slide 24 text
No content
Slide 25
Slide 25 text
No content
Slide 26
Slide 26 text
No content
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
Insignificance can't
be used to argue for
“no effect.”
Slide 29
Slide 29 text
But doesn't everyone
already know that?
“
”
A skeptic
Slide 30
Slide 30 text
Political scientists draw
strong conclusions
from insignificance.
Slide 31
Slide 31 text
Recessions have no effect
on whether a democracy is
consolidated.
“
”
Svolik (2008)
Slide 32
Slide 32 text
Our arguments
can be compelling
Step 3:
Slide 33
Slide 33 text
just like any
other hypothesis
Slide 34
Slide 34 text
Argue against
relationships inconsistent
with the hypothesis
Slide 35
Slide 35 text
Define
substantively
meaningful
Step 1:
Slide 36
Slide 36 text
Check the
90% CI
Step 2:
Slide 37
Slide 37 text
Only negativity about the respondent’s
preferred candidate should have a significant
demobilizing effect on voter turnout.
“
”
Krupnikov (2011)