Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Testing Hypotheses of No Meaningful Effect
Search
Carlisle Rainey
January 04, 2013
Research
1
400
Testing Hypotheses of No Meaningful Effect
Carlisle Rainey
January 04, 2013
Tweet
Share
More Decks by Carlisle Rainey
See All by Carlisle Rainey
Topic Sampling @ SPSA 2024
carlislerainey
0
170
Topic Sampling
carlislerainey
1
350
ggcarly:theme_carly_presents()
carlislerainey
0
75
Unnecessary
carlislerainey
0
520
When BLUE Is Not Best: Non-Normal Errors and the Linear Model
carlislerainey
0
190
Dealing with Separation in Logistic Regression Models
carlislerainey
0
130
Logit Models in Small Samples
carlislerainey
0
110
When BLUE is Not Best
carlislerainey
0
430
Dealing with Separation in Logistic Regression Models
carlislerainey
0
460
Other Decks in Research
See All in Research
研究効率化Tips_2024 / Research Efficiency Tips 2024
ryo_nakamura
5
4.1k
機械学習を用いたポケモン対戦選出予測
fufufukakaka
1
560
SSII2024 [PD] SSII、次の30年への期待
ssii
PRO
2
1.3k
SSII2024 [TS2] 深層学習に潜むバイアス
ssii
PRO
0
520
論文紹介 AST: Audio Spectrogram Transformer
kazu07
0
190
SSII2024 [OS1] 現場の課題を解決する ロボットラーニング
ssii
PRO
0
420
SSII2024 [PD] 画像センシングの未来
ssii
PRO
0
290
20240523 Japan Investment Conference 2024
cfajapan
0
730
「確率的なオウム」にできること、またそれがなぜできるのかについて
eumesy
PRO
7
2.5k
生成AIエージェントの現状を俯瞰する
isidaitc
0
2.4k
初めての研究発表を成功させよう! スライド作成の基本
ayaco0
10
4.1k
ICLR2024 LLMエージェントの研究動向
masatoto
13
9.1k
Featured
See All Featured
Leading Effective Engineering Teams 2024
addyosmani
3
300
How GitHub Uses GitHub to Build GitHub
holman
471
290k
For a Future-Friendly Web
brad_frost
173
9.2k
A Philosophy of Restraint
colly
200
16k
Design by the Numbers
sachag
277
18k
Testing 201, or: Great Expectations
jmmastey
33
6.9k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
36
9.1k
Gamification - CAS2011
davidbonilla
78
4.9k
Git: the NoSQL Database
bkeepers
PRO
423
64k
Agile that works and the tools we love
rasmusluckow
325
20k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
35
6.3k
Building Adaptive Systems
keathley
34
2k
Transcript
Testing Hypotheses of No Meaningful Effect Carlisle Rainey
I'd like to convince you of three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
Important Step 1:
Hypotheses of no meaningful effect are crucial to complete evaluation
of theories
Interaction
Social heterogeneity increases the number of parties, but only when
electoral institutions are sufficiently permissive. “ ” Clark and Golder (2006)
Adjudication
None
None
How often do these examples occur?
30%
Our arguments are not compelling Step 2:
No American president since FDR has won a second term
when the unemployment rate topped 7.2 percent... Obama must defy that trend to keep his job. “ ” New York Times June 1, 2011 “ ”
None
None
The nation's unemployment status by itself is not going to
affect Obama's. “ ” Seth Masket June 2, 2011
My critique
Rule out implausible relationships
None
None
None
None
None
None
Insignificance can't be used to argue for “no effect.”
But doesn't everyone already know that? “ ” A skeptic
Political scientists draw strong conclusions from insignificance.
Recessions have no effect on whether a democracy is consolidated.
“ ” Svolik (2008)
Our arguments can be compelling Step 3:
just like any other hypothesis
Argue against relationships inconsistent with the hypothesis
Define substantively meaningful Step 1:
Check the 90% CI Step 2:
Only negativity about the respondent’s preferred candidate should have a
significant demobilizing effect on voter turnout. “ ” Krupnikov (2011)
What's a meaningful effect? 1% 3% 5% 7% 9%
The demobilizing effect could be as large as 9%
Those are the three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
But what should I do about this? “ ” You
Go read the details crain.co/nme 1
Think about your own work. 2
Keep in mind when reviewing others' work 3
None