Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Testing Hypotheses of No Meaningful Effect
Search
Carlisle Rainey
January 04, 2013
Research
1
400
Testing Hypotheses of No Meaningful Effect
Carlisle Rainey
January 04, 2013
Tweet
Share
More Decks by Carlisle Rainey
See All by Carlisle Rainey
Topic Sampling @ SPSA 2024
carlislerainey
0
200
Topic Sampling
carlislerainey
1
420
ggcarly:theme_carly_presents()
carlislerainey
0
79
Unnecessary
carlislerainey
0
560
When BLUE Is Not Best: Non-Normal Errors and the Linear Model
carlislerainey
0
210
Dealing with Separation in Logistic Regression Models
carlislerainey
0
140
Logit Models in Small Samples
carlislerainey
0
110
When BLUE is Not Best
carlislerainey
0
450
Dealing with Separation in Logistic Regression Models
carlislerainey
0
510
Other Decks in Research
See All in Research
EarthMarker: A Visual Prompting Multimodal Large Language Model for Remote Sensing
satai
3
280
[輪講] SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
nk35jk
2
420
研究テーマのデザインと研究遂行の方法論
hisashiishihara
5
1.4k
ストレス計測方法の確立に向けたマルチモーダルデータの活用
yurikomium
0
460
CHaserWeb:ブラウザ上で動作する対戦型プログラミング学習環境の提案と評価 / i2025-inoue
yumulab
0
190
Mathematics in the Age of AI and the 4 Generation University
hachama
0
160
ノンパラメトリック分布表現を用いた位置尤度場周辺化によるRTK-GNSSの整数アンビギュイティ推定
aoki_nosse
0
320
SSII2025 [SS1] レンズレスカメラ
ssii
PRO
2
900
(NULLCON Goa 2025)Windows Keylogger Detection: Targeting Past and Present Keylogging Techniques
asuna_jp
1
510
Computational OT #4 - Gradient flow and diffusion models
gpeyre
0
260
SI-D案内資料_京都文教大学
ryojitakeuchi1116
0
1.6k
Computational OT #1 - Monge and Kantorovitch
gpeyre
0
170
Featured
See All Featured
Why You Should Never Use an ORM
jnunemaker
PRO
56
9.4k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
331
22k
Side Projects
sachag
454
42k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
BBQ
matthewcrist
89
9.7k
Typedesign – Prime Four
hannesfritz
42
2.7k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
GraphQLとの向き合い方2022年版
quramy
46
14k
Bash Introduction
62gerente
614
210k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
107
19k
Transcript
Testing Hypotheses of No Meaningful Effect Carlisle Rainey
I'd like to convince you of three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
Important Step 1:
Hypotheses of no meaningful effect are crucial to complete evaluation
of theories
Interaction
Social heterogeneity increases the number of parties, but only when
electoral institutions are sufficiently permissive. “ ” Clark and Golder (2006)
Adjudication
None
None
How often do these examples occur?
30%
Our arguments are not compelling Step 2:
No American president since FDR has won a second term
when the unemployment rate topped 7.2 percent... Obama must defy that trend to keep his job. “ ” New York Times June 1, 2011 “ ”
None
None
The nation's unemployment status by itself is not going to
affect Obama's. “ ” Seth Masket June 2, 2011
My critique
Rule out implausible relationships
None
None
None
None
None
None
Insignificance can't be used to argue for “no effect.”
But doesn't everyone already know that? “ ” A skeptic
Political scientists draw strong conclusions from insignificance.
Recessions have no effect on whether a democracy is consolidated.
“ ” Svolik (2008)
Our arguments can be compelling Step 3:
just like any other hypothesis
Argue against relationships inconsistent with the hypothesis
Define substantively meaningful Step 1:
Check the 90% CI Step 2:
Only negativity about the respondent’s preferred candidate should have a
significant demobilizing effect on voter turnout. “ ” Krupnikov (2011)
What's a meaningful effect? 1% 3% 5% 7% 9%
The demobilizing effect could be as large as 9%
Those are the three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
But what should I do about this? “ ” You
Go read the details crain.co/nme 1
Think about your own work. 2
Keep in mind when reviewing others' work 3
None