Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Testing Hypotheses of No Meaningful Effect
Search
Carlisle Rainey
January 04, 2013
Research
1
410
Testing Hypotheses of No Meaningful Effect
Carlisle Rainey
January 04, 2013
Tweet
Share
More Decks by Carlisle Rainey
See All by Carlisle Rainey
Topic Sampling @ SPSA 2024
carlislerainey
0
230
Topic Sampling
carlislerainey
1
440
ggcarly:theme_carly_presents()
carlislerainey
0
87
Unnecessary
carlislerainey
0
570
When BLUE Is Not Best: Non-Normal Errors and the Linear Model
carlislerainey
0
230
Dealing with Separation in Logistic Regression Models
carlislerainey
0
150
Logit Models in Small Samples
carlislerainey
0
120
When BLUE is Not Best
carlislerainey
0
460
Dealing with Separation in Logistic Regression Models
carlislerainey
0
520
Other Decks in Research
See All in Research
令和最新技術で伝統掲示板を再構築: HonoX で作る型安全なスレッドフロート型掲示板 / かろっく@calloc134 - Hono Conference 2025
calloc134
0
330
Integrating Static Optimization and Dynamic Nature in JavaScript (GPCE 2025)
tadd
0
100
J-RAGBench: 日本語RAGにおける Generator評価ベンチマークの構築
koki_itai
0
800
MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generation
satai
4
330
多言語カスタマーインタビューの“壁”を越える~PMと生成AIの共創~ 株式会社ジグザグ 松野 亘
watarumatsuno
0
130
AI エージェントを活用した研究再現性の自動定量評価 / scisci2025
upura
1
170
超高速データサイエンス
matsui_528
1
150
SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images
satai
3
330
CVPR2025論文紹介:Unboxed
murakawatakuya
0
180
ip71_contraflow_reconfiguration
stkmsd
0
110
Adaptive Experimental Design for Efficient Average Treatment Effect Estimation and Treatment Choice
masakat0
0
120
Combinatorial Search with Generators
kei18
0
1k
Featured
See All Featured
A Tale of Four Properties
chriscoyier
161
23k
Writing Fast Ruby
sferik
629
62k
Leading Effective Engineering Teams in the AI Era
addyosmani
7
540
The Straight Up "How To Draw Better" Workshop
denniskardys
238
140k
Become a Pro
speakerdeck
PRO
29
5.6k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
9
920
A better future with KSS
kneath
239
18k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
34
2.3k
RailsConf 2023
tenderlove
30
1.3k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
10
600
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
253
22k
Product Roadmaps are Hard
iamctodd
PRO
55
11k
Transcript
Testing Hypotheses of No Meaningful Effect Carlisle Rainey
I'd like to convince you of three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
Important Step 1:
Hypotheses of no meaningful effect are crucial to complete evaluation
of theories
Interaction
Social heterogeneity increases the number of parties, but only when
electoral institutions are sufficiently permissive. “ ” Clark and Golder (2006)
Adjudication
None
None
How often do these examples occur?
30%
Our arguments are not compelling Step 2:
No American president since FDR has won a second term
when the unemployment rate topped 7.2 percent... Obama must defy that trend to keep his job. “ ” New York Times June 1, 2011 “ ”
None
None
The nation's unemployment status by itself is not going to
affect Obama's. “ ” Seth Masket June 2, 2011
My critique
Rule out implausible relationships
None
None
None
None
None
None
Insignificance can't be used to argue for “no effect.”
But doesn't everyone already know that? “ ” A skeptic
Political scientists draw strong conclusions from insignificance.
Recessions have no effect on whether a democracy is consolidated.
“ ” Svolik (2008)
Our arguments can be compelling Step 3:
just like any other hypothesis
Argue against relationships inconsistent with the hypothesis
Define substantively meaningful Step 1:
Check the 90% CI Step 2:
Only negativity about the respondent’s preferred candidate should have a
significant demobilizing effect on voter turnout. “ ” Krupnikov (2011)
What's a meaningful effect? 1% 3% 5% 7% 9%
The demobilizing effect could be as large as 9%
Those are the three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
But what should I do about this? “ ” You
Go read the details crain.co/nme 1
Think about your own work. 2
Keep in mind when reviewing others' work 3
None