Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Testing Hypotheses of No Meaningful Effect
Search
Carlisle Rainey
January 04, 2013
Research
1
400
Testing Hypotheses of No Meaningful Effect
Carlisle Rainey
January 04, 2013
Tweet
Share
More Decks by Carlisle Rainey
See All by Carlisle Rainey
Topic Sampling @ SPSA 2024
carlislerainey
0
200
Topic Sampling
carlislerainey
1
420
ggcarly:theme_carly_presents()
carlislerainey
0
79
Unnecessary
carlislerainey
0
560
When BLUE Is Not Best: Non-Normal Errors and the Linear Model
carlislerainey
0
210
Dealing with Separation in Logistic Regression Models
carlislerainey
0
140
Logit Models in Small Samples
carlislerainey
0
110
When BLUE is Not Best
carlislerainey
0
450
Dealing with Separation in Logistic Regression Models
carlislerainey
0
510
Other Decks in Research
See All in Research
生成的推薦の人気バイアスの分析:暗記の観点から / JSAI2025
upura
0
180
引力・斥力を制御可能なランダム部分集合の確率分布
wasyro
0
150
When Submarine Cables Go Dark: Examining the Web Services Resilience Amid Global Internet Disruptions
irvin
0
200
ことばの意味を計算するしくみ
verypluming
11
2.6k
ASSADS:ASMR動画に合わせて撫でられる感覚を提示するシステムの開発と評価 / ec75-shimizu
yumulab
1
380
rtrec@dbem6
myui
6
860
SkySense : A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery
satai
3
240
Looking for Escorts in Sydney?
lunsophia
1
120
Towards a More Efficient Reasoning LLM: AIMO2 Solution Summary and Introduction to Fast-Math Models
analokmaus
2
220
Ad-DS Paper Circle #1
ykaneko1992
0
5.5k
LLM-as-a-Judge: 文章をLLMで評価する@教育機関DXシンポ
k141303
3
810
Pix2Poly: A Sequence Prediction Method for End-to-end Polygonal Building Footprint Extraction from Remote Sensing Imagery
satai
3
480
Featured
See All Featured
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
We Have a Design System, Now What?
morganepeng
53
7.7k
Building Adaptive Systems
keathley
43
2.6k
Facilitating Awesome Meetings
lara
54
6.4k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.5k
Building an army of robots
kneath
306
45k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
130
19k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Why You Should Never Use an ORM
jnunemaker
PRO
57
9.4k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Making Projects Easy
brettharned
116
6.3k
Transcript
Testing Hypotheses of No Meaningful Effect Carlisle Rainey
I'd like to convince you of three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
Important Step 1:
Hypotheses of no meaningful effect are crucial to complete evaluation
of theories
Interaction
Social heterogeneity increases the number of parties, but only when
electoral institutions are sufficiently permissive. “ ” Clark and Golder (2006)
Adjudication
None
None
How often do these examples occur?
30%
Our arguments are not compelling Step 2:
No American president since FDR has won a second term
when the unemployment rate topped 7.2 percent... Obama must defy that trend to keep his job. “ ” New York Times June 1, 2011 “ ”
None
None
The nation's unemployment status by itself is not going to
affect Obama's. “ ” Seth Masket June 2, 2011
My critique
Rule out implausible relationships
None
None
None
None
None
None
Insignificance can't be used to argue for “no effect.”
But doesn't everyone already know that? “ ” A skeptic
Political scientists draw strong conclusions from insignificance.
Recessions have no effect on whether a democracy is consolidated.
“ ” Svolik (2008)
Our arguments can be compelling Step 3:
just like any other hypothesis
Argue against relationships inconsistent with the hypothesis
Define substantively meaningful Step 1:
Check the 90% CI Step 2:
Only negativity about the respondent’s preferred candidate should have a
significant demobilizing effect on voter turnout. “ ” Krupnikov (2011)
What's a meaningful effect? 1% 3% 5% 7% 9%
The demobilizing effect could be as large as 9%
Those are the three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
But what should I do about this? “ ” You
Go read the details crain.co/nme 1
Think about your own work. 2
Keep in mind when reviewing others' work 3
None