Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Testing Hypotheses of No Meaningful Effect
Search
Carlisle Rainey
January 04, 2013
Research
1
400
Testing Hypotheses of No Meaningful Effect
Carlisle Rainey
January 04, 2013
Tweet
Share
More Decks by Carlisle Rainey
See All by Carlisle Rainey
Topic Sampling @ SPSA 2024
carlislerainey
0
140
Topic Sampling
carlislerainey
1
320
ggcarly:theme_carly_presents()
carlislerainey
0
70
Unnecessary
carlislerainey
0
490
When BLUE Is Not Best: Non-Normal Errors and the Linear Model
carlislerainey
0
190
Dealing with Separation in Logistic Regression Models
carlislerainey
0
120
Logit Models in Small Samples
carlislerainey
0
100
When BLUE is Not Best
carlislerainey
0
430
Dealing with Separation in Logistic Regression Models
carlislerainey
0
440
Other Decks in Research
See All in Research
クロスモーダル表現学習の研究動向: 音声関連を中心として
ryomasumura
3
500
Rの機械学習フレームワークの紹介〜tidymodelsを中心に〜 / machine_learning_with_r2024
s_uryu
0
180
Alexander Mielke Hellinger--Kantorovich (a.k.a. Wasserstein-Fisher-Rao) Spaces and Gradient Flows
jjzhu
3
140
精神疾患患者のアクティビティデータを利用したリハビリテーションのためのシステムに関する研究
comfortdesignlab
0
120
2024-01-23-az
sofievl
1
610
Cross-Media Information Spaces and Architectures
signer
PRO
0
120
脳卒中患者・家族からみた循環器病対策推進基本計画の進捗に関する調査
japanstrokeassociation
0
510
生成AIを用いたText to SQLの最前線
masatoto
1
1.2k
第12回全日本コンピュータビジョン勉強会:画像の自己教師あり学習における大規模データセット
naok615
0
480
My Journey as a UX Researcher
aranciap
0
1.1k
Феномен воли и техника самоопределения
ashapiro
0
120
CASCON 2023 Most Influential Paper Award Talk
tsantalis
0
110
Featured
See All Featured
The Cult of Friendly URLs
andyhume
72
5.6k
StorybookのUI Testing Handbookを読んだ
zakiyama
10
4.5k
RailsConf 2023
tenderlove
0
510
No one is an island. Learnings from fostering a developers community.
thoeni
14
2k
Adopting Sorbet at Scale
ufuk
66
8.5k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
6
950
Testing 201, or: Great Expectations
jmmastey
27
6.3k
Fantastic passwords and where to find them - at NoRuKo
philnash
35
2.4k
How GitHub Uses GitHub to Build GitHub
holman
467
290k
The MySQL Ecosystem @ GitHub 2015
samlambert
242
12k
Happy Clients
brianwarren
91
6.3k
How to train your dragon (web standard)
notwaldorf
71
5.1k
Transcript
Testing Hypotheses of No Meaningful Effect Carlisle Rainey
I'd like to convince you of three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
Important Step 1:
Hypotheses of no meaningful effect are crucial to complete evaluation
of theories
Interaction
Social heterogeneity increases the number of parties, but only when
electoral institutions are sufficiently permissive. “ ” Clark and Golder (2006)
Adjudication
None
None
How often do these examples occur?
30%
Our arguments are not compelling Step 2:
No American president since FDR has won a second term
when the unemployment rate topped 7.2 percent... Obama must defy that trend to keep his job. “ ” New York Times June 1, 2011 “ ”
None
None
The nation's unemployment status by itself is not going to
affect Obama's. “ ” Seth Masket June 2, 2011
My critique
Rule out implausible relationships
None
None
None
None
None
None
Insignificance can't be used to argue for “no effect.”
But doesn't everyone already know that? “ ” A skeptic
Political scientists draw strong conclusions from insignificance.
Recessions have no effect on whether a democracy is consolidated.
“ ” Svolik (2008)
Our arguments can be compelling Step 3:
just like any other hypothesis
Argue against relationships inconsistent with the hypothesis
Define substantively meaningful Step 1:
Check the 90% CI Step 2:
Only negativity about the respondent’s preferred candidate should have a
significant demobilizing effect on voter turnout. “ ” Krupnikov (2011)
What's a meaningful effect? 1% 3% 5% 7% 9%
The demobilizing effect could be as large as 9%
Those are the three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
But what should I do about this? “ ” You
Go read the details crain.co/nme 1
Think about your own work. 2
Keep in mind when reviewing others' work 3
None