Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Testing Hypotheses of No Meaningful Effect
Carlisle Rainey
January 04, 2013
Research
1
370
Testing Hypotheses of No Meaningful Effect
Carlisle Rainey
January 04, 2013
Tweet
Share
More Decks by Carlisle Rainey
See All by Carlisle Rainey
ggcarly:theme_carly_presents()
carlislerainey
0
49
Unnecessary
carlislerainey
0
360
When BLUE Is Not Best: Non-Normal Errors and the Linear Model
carlislerainey
0
150
Dealing with Separation in Logistic Regression Models
carlislerainey
0
110
Logit Models in Small Samples
carlislerainey
0
88
When BLUE is Not Best
carlislerainey
0
380
Dealing with Separation in Logistic Regression Models
carlislerainey
0
420
Strategic Mobilization: Why Proportional Representation Decreases Voter Mobilization.
carlislerainey
0
360
Politics and Need
carlislerainey
0
350
Other Decks in Research
See All in Research
Output Image - 10INC.
10inc
0
220
第10回チャンピオンズミーティング・アクエリアス杯ラウンド1集計 / Umamusume Aquarius 2022 Round1
kitachan_black
0
1.5k
第9回全日本コンピュータビジョン勉強会「StyleNeRF: A Style-based 3D Aware Generator for High-resolution Image Synthesis」発表資料
maguro27
1
890
2022 東工大 情報通信系 研究室紹介 (大岡山) / Research@ICT, Tokyo Tech (Ookayama Campus) 2022
icttitech
0
2.8k
まだ生態学に本格導入されていない統計的因果推論手法の紹介:傾向スコア、回帰分断デザイン、操作変数法を中心に
takehikoihayashi
2
860
Iterative source steering を用いたオンライン補助関数型独立ベクトル分析に基づくブラインド音源分離 / Blind source separation using online auxiliary-function-based independent vector analysis with iterative source steering
taishi
0
140
第13回チャンピオンズミーティング・タウラス杯ラウンド2集計 / Umamusume Taurus 2022 Round2
kitachan_black
0
120
Making CRDTs Byzantine fault tolerant
ept
0
190
スマートシティ基盤FIWAREをNode-REDで使う
kikuzo
0
330
第13回チャンピオンズミーティング・タウラス杯ラウンド1集計 / Umamusume Taurus 2022 Round1
kitachan_black
0
360
幼少期の自然体験が理科学習への態度に及ぼす影響
arumakan
0
830
SketchODE: Learning neural sketch representation in continuous time
dasayan05
0
120
Featured
See All Featured
What the flash - Photography Introduction
edds
61
10k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
7
1k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
181
15k
For a Future-Friendly Web
brad_frost
164
7.4k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
236
1M
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
212
20k
GraphQLの誤解/rethinking-graphql
sonatard
24
6.2k
Producing Creativity
orderedlist
PRO
333
37k
Building a Scalable Design System with Sketch
lauravandoore
447
30k
Music & Morning Musume
bryan
35
4.2k
How To Stay Up To Date on Web Technology
chriscoyier
780
250k
We Have a Design System, Now What?
morganepeng
35
2.9k
Transcript
Testing Hypotheses of No Meaningful Effect Carlisle Rainey
I'd like to convince you of three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
Important Step 1:
Hypotheses of no meaningful effect are crucial to complete evaluation
of theories
Interaction
Social heterogeneity increases the number of parties, but only when
electoral institutions are sufficiently permissive. “ ” Clark and Golder (2006)
Adjudication
None
None
How often do these examples occur?
30%
Our arguments are not compelling Step 2:
No American president since FDR has won a second term
when the unemployment rate topped 7.2 percent... Obama must defy that trend to keep his job. “ ” New York Times June 1, 2011 “ ”
None
None
The nation's unemployment status by itself is not going to
affect Obama's. “ ” Seth Masket June 2, 2011
My critique
Rule out implausible relationships
None
None
None
None
None
None
Insignificance can't be used to argue for “no effect.”
But doesn't everyone already know that? “ ” A skeptic
Political scientists draw strong conclusions from insignificance.
Recessions have no effect on whether a democracy is consolidated.
“ ” Svolik (2008)
Our arguments can be compelling Step 3:
just like any other hypothesis
Argue against relationships inconsistent with the hypothesis
Define substantively meaningful Step 1:
Check the 90% CI Step 2:
Only negativity about the respondent’s preferred candidate should have a
significant demobilizing effect on voter turnout. “ ” Krupnikov (2011)
What's a meaningful effect? 1% 3% 5% 7% 9%
The demobilizing effect could be as large as 9%
Those are the three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
But what should I do about this? “ ” You
Go read the details crain.co/nme 1
Think about your own work. 2
Keep in mind when reviewing others' work 3
None