Testing Hypotheses of No Meaningful Effect

Testing Hypotheses of No Meaningful Effect

Bf99409063473973c7f9d3cf4f882492?s=128

Carlisle Rainey

January 04, 2013
Tweet

Transcript

  1. Testing Hypotheses of No Meaningful Effect Carlisle Rainey

  2. I'd like to convince you of three things

  3. Important 1

  4. Our arguments are not compelling 2

  5. Our arguments can be compelling 3

  6. Important Step 1:

  7. Hypotheses of no meaningful effect are crucial to complete evaluation

    of theories
  8. Interaction

  9. Social heterogeneity increases the number of parties, but only when

    electoral institutions are sufficiently permissive. “ ” Clark and Golder (2006)
  10. Adjudication

  11. None
  12. None
  13. How often do these examples occur?

  14. 30%

  15. Our arguments are not compelling Step 2:

  16. No American president since FDR has won a second term

    when the unemployment rate topped 7.2 percent... Obama must defy that trend to keep his job. “ ” New York Times June 1, 2011 “ ”
  17. None
  18. None
  19. The nation's unemployment status by itself is not going to

    affect Obama's. “ ” Seth Masket June 2, 2011
  20. My critique

  21. Rule out implausible relationships

  22. None
  23. None
  24. None
  25. None
  26. None
  27. None
  28. Insignificance can't be used to argue for “no effect.”

  29. But doesn't everyone already know that? “ ” A skeptic

  30. Political scientists draw strong conclusions from insignificance.

  31. Recessions have no effect on whether a democracy is consolidated.

    “ ” Svolik (2008)
  32. Our arguments can be compelling Step 3:

  33. just like any other hypothesis

  34. Argue against relationships inconsistent with the hypothesis

  35. Define substantively meaningful Step 1:

  36. Check the 90% CI Step 2:

  37. Only negativity about the respondent’s preferred candidate should have a

    significant demobilizing effect on voter turnout. “ ” Krupnikov (2011)
  38. What's a meaningful effect? 1% 3% 5% 7% 9%

  39. The demobilizing effect could be as large as 9%

  40. Those are the three things

  41. Important 1

  42. Our arguments are not compelling 2

  43. Our arguments can be compelling 3

  44. But what should I do about this? “ ” You

  45. Go read the details crain.co/nme 1

  46. Think about your own work. 2

  47. Keep in mind when reviewing others' work 3

  48. None