Meta thinking: defence for dark arts

Cdb758075f7b54ea4ef646898497ecf3?s=47 cjlortie
February 23, 2019

Meta thinking: defence for dark arts

Criteria and philosophy for meta thinking as an antidote against the dark arts (in science). Big picture, reproducible synthesis science is proposed as a means to shift thinking on evidence, decision making, p-hacking, the weight we place on singular findings.



February 23, 2019


  1. meta thinking: defense for dark arts @cjlortie

  2. None
  3. None
  4. None
  5. None
  6. None
  7. evidence-based decision making

  8. None
  9. lack of evidence

  10. meta thinking process-based mind set

  11. big picture scientific synthesis iteration product and process reduced significance

    testing & p-hacking data differences likelihood & confidence limits transparency sensitivity analyses & scale dependence efficacy evidence thresholds & gaps constellations of evidence models: descriptive or predictive
  12. evidence thresholds

  13. heuristics from thresholds

  14. >1 inspiration >15 estimates >100(0) at scale == likelihoods

  15. constellations of evidence

  16. each study can shine brightly but collective effects can explore

    larger patterns
  17. truth of specific principles elementary evidence contrast to predictive syntheses

    meta models purpose
  18. systematic review versus meta?

  19. (who, what, why, how, where, when, with what) how well

    did it work? SRs metas
  20. R versus other languages or platforms to construct models?

  21. transparent reproducible do-over workflow capacity to explore models & bias

  22. How do you best frame the problem & find the

    ‘right’ set of evidence?
  23. team science, experts, iteration & sensitivity

  24. What does the future look like for metas?

  25. the future open data effect sizes in primary research contrasts

    reuse iterative metas novel thinking less Ps, more Qs