Systematic review and meta-analysis: open synthesis science supports effective decision-making

60815911163520463199b5ebd5f48db0?s=47 cjlortie
May 26, 2020

Systematic review and meta-analysis: open synthesis science supports effective decision-making

A summary of the contemporary community of practice for synthesis science including systematic reviews and meta-analyses. The primary goal is to examine how synthesis can be used to inform decision making. Context, synthesis, and transparency tools structure this description of an open science philosophy to approach environmental grand challenges. There are three major implications of adopting this potential workflow for scientific syntheses. 1. Synthesis can confront biases and assumptions and thus change direction of science-based policy. 2. Transparency promotes interactions between the community of practice and our philosophy of science. Sandbox-thinking and tools that support version control, iteration, review, and reuse all promote better scientific synthesis. 3. Policy always uses evidence in some form, and contemporary scientific syntheses can provide a wider breadth of elements to draw from and support stronger evidence implementation. We must endeavor to provide a diverse, accessible set of evidence outcomes from our scientific syntheses to better support policy.

This work supported by this repository: https://github.com/cjlortie/synthesis4decisions and inspired by work with Florencia Miguel and Scott Butterfield in exploring evidence for dryland restoration https://cjlortie.github.io/dryland_restoration_synthesis/#a_synthesis_of_dryland_restoration_techniques

Resources
https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/1365-2656.12949
https://royalsocietypublishing.org/doi/10.1098/rsbl.2009.0546
https://academic.oup.com/icesjms/article/58/5/937/730862
https://www.environmentalevidence.org/completed-reviews/effects-of-wind-turbines-on-bird-abundance
https://royalsocietypublishing.org/doi/full/10.1098/rspb.2017.0829
https://academic.oup.com/bioscience/article/59/8/699/256190
https://esajournals.onlinelibrary.wiley.com/doi/full/10.1002/ecs2.2367
https://r4ds.had.co.nz
https://bioone.org/journals/freshwater-science/volume-29/issue-1/09-092.1/The-reference-condition--predicting-benchmarks-for-ecological-and-water/10.1899/09-092.1.full

60815911163520463199b5ebd5f48db0?s=128

cjlortie

May 26, 2020
Tweet

Transcript

  1. 2.
  2. 13.

    open synthesis science supports effective decision-making zoomed out, big picture

    
 however 
 better thinking, better data, & better tools
 are evolving its capacities
  3. 14.

    community of practice in synthesis science first principles of inquiry

    still prevail logic relevance common practices scientific principles shared challenges & acceptable solution sets strength of evidence paradigm increasingly open workflows and data
  4. 16.

    how to use synthesis decisions Gavin Stewart 2010 validate ideas

    & assumptions with evidence 
 (extent of what we test and we know) estimate strength of evidence 
 (effect sizes to capture relative difference) identify gaps, bias, and big-picture thinking*
  5. 18.
  6. 19.

    implicit bias, compelling case studies, & big data can inadvertently

    misdirect assessment of key drivers and processes that broadly describe patterns
  7. 20.
  8. 23.
  9. 24.
  10. 25.
  11. 26.
  12. 29.
  13. 31.

    short-term bird abundance studies do not provide robust indicators of

    the potentially deleterious impacts of wind farms on bird abundance small sample sizes are a critical issue
  14. 34.

    scientific synthesis combines theory, data, and tools to engineer and

    guide knowledge inspired by Carpenter et al. 2009
  15. 35.

    inspired by Carpenter et al. 2009 practically want to catch

    fishes (evidence) prep (community of practice ideals) then do some cooking (SR/meta)
  16. 36.

    meta-analyses and systematic reviews are most common but not the

    sole forms of synthesis for decision making environmental science estimated meta frequencies
  17. 37.
  18. 38.

    novelty today for synthesis for decision making needs driven rapidly

    increasing diversity in people & evidence cultural changes in acceptance and openness
  19. 43.
  20. 48.

    Transparency promotes interactions between community of practice and our philosophy

    of science. Sandbox-thinking and tools that support version control, iteration, review, and reuse all promote better scientific synthesis.
  21. 52.

    develop a workflow consider on open, collaborative set of mechanisms

    1. Search 2. Sort 3. Synthesize 4. Summarize 5. Statistics
  22. 53.

    tools that enable these steps dialog with stakeholders GitHub (version

    control, distributed work) open, published data in repositories team data checking R and scripts team science code review before writing
  23. 61.
  24. 63.

    Policy always uses evidence in some form, and contemporary scientific

    syntheses 
 can provide a wider breadth of products to support 
 stronger evidence implementation.