Avoiding misleading reporting on systematic reviews & meta-analyses

Slides from a short presentation at the World Conference of Science Journalists in 2019, with links to backgrounder posts.

Transcript

1. Avoiding Misleading
   Reporting on
   Systematic Reviews &
   Meta-analyses
   World Conference of Science Journalists
   Lausanne
   2 July 2019
   Hilda Bastian
   @hildabast
   hildabastian.net
   

   View Slide

2. Studies are a collection of parts of
   uneven quality & quantity [+ spin]
   

   View Slide

3. Pitfall 1: Exaggerating the quantity of
   evidence
   https://well.blogs.nytimes.com/2016/06/27/putting-breast-cancer-on-a-diet/
   1
   

   View Slide

4. •  82 studies, published 31 meta-analyses
   •  84% of those analyses combined 20 or fewer
   studies – 58% had 10 or fewer
   •  The most studies in any one meta-analysis
   was 24
   https://academic.oup.com/annonc/article/25/10/1901/2801213
   

   View Slide

5. Common error: exaggerating the amount
   of evidence
   https://well.blogs.nytimes.com/2016/06/27/putting-breast-cancer-on-a-diet/
   

   View Slide

6. 21 studies with approx 42,000 women
   

   View Slide

7. 2
   . . .
   

   View Slide

8. Sherrington et al (2019). Exercise for preventing falls in older people living in the community.
   https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD012424.pub2/full
   8 studies with fewer than 2,700 people
   

   View Slide

9. From the Summary of Findings table:
   

   View Slide

10. Pitfall 2: Overestimating the strength of
    evidence
    

    View Slide

11. View Slide

12. View Slide

13. From 2008 Cochrane review:
    https://doi.org/10.1002/14651858.CD001321.pub4
    

    View Slide

14. From 2008:
    Update in 2012:
    

    View Slide

15. 5 tips for understanding data in meta-analysis: bit.ly/5TipsData
    5 key things to know about meta-analysis: bit.ly/MetaBasics
    Another 5 things to know about meta-analysis: bit.ly/MetaTraps
    Systematic reviews & meta-analyses 5 step checkup: bit.ly/MetaCheckup
    

    View Slide