Cluster failure (fMRI data analysis) paper

Presentation of the cluster failure paper (fMRI data analysis, fMRI 101, software vs non-software bug) during papers we love #022 @ hackerspace.sg

Transcript

1. Ivan (vanzaj@gmail.com) Papers We Love #022 Aug 29, 2016 http://www.pnas.org/content/early/2016/06/27/1602413113.full
3. f?
5. the most common software packages for fMRI analysis … can result in false-positive rates of up to 70%. These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results. a bug that's been sitting in a package <xyz> for 15 years, fixed in May 2015, produced bad results. “ “
6. randcraw says: …For a couple years now philosophers have insisted that fMRI images prove there is no such thing as free will. pfooti says: The dead salmon study seems relevant here in discussion of how fMRI is used. “ “
7. honkhonkpants says: Doesn't sound like a straight up bug, but rather unsound statistical methods which can happen with or without software… greenyoda replies: I'd say it's a bug, since the unsound statistical methods are incorporated into the three most common software packages… “ “
9. MAGNETIC RESONANCE IMAGING (MRI)
10. “FMRI SCANNER” MRI + STIMULATION + RESPONSE by James Dankert, http://slideplayer.com/slide/6939789/
11. MR IMAGES: ANATOMY
12. MR IMAGES: PHYSIOLOGY functional MRI diffusion MRI (Dr. Rose, UQCCR) perfusion MRI
13. FMRI 101 rest (normal condition) increased activity (stimulus) Blood Oxygenation Level Dependent (BOLD) effect brain activity -> local increase on blood flow -> [oxy/deoxy hemoglobin] -> MR signal change (1-5%)
15. FMRI 101 EPI acquisition (brain volume) 10 to 15 slices, ~5mm each 64x64 to 128x128 in-plane res. (2 to 4mm) ~50 to 100K voxels/volume 100 to 200 volumes every 2 to 3 seconds Data Acquisition
16. FMRI 101 Data Analysis 1. Pre processing: motion correction, slice-timing, (spatial smoothing) 2. Statistical analysis: t-test, correlation, OLS, PCA, ICA, … 3. Visualization
18. METHODS • Resting state fMRI date from 499 subjects • 4 simulated activation conditions (2 block-, 2 event-related) • 4 levels of spatial smoothing • one- or two-sample t-test • voxel or cluster level inference
19. RESULTS Fig.1. Results for one-sample t test, showing estimated FWE rates
20. RESULTS Fig. 2. Results for two-sample t test and ad hoc cluster-wise inference, showing estimated FWE rates for 6 mm of smoothing
22. WHAT DOES IT MEAN? • Is all of fMRI junk science? NO • Is it a software bug? NO (AFNI’s 3dClustSim was a minor issue) • What about “philosophical/political” fMRI studies? ??? (read with a big grain of salt) • What about the “Salmon study”? GOOD STORY for STATS 102 (multiple comparisons)
23. http://blogs.discovermagazine.com/neuroskeptic/2016/07/07/false-positive-fmri-mainstream/ http://blogs.warwick.ac.uk/nichols/entry/bibliometrics_of_cluster/
24. time Hemodynamic Response Function OPEN DATA jballanc says: The real takeaway lesson from this research should be the vital importance of Open Data to the modern scientific enterprise “ adapted from https://xkcd.com/54/