Denis A. Engemann, PhD 7/15/20 MNI Challenges of building clinical biomarkers from M/EEG Multimodal modeling with missing data and robust regression on power spectra [email protected] www.denis-engemann.de github: @dengemann twitter: @dngman 

Woo et al. 2017 Nat Neuro Rev. Predicting clinical endpoints from multiple brain signals Mass-Univariate statistics Combine multiple inputs into single prediction Too many models … AI! Yeah! 

Woo et al. 2017 Nat Neuro Rev. Predicting clinical endpoints from multiple brain signals Mass-Univariate statistics Combine multiple inputs into single prediction AI !! Too many models … 

Predicting clinical endpoints from multiple brain signals 

Predicting clinical endpoints from multiple brain signals NO T SO FAST !! 

Caveat: generative mechanisms mostly unknown Jonas & Kording 2017, PLOS Comp. Biol. 

Bzdok, Engemann et al. 2018 bioRxiv [https://github.com/banilo/inf_vs_pred_2018/] Caveat: inference is not prediction Significance does not imply that prediction will work! 

Caveat: Prediction accuracy depends on sample size. Often in a bad way … Varoquaux 2017, Neuroimage 

Interim-Summary: Are we doomed? Generative mechanism? Inference? Measuring performance? 

Generative mechanism? Inference? Measuring performance? In high-dimensional small-data regimes: Have good priors! Interim-Summary: Are we doomed? 

Challenge: Predicting a precious clinical outcome •Problem: few / expensive data on outcome e.g. cognitive decline •Brain Age Delta = predicted age (PAD) - passport age
 •Precocious aging induces cognitive dysfunction (CD) and risk of mortality
 •Typically estimated with MRI! 

•Problem: few / expensive data on outcome e.g. cognitive decline •Idea: Predict widely available outcome; exploit correlation with the outcome of interest, e.g. age •Brain Age Delta = predicted age (PAD) - passport age
 •Precocious aging induces cognitive dysfunction (CD) and risk of mortality
 •Typically estimated with MRI! Challenge: Predicting a precious clinical outcome 

Solution: Surrogate Biomarker e.g. Brain Age Cole et al. Mol. Psych. 2018 •Problem: few / expensive data on outcome e.g. cognitive decline •Idea: Predict widely available outcome; exploit correlation with the outcome of interest, e.g. age •Brain Age Delta = predicted age (PAD) - passport age
 •Precocious aging induces cognitive dysfunction (CD) and risk of mortality
 •Typically estimated with MRI! 

•Problem: few / expensive data on outcome e.g. cognitive decline •Idea: Predict widely available outcome; exploit correlation with the outcome of interest, e.g. age •Brain Age Delta = predicted age (PAD) - passport age
 •High PAD is not good … •Typically estimated with MRI! Cole et al. Mol. Psych. 2018 Liem et al 2017 NIMG Liem et al 2017 NIMG Solution: Surrogate Biomarker e.g. Brain Age 

Shall we bother about M/EEG? Brookes et al. 2011, PNAS fMRI resting state networks can be reconstructed from MEG Historically strong emphasis on similarities between M/EEG and fMRI 

Shall we bother about M/EEG? Hipp & Siegel 2015, Curr. Biol. BOLD and MEG show spatial correlations across many frequency bands. Historically strong emphasis on similarities between M/EEG and fMRI 

But perhaps … Kumral et al. 2020 NIMG BOLD and EEG signal variability at rest differently relate to aging. 

But perhaps we should … Nentwich et al. 2020 NIMG fMRI and EEG connectivity is different. 

Using the MEG system at Neurospin … How do we get M/EEG signals? 

… or using Python How do we get M/EEG signals? 

… M/EEG signals! 

1e−28 1e−27 1e−26 1 3 10 30 Frequecy (Hz) log10(MEG2) age group (17.9,28] (28,38] (38,48] (48,58] (58,68] (68,78] (78,88.1] What MEG features shall we use? 

1e−28 1e−27 1e−26 1 3 10 30 Frequecy (Hz) log10(MEG2) age group (17.9,28] (28,38] (38,48] (48,58] (58,68] (68,78] (78,88.1] α-power Engemann 2018 Brain What MEG features shall we use? 

Engemann, Raimondo, …, Dehaene & Sitt, Brain, 2018 Alpha band power enables EEG-based cross-site classification in disorders of consciousness. 

1e−28 1e−27 1e−26 1 3 10 30 Frequecy (Hz) log10(MEG2) age group (17.9,28] (28,38] (38,48] (48,58] (58,68] (68,78] (78,88.1] α-peak α-power Babiloni 2006 HBM Engemann 2018 Brain Age prediction: Which M/EEG features? 

1e−28 1e−27 1e−26 1 3 10 30 Frequecy (Hz) log10(MEG2) age group (17.9,28] (28,38] (38,48] (48,58] (58,68] (68,78] (78,88.1] α-peak Power topography Gaubert 2019 Brain Fruehwirt 2017 NeurIPS workshop α-power Babiloni 2006 HBM Engemann 2018 Brain Age prediction: Which M/EEG features? 

1e−28 1e−27 1e−26 1 3 10 30 Frequecy (Hz) log10(MEG2) age group (17.9,28] (28,38] (38,48] (48,58] (58,68] (68,78] (78,88.1] 1/f fits Voytek et al. 2015 JoN α-peak Power topography Gaubert 2019 Brain Fruehwirt 2017 NeurIPS workshop α-power Babiloni 2006 HBM Engemann 2018 Brain Age prediction: Which M/EEG features? 

1e−28 1e−27 1e−26 1 3 10 30 Frequecy (Hz) log10(MEG2) age group (17.9,28] (28,38] (38,48] (48,58] (58,68] (68,78] (78,88.1] 1/f fits Voytek et al. 2015 JoN α-peak Power topography Gaubert 2019 Brain Fruehwirt 2017 NeurIPS workshop α-power Babiloni 2006 HBM Engemann 2018 Brain Other classical features • Evoked latency (Price 2017 Nat Coms) • 1/f topography Age prediction: Which M/EEG features? 

1e−28 1e−27 1e−26 1 3 10 30 Frequecy (Hz) log10(MEG2) age group (17.9,28] (28,38] (38,48] (48,58] (58,68] (68,78] (78,88.1] 1/f fits Voytek et al. 2015 JoN α-peak Power topography Gaubert 2019 Brain Fruehwirt 2017 NeurIPS workshop α-power Babiloni 2006 HBM Engemann 2018 Brain Other classical features • Evoked latency (Price 2017 Nat Coms) • 1/f topography Age prediction: Which M/EEG features? Possible enhancement: Analyze power in source space! 

Objective: predict target fro x Sabbagh et al. 2019 (NeurIPS) 2020 (NIMG) How to build MEG-based regression models? 

Objective: predict target fro x Sabbagh et al. 2019 (NeurIPS) 2020 (NIMG) Use AI ? How to build MEG-based regression models? 

Objective: predict target fro x Sabbagh et al. 2019 (NeurIPS) 2020 (NIMG) Use AI ? NOPE How to build MEG-based regression models? 

z ? Objective: predict target from M/EEG Neurophysiological genera or Maxwell's eq. neural mechanism Sabbagh et al. 2019 (NeurIPS) 2020 (NIMG) Primary currents How to build MEG-based regression models? 

How to build MEG-based regression models? z s ? Objective: predict target from M/EEG Neurophysiological genera or a is ical o el statistical sources Maxwell's eq. neural mechanism Sabbagh et al. 2019 (NeurIPS) 2020 (NIMG) 

z s ? Objective: predict target from M/EEG Neurophysiological genera or a is ical o el statistical sources M/EEG signals Maxwell's eq. neural mechanism Sabbagh et al. 2019 (NeurIPS) 2020 (NIMG) How to build MEG-based regression models? 

How to build an MEG-base regression model? z s ? Objective: predict target from M/EEG Neurophysiological genera or a is ical o el biomedical outcome statistical sources M/EEG signals Maxwell's eq. neural mechanism Sabbagh et al. 2019 (NeurIPS) 2020 (NIMG) 

z s ? Objective: predict target from M/EEG Neurophysiological genera or a is ical o el biomedical outcome statistical sources M/EEG signals Maxwell's eq. neural mechanism f log s Sabbagh et al. 2019 (NeurIPS) 2020 (NIMG) How to build MEG-based regression models? 

And is it worth the effort? Multimodal input data anatomical MRI functional MRI MEG Layer I: Ridge Regression Age predictions Missing value coding Layer II: Random Forest Regressor tree = 1 tree = 1 subject #i age = 50 age #i = 53 age = 57 age = 5 tree = ... tree = B Multimodal input data anatomical MRI functional MRI MEG Layer I: Ridge Regression Age predictions Missing value coding Layer II: Random Forest Regressor tree = 1 tree = 1 subject #i age = 50 age #i = 53 age = 57 age = 5 tree = ... tree = B 2 1 2 21 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 true age 2 21 true age 2 1 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 Engemann et al. 2020 eLife Stacking method: Wolpert 1992, Liem et al. 2017, NIMG; Karrer et al. 2019, HBM, … How shall we combine MEG with MRI? 

Multimodal input data anatomical MRI functional MRI MEG Layer I: Ridge Regression Age predictions Missing value coding Layer II: Random Forest Regressor tree = 1 tree = 1 subject #i age = 50 age #i = 53 age = 57 age = 5 tree = ... tree = B Multimodal input data anatomical MRI functional MRI MEG Layer I: Ridge Regression Age predictions Missing value coding Layer II: Random Forest Regressor tree = 1 tree = 1 subject #i age = 50 age #i = 53 age = 57 age = 5 tree = ... tree = B 2 1 2 21 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 true age 2 21 true age 2 1 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 Engemann et al. 2020 eLife Stacking method: Wolpert 1992, Liem et al. 2017, NIMG; Karrer et al. 2019, HBM, … And is it worth the effort? How shall we combine MEG with MRI? 

Multimodal input data anatomical MRI functional MRI MEG Layer I: Ridge Regression Age predictions Missing value coding Layer II: Random Forest Regressor tree = 1 tree = 1 subject #i age = 50 age #i = 53 age = 57 age = 5 tree = ... tree = B Multimodal input data anatomical MRI functional MRI MEG Layer I: Ridge Regression Age predictions Missing value coding Layer II: Random Forest Regressor tree = 1 tree = 1 subject #i age = 50 age #i = 53 age = 57 age = 5 tree = ... tree = B 2 1 2 21 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 true age 2 21 true age 2 1 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 Engemann et al. 2020 eLife Stacking method: Wolpert 1992, Liem et al. 2017, NIMG; Karrer et al. 2019, HBM, … And is it worth the effort? How shall we combine MEG with MRI? 

Multimodal input data anatomical MRI functional MRI MEG Layer I: Ridge Regression Age predictions Missing value coding Layer II: Random Forest Regressor tree = 1 tree = 1 subject #i age = 50 age #i = 53 age = 57 age = 5 tree = ... tree = B Multimodal input data anatomical MRI functional MRI MEG Layer I: Ridge Regression Age predictions Missing value coding Layer II: Random Forest Regressor tree = 1 tree = 1 subject #i age = 50 age #i = 53 age = 57 age = 5 tree = ... tree = B 2 1 2 21 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 true age 2 21 true age 2 1 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 Engemann et al. 2020 eLife Stacking method: Wolpert 1992, Liem et al. 2017, NIMG; Karrer et al. 2019, HBM, … And is it worth the effort? How shall we combine MEG with MRI? 

Is it worth the effort? Multimodal input data anatomical MRI functional MRI MEG Layer I: Ridge Regression Age predictions Missing value coding Layer II: Random Forest Regressor tree = 1 tree = 1 subject #i age = 50 age #i = 53 age = 57 age = 5 tree = ... tree = B Multimodal input data anatomical MRI functional MRI MEG Layer I: Ridge Regression Age predictions Missing value coding Layer II: Random Forest Regressor tree = 1 tree = 1 subject #i age = 50 age #i = 53 age = 57 age = 5 tree = ... tree = B 2 1 2 21 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 true age 2 21 true age 2 1 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 Engemann et al. 2020 eLife Stacking method: Wolpert 1992, Liem et al. 2017, NIMG; Karrer et al. 2019, HBM, … How shall we combine MEG with MRI? 

Overview on all features Engemann et al. 2020 eLife Liem et al 2017 NIMG NEW! 

Predicting brain age from MRI & MEG enhances predictive performance & cognitive phenotyping Engemann et al. (2020) eLife - Cam-CAN dataset 4.7 5.1 5.2 6.0 −3.0 −2.5 −2.0 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 MAE difference (years) Multimodal stacking m ro ement o er anatomical M M fM ME M fM M ME M no M anat. M anat. added 0 10 20 30 0 10 20 30 0 10 20 30 MAEfM (years) MAEME (years) age 20 40 60 0 A B 

Engemann et al. (2020) eLife - Cam-CAN dataset 4.7 5.1 5.2 6.0 −3.0 −2.5 −2.0 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 MAE difference (years) Multimodal stacking m ro ement o er anatomical M M fM ME M fM M ME M no M anat. M anat. added 0 10 20 30 0 10 20 30 0 10 20 30 MAEfM (years) MAEME (years) age 20 40 60 0 A B Predicting brain age from MRI & MEG enhances predictive performance & cognitive phenotyping 

What aspects of MEG are most influential? Peaks - Latencies - 1/f - Power - Connectivity Engemann et al. 2020 eLife α βlow βlow βlow Ecat Pcat 10 15 0.01 0.10 1.00 Variable importance (MAE) MAE (years) family source activity source connectivity sensor mixed input/feature si nal envelope 1 f slope α pea E lat B predictin y ensor Mixed 1 15 1 

What if some modalities are (sometimes) missing? opportunistic missing value handling Engemann et al. 2020 eLife on Age predictions Missing value coding Layer II: Random Forest Regressor on Age predictions Missing value coding Layer II: Random Forest Regressor 2 1 2 21 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 true age 2 21 true age 2 1 A A A A 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 2 1 1e 1e 1e 1e 1 2 2 2 21 1 1 4.4 4.6 4.6 4.7 8.6 14.6 MRI fMRI fMRI MEGsens MRI fMRI MEG MRI fMRI MEGsens MEG MEGsens 0 10 20 30 MAE Available inputs A B 

Interim-Summary 1. MEG contains unique information on cognitive aging 2. MEG source power is a good feature 3. Tree-based methods bring flexible handling of NA’s 

Interim-Summary 1. MEG contains unique information on cognitive aging 2. MEG source power is a good feature 3. Tree-based methods bring flexible handling of NA’s Limitations: What we find in most hospitals looks a bit different … do I need MEG + MRI for source localization ? 

WHAT IF I TOLD YOU … … that we don’t need MRI 

… hack the Covariance Matrix ! 

M/EEG Covariance Matrix Ci = Xi X⊤/T ∈ ℝP×P 

M/EEG Covariance Matrix: Ci = Xi X⊤ i /T ∈ ℝP×P 

M/EEG Covariance Matrix: var(Xi ) = diag(Ci ) ∈ ℝP Power is variance Ci = Xi X⊤ i /T ∈ ℝP×P 

M/EEG Covariance Matrix: Power is variance var(Xi ) = diag(Ci ) ∈ ℝP Ci = Xi X⊤ i /T ∈ ℝP×P “diag” — our baseline representation! 

Predicting from M/EEG source power Without biophysical source localization z s ? Objective: predict target from M/EEG Neurophysiological genera or a is ical o el biomedical outcome statistical sources M/EEG signals Maxwell's eq. neural mechanism f log s Problem: field spread is evil. Sabbagh et al. 2019 (NeurIPS) 2020 (NIMG) Ci = Xi X⊤ i /T ∈ ℝP×P Use cov. as representation 

Predicting from M/EEG source power Without biophysical source localization z s ? Objective: predict target from M/EEG Neurophysiological genera or a is ical o el biomedical outcome statistical sources M/EEG signals Maxwell's eq. neural mechanism f log s Problem: field spread is evil. Idea: get immunized against this evil Sabbagh et al. 2019 (NeurIPS) 2020 (NIMG) Take out volume conduction using Riemannian embeddings, or spatial filters Ci = Xi X⊤ i /T ∈ ℝP×P Use cov. as representation logm ( ¯ C−1/2Ci ¯ C−1/2) 

Simulations Are shortcuts — in principle — possible? Sabbagh et al. 2019 (NeurIPS) 2020 (NIMG) distance from identity chance level 0.00 0.25 0.50 0.75 1.00 0.0 0.5 1.0 1.5 2.0 2.5 3.0 µ Normalized MAE noise on target 0.00 0.25 0.50 0.75 1.00 0.01 0.10 1.00 10.00 σ 0.00 0.25 0.50 0.75 1.00 upper diag S o iemann noise on mi ing matri 0.01 0.10 1.00 σ z s ? Objective: predict target from M/EEG Neurophysiological genera or a is ical o el biomedical outcome statistical sources M/EEG signals Maxwell's eq. neural mechanism f log s 

Empirical benchmarks How do these models compare with real data and unknown degrees model violations? Sabbagh et al. 2020 (NIMG) 7.98 8.11 8.76 8.76 9.17 10.89 predicting age Riemann53 Riemann SPoC67 SPoC diag upper 8 10 12 14 16 MAE upper diag SPoC Riemann MEG — sensor space MEG — source space 

Empirical benchmarks How do these models compare with real data and unknown degrees model violations? Sabbagh et al. 2020 (NIMG) 7.98 8.11 8.76 8.76 9.17 10.89 predicting age Riemann53 Riemann SPoC67 SPoC diag upper 8 10 12 14 16 MAE upper diag SPoC Riemann MEG — sensor space MEG — source space Q: Do we expect the same ranking? 

Empirical benchmarks How do these models compare with real data and unknown degrees model violations? Sabbagh et al. 2020 (NIMG) 7.69 9.50 10.98 11.67 11.86 12.46 predicting age diag Riemann11 Riemann upper SPoC20 SPoC 5.0 7.5 10.0 12.5 15.0 17.5 MAE upper diag SPoC Riemann 7.98 8.11 8.76 8.76 9.17 10.89 predicting age Riemann53 Riemann SPoC67 SPoC diag upper 8 10 12 14 16 MAE upper diag SPoC Riemann MEG — sensor space MEG — source space Observations: (1) the baseline model is the best in source space 

Empirical benchmarks How do these models compare with real data and unknown degrees model violations? Sabbagh et al. 2020 (NIMG) 7.69 9.50 10.98 11.67 11.86 12.46 predicting age diag Riemann11 Riemann upper SPoC20 SPoC 5.0 7.5 10.0 12.5 15.0 17.5 MAE upper diag SPoC Riemann 7.98 8.11 8.76 8.76 9.17 10.89 predicting age Riemann53 Riemann SPoC67 SPoC diag upper 8 10 12 14 16 MAE upper diag SPoC Riemann MEG — sensor space MEG — source space Observations: (1) the baseline model is the best in source space (2) Riemannian embeddings get closest to results with source localization 

Empirical benchmarks How do these models compare with real data and unknown degrees model violations? Sabbagh et al. 2020 (NIMG) 7.98 8.11 8.76 8.76 9.17 10.89 predicting age Riemann53 Riemann SPoC67 SPoC diag upper 8 10 12 14 16 MAE upper diag SPoC Riemann MEG — sensor space EEG— sensor space (TUH data, n=1385) Q: Can we get similar results on 21-chan. EEG as compared to 306-chan. MEG? 

Empirical benchmarks How do these models compare with real data and unknown degrees model violations? Sabbagh et al. 2020 (NIMG) 8.21 8.27 9.63 10.72 predicting age Riemann19 Riemann SPoC21 diag upper 5.0 7.5 10.0 12.5 15.0 MAE upper diag SPoC Riemann 7.98 8.11 8.76 8.76 9.17 10.89 predicting age Riemann53 Riemann SPoC67 SPoC diag upper 8 10 12 14 16 MAE upper diag SPoC Riemann MEG — sensor space EEG— sensor space (TUH data, n=1385) EEG can in principle be substituted for MEG 

Summary What shall I remember? 1. MEG contains unique information on cognitive aging 2. MEG source power is a good feature 3. Tree-based methods bring flexible handling of NA’s 4. Statistical-mathematical shortcuts can help avoid source localization 5. EEG may be substituted for MEG (when predicting age) 

Resources - scalable processing of M/EEG data http://mne.tools 

Resources - scalable processing of M/EEG data Jas, Engemann, et al. (2017) NIMG http://autoreject.github.io http://mne.tools 

Resources - scalable processing of M/EEG data http://autoreject.github.io Covariance estimation: Engemann & Gramfort (2015) NIMG Jas, Engemann, et al. (2017) NIMG http://mne.tools 

Resources - scalable processing of M/EEG data http://autoreject.github.io Covariance estimation: Engemann & Gramfort (2015) NIMG https://mne.tools/mne-hcp/ Jas, Engemann, et al. (2017) NIMG http://mne.tools 

Resources - scalable processing of M/EEG data http://autoreject.github.io http://mne.tools Covariance estimation: Engemann & Gramfort (2015) NIMG https://mne.tools/mne-r/ https://mne.tools/mne-hcp/ Jas, Engemann, et al. (2017) NIMG 

Thank you! Oleh Kozynets Guillaume Lemaître David Sabbagh Pierre Ablin Franz Liem Gaël Varoquaux Alexandre Gramfort Contact [email protected] www.denis-engemann.de github: @dengemann twitter: @dngman Danilo Bzdok Bertrand Thirion