OpenTalks.AI - Михаил Беляев, Deep Learning for Neuroimaging

Transcript

1. Deep Learning for Neuroimaging 07.02.18 Mikhail Belyaev Research Scientist Cobrain-Analytics Project, Skolkovo Institute of Science and Technology
2. Page 2 CoBrain project CoBrain-Analytics is a National Technological Initiative project. We are creating a platform for collecting and analysis of clinical neuroscience data. • Partners: leading russian medical centers, including ◦ Burdenko Neurosurgery Institute ◦ Scientific Center of Neurology ◦ And more • Data: ◦ More than 50000 patients with MRI / CT imaging ◦ Other modalities: сlinical data, genetics, EEG data. • Algorithms: specially developed algorithms for medical data analysis • Hardware: 2 PB of storage & heavy computational power
3. ➜ Introduction ➜ Medical images & CAD systems ➜ Typical data sets ➜ Deep Learning for Neuroimaging ➜ Architectures ➜ Prior information ➜ Augmentation ➜ Digital Pathology Images Page 3 Outline
4. Data: various types of Magnetic Resonance Imaging and Computer Tomography ➔structural MRI (like 3D image, each voxel contains a number), ➔diffusion MRI (each voxel contains a tensor), ➔functional MRI (time series of 3D images). Neuroimaging Data
5. Artificial Intelligence for Neuroimaging Key application areas: ➔ Automatic segmentation of medical images ➔ Identification of high-risk subjects in screening studies ➔ Predict treatment outcome (stroke, etc.) ➔ Medical Image Synthesis (MRI <-> CT) ➔ Early prediction of neurodegenerative disorders (Alzheimer's disease, Parkinson disease, Huntington disease, etc.) It’s all about Computer Aided Diagnosis (CAD) Page 5
6. First generation CAD: negative results Fenton, J. J., Taplin, S. H., Carney, P. A., ... & Elmore, J. G. (2007). Influence of computer-aided detection on performance of screening mammography. New England Journal of Medicine, 356(14), 1399-1409 • Diagnostic specificity decreased from 90.2% before implementation to 87.2% after implementation (P<0.001), • The rate of biopsy increased by 19.7% (P<0.001). • The increase in sensitivity from 80.4% to 84.0% was not significant (P=0.32). • Use of computer-aided detection was associated with significantly lower overall accuracy than was nonuse (area under the ROC curve, 0.871 vs. 0.919; P=0.005). The detection of small, invasive breast cancers also decreased
7. A set of rules to detect acute stroke in MRI images, ie: ➜ strokes are often one-sided; ➜ the subtle “graying” of tissue; ➜ a loss of anatomical borders Page 7 Rule-based CAD vs learning based ones How do radiologists find lesions?
8. Rule-based CAD vs learning based ones Melo, M., Scarpin, D. J., Amaro Jr, E., Passos, R. B., ..., & Price, C. J. (2011). How doctors generate diagnostic hypotheses: a study of radiological diagnosis with functional magnetic resonance imaging. PloS one, 6(12), e28752. • The mean response time for diagnosing lesions was 1.33 (SD ±0.14) seconds and 1.23 (SD ±0.13) seconds for naming animals. • The overall pattern of cortical activations was remarkably similar for both types of targets.
9. Rule-based CAD vs learning based ones Melo, M., Scarpin, D. J., Amaro Jr, E., Passos, R. B., ..., & Price, C. J. (2011). How doctors generate diagnostic hypotheses: a study of radiological diagnosis with functional magnetic resonance imaging. PloS one, 6(12), e28752. Radiologists learn how to recognize a lesion based on thousands of images!
10. Deep Learning, the key emerging AI technology, is based on the same principle. It uses large sets of annotated examples to automatically perform an analysis. One of the most discussed application of these technologies is medical image analysis [1]. 1. Mukherjee, Siddhartha A.I. Versus M.D. - The New Yorker, 2017. 2. First FDA Approval For Clinical Cloud-Based Deep Learning In Healthcare. Forbes. 2017. Page 10 Deep Learning for Medical Images - Why Now? Arterys, a startup company, received FDA approval for a deep learning based system in 2017 [2]. It was the first approval for such systems.
11. Non medical images: AlphaGo Page 11 The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. AlphaGo, a computer system, achieved a 99.8% winning rate against other Go programs, and defeated the human Go champion Silver, David, et al. "Mastering the game of Go with deep neural networks and tree search." Nature 529.7587 (2016): 484-489.
12. Non medical images: AlphaGo Page 12 They considered the board position as a 19×19 image 1. Predict the next move: a. Training data base: 30 millions positions from human games b. Play with the previous version of the algorithm to generate new positions 2. Build DL networks to predict a. the next turn b. game outcome 3. Use Monte Carlo tree search to find the best possible turn Silver, David, et al. "Mastering the game of Go with deep neural networks and tree search." Nature 529.7587 (2016): 484-489.
13. Non medical images: ImageNet 13 ImageNet 1k dataset: • 1.2 millions of images • 1000 of classes, ImageNet full • 14 millions of images, • >20k of classes Typical image resolution equals to 256x256 Page 13
14. Neuroimaging data 14 ImageNet • Data set size: 14 millions of images, • Image size: 256x256 A typical Neuroimaging dataset • Data set size: hundreds (up to a couple of thousands) • Image size: 150x150x150 for 1mm resolution Page 14
15. Neuroimaging data 15 ImageNet • Data set size: 14 millions of images, • Image size: 256x256 A typical Neuroimaging dataset • Data set size: hundreds (up to a couple of thousands) • Image size: 150x150x150 for 1mm resolution Page 15 How does Deep Learning work for NeuroImaging?
16. ➜ Introduction ➜ Medical images & CAD systems ➜ Typical data sets ➜ Deep Learning for Neuroimaging ➜ Architectures ➜ Prior information ➜ Augmentation ➜ Digital Pathology Images Page 16 Outline
17. Problem statements Page 17 • Semantic segmentation: ◦ Brain Lesions (multiple sclerosis, vascular diseases, cancer) ◦ Tracking of lesions dynamics • Classification ◦ Prediction of disease progression (e.g. Alzheimer disease) ◦ Quality control • Image retrieval • Image synthesis We’ll discuss semantic segmentation mainly
18. MRI Segmentation: an example Page 18 White Matter Hyperintensity (WMH) segmentation. Quantification of WMH volume, location and tracking the dynamics of these characteristics to support • diagnosis, • prognosis • monitoring of treatment for dementia and other neurodegenerative diseases.
19. Medical data normalization Page 19 Medical images are way more standardized than other images: • The target is usually located in the centre • There is a predefined number of their instances in each image There are two opposite strategies to deal with data variability: • decrease variability in all data by some data preprocessing or • use extensive data augmentation, increasing data variability and the size of the training set. The medical image computing community historically prefers preprocessing.
20. Page 20 MRI standardization There are several well-developed tools for neuroimaging, including Freesurfer, ANTs, FSL, AFNI. The following preprocessing steps are frequently used Bias field correction Brain extraction
21. Page 21 An example of our pipelines MRI Preprocessing steps: • Bias field correction • Skull Stripping • Coregistration of T1 and other series (T1c, T2, Flair; depends on a medical problem) Optional steps: • Resampling to isotropic resolution Deep learning steps: • Histogram-based data augmentation • A 3D convolutional network for segmentation
22. • Images intensities are highly variable • Careful preprocessing & direct usage of prior white matter masks allows to increase dice coefficient from 0.65 to 0.80 (white matter hyperintensity) Combination of preprocessing and Deep Learning
23. • Large 3D images -> high GPU memory consumption (batch size 2 isn’t something surprising) • High standardization -> localization matters • Simple approaches like DeepMedic works Deep Medic Architecture Kamnitsas, K., et al. (2017). Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical image analysis, 36, 61- 78.
24. • Classical data augmentation methods such as random rotations can break the alignment; too aggressive elastic deformations may lead to non-physiological shapes of a brain. • We propose a coregistration-based data augmentation method which preserves the normalization of medical images Data Augmentation MRI Augmentation via Elastic Registration for Brain Lesions Segmentation/ E. Krivov, M. Pisov, M. Belyaev. Brain Lesion Workshop at MICCAI 2017. 12 p.
25. Data Augmentation: Coregistration MRI Augmentation via Elastic Registration for Brain Lesions Segmentation/ E. Krivov, M. Pisov, M. Belyaev. Brain Lesion Workshop at MICCAI 2017. 12 p. To co-register two MRI, we heed to minimize L is a regularization functional which controls the smoothness of the transformation, λ is the regularization parameter Ω is image domain П~ is a measure of similarity of two images (e.g., mutual information), φ – is a set of transformations Augmentation boost Unet dice score from 0.43 to 0.51 (sub acute stroke lesion segmentation)
26. ➜ Introduction ➜ Medical images & CAD systems ➜ Typical data sets ➜ Deep Learning for Neuroimaging ➜ Architectures ➜ Prior information ➜ Augmentation ➜ Digital Pathology Images Page 26 Outline
27. Analysis (here we’ll talk about classification) of digital pathology images is an another type of medical images with similar problems • Sample size: hundreds / thousands • Whole slide image size: up to ~50000x50000 (up to 10 gigabytes per image!) Tissue type (e.g. normal, benign, specific cancerous subtype) Digital Pathology Images
28. Bach Challenge: • Data size: 2048 x 1536 • Image size: 400 (100 images per class) • Labels Normal, Benign, Carcinoma in situ, Invasive Carcinoma The simplest approach is to feed entire images into the network. However, due to large images’ shape, this approach yield poor results: Accuracy score*: 0.56 ± 0.06 *Calculated using 3-fold cross-validation for ResNet50 - one of the state-of-the art networks for images classification. Naive approach to microscopy images
29. Images are heterogeneous, but have “regular” cell structure. We can exploit this two features of the problem to build a better solution Statistics from probabilities distribution Final classific ation In Situ This yields a significantly better accuracy score: 0.90 ± 0.04. Path-based classification Ensembling Neural Networks for Digital Pathology Images Classification and Segmentation // G. Makarchuk, V. Kondratenko, M. Pisov, A. Pimkin, E. Krivov, M. Belyaev. arXiv1802.00947 (2018)
30. Conclusions • Medical image analysis problems are challenging, especially due to limited amount of data available • Deep learning still is a very powerful set of methods especially if data is carefully prepared and network architectures are carefully adopted
31. Conclusions • Medical image analysis problems are challenging, especially due to limited amount of data available • Deep learning still is a very powerful set of methods especially if data is carefully prepared and network architectures are carefully adopted Cobrain-Analytics project is looking for a business partners who wants to build products and services based on brain-related data analysis.
32. Thank you! Mikhail Belyaev m.belyaev@skoltech.ru