Nuclei segmentation with CNNs

Transcript

1. NUCLEI SEGMENTATION WITH CNNS Sanuj Sharma IIT - Guwahati With Surabhi Bhargava and Prof Amit Sethi Special Thanks to: Abhishek Vahadane and Neeraj Kumar
2. Goal Detect and segment nuclei in medical images. IIT - Guwahati Touching, overlapping and occluded nuclei Motivation Useful features can be extracted which help in prediction of cancer occurrences like breast, prostate cancer: • Number of cells • Orientation, shape and arrangement of nuclei • Density
3. Nuclei image M-ary map IIT - Guwahati Different colors denote different nuclei.
4. IIT - Guwahati Previous work Watershed, graph-cuts algorithms Not able to deal with diversity and variability in images
5. IIT - Guwahati DATA • CPCTR data set (prostate gland biopsies: 20x images) • TDLU (breast biopsies : 20x images) • REDUCE trial data set (prostate gland biopsies : 40x images)
6. IIT - Guwahati
7. IIT - Guwahati Annotations Used Aperio Imagescope to annotate ~5,000 nuclei. Original image Annotations
8. IIT - Guwahati Color Normalization Target image Original image Normalized image Vahadane A, Peng T, Albarqouni S, Baust M, Steiger K, Schlitter AM, Sethi A, Esposito I, Navab N. Structure-preserved color normalization for histological images. In2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI) 2015 Apr 16 (pp. 1012-1015). IEEE.
9. Supervised Learning with Convolutional Neural Networks IIT - Guwahati
10. IIT - Guwahati Classify every pixel in the image into three classes: ◦ Inside a nucleus Inside-nucleus ◦ Inside boundary of a nucleus Nucleus boundary ◦ Outside of a nucleus Non-nucleus Nucleus boundary Non-nucleus Non-nucleus Inside-nucleus Inside-nucleus Nucleus boundary Nucleus boundary Non-nucleus Window of size 51 x 51 to predict the class of the center pixel. IIT - Guwahati
11. IIT - Guwahati Data Preparation Using Imagescope annotations Original image Ternary map
12. IIT - Guwahati
13. Probability maps of three classes generated by the CNN: Original image Nucleus boundary map Inside-nucleus map Outside-nucleus map White pixels denote high probability. IIT - Guwahati
14. IIT - Guwahati Post-processing Original image Final color map
15. Results IIT - Guwahati
16. IIT - Guwahati Cooperative prostate cancer tissue resource (CPCTR) Original images Color maps
17. IIT - Guwahati Overlapped color map
18. IIT - Guwahati Terminal duct lobular unit (TDLU) Original image Color map
19. IIT - Guwahati REDUCE trial Data-set Original image Color map
20. IIT - Guwahati Future Possibilities • Deep learning is the way to go! • Residual neural networks with stochastic depth He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385. 2015 Dec 10. Huang G, Sun Y, Liu Z, Sedra D, Weinberger K. Deep networks with stochastic depth. arXiv preprint arXiv:1603.09382. 2016 Mar 30.
21. IIT - Guwahati • Treat it as video labeling problem - can use LSTMs, CNN+LSTM, 3-D convolution: Stollenga MF, Byeon W, Liwicki M, Schmidhuber J. Parallel multi-dimensional LSTM, with application to fast biomedical volumetric image segmentation. In Advances in Neural Information Processing Systems 2015 (pp. 2998-3006).
22. IIT - Guwahati • One shot learning. Santoro A, Bartunov S, Botvinick M, Wierstra D, Lillicrap T. One-shot Learning with Memory-Augmented Neural Networks. arXiv preprint arXiv:1605.06065. 2016 May 19.