NUCLEI SEGMENTATION WITH CNNS Sanuj Sharma IIT - Guwahati With Surabhi Bhargava and Prof Amit Sethi Special Thanks to: Abhishek Vahadane and Neeraj Kumar 

Goal Detect and segment nuclei in medical images. IIT - Guwahati Touching, overlapping and occluded nuclei Motivation Useful features can be extracted which help in prediction of cancer occurrences like breast, prostate cancer: ● Number of cells ● Orientation, shape and arrangement of nuclei ● Density 

Nuclei image M-ary map IIT - Guwahati Different colors denote different nuclei. 

IIT - Guwahati Previous work Watershed, graph-cuts algorithms Not able to deal with diversity and variability in images 

IIT - Guwahati DATA ● CPCTR data set (prostate gland biopsies: 20x images) ● TDLU (breast biopsies : 20x images) ● REDUCE trial data set (prostate gland biopsies : 40x images) 

IIT - Guwahati 

IIT - Guwahati Annotations Used Aperio Imagescope to annotate ~5,000 nuclei. Original image Annotations 

IIT - Guwahati Color Normalization Target image Original image Normalized image Vahadane A, Peng T, Albarqouni S, Baust M, Steiger K, Schlitter AM, Sethi A, Esposito I, Navab N. Structure-preserved color normalization for histological images. In2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI) 2015 Apr 16 (pp. 1012-1015). IEEE. 

Supervised Learning with Convolutional Neural Networks IIT - Guwahati 

IIT - Guwahati Classify every pixel in the image into three classes: ○ Inside a nucleus Inside-nucleus ○ Inside boundary of a nucleus Nucleus boundary ○ Outside of a nucleus Non-nucleus Nucleus boundary Non-nucleus Non-nucleus Inside-nucleus Inside-nucleus Nucleus boundary Nucleus boundary Non-nucleus Window of size 51 x 51 to predict the class of the center pixel. IIT - Guwahati 

IIT - Guwahati Data Preparation Using Imagescope annotations Original image Ternary map 

IIT - Guwahati 

Probability maps of three classes generated by the CNN: Original image Nucleus boundary map Inside-nucleus map Outside-nucleus map White pixels denote high probability. IIT - Guwahati 

IIT - Guwahati Post-processing Original image Final color map 

Results IIT - Guwahati 

IIT - Guwahati Cooperative prostate cancer tissue resource (CPCTR) Original images Color maps 

IIT - Guwahati Overlapped color map 

IIT - Guwahati Terminal duct lobular unit (TDLU) Original image Color map 

IIT - Guwahati REDUCE trial Data-set Original image Color map 

IIT - Guwahati Future Possibilities ● Deep learning is the way to go! ● Residual neural networks with stochastic depth He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385. 2015 Dec 10. Huang G, Sun Y, Liu Z, Sedra D, Weinberger K. Deep networks with stochastic depth. arXiv preprint arXiv:1603.09382. 2016 Mar 30. 

IIT - Guwahati ● Treat it as video labeling problem - can use LSTMs, CNN+LSTM, 3-D convolution: Stollenga MF, Byeon W, Liwicki M, Schmidhuber J. Parallel multi-dimensional LSTM, with application to fast biomedical volumetric image segmentation. In Advances in Neural Information Processing Systems 2015 (pp. 2998-3006). 

IIT - Guwahati ● One shot learning. Santoro A, Bartunov S, Botvinick M, Wierstra D, Lillicrap T. One-shot Learning with Memory-Augmented Neural Networks. arXiv preprint arXiv:1605.06065. 2016 May 19.