Share
Embed
Understanding Geometry of Encoder-Decoder CNNs for Inverse Problems
Plenary Talk, Applied Inverse Problems (AIP) conference, Grenoble, July 11th, 2019
Jong Chul Ye
July 11, 2019
Transcript
102 slides
-
Understanding Geometry of Encoder Decoder CNN for Inverse Problems Jong
Chul Ye, Ph.D Endowed Chair Professor BISPL - BioImaging, Signal Processing, and Learning lab. Dept. Bio & Brain Engineering Dept. Mathematical Sciences KAIST, Korea -
Deep Learning Revolution • Deep learning has been successfully used
for classification, low- level computer vision, etc • Even outperforms human observers Figure modified from Kaiming He’s presentation -
Classical Learning vs Deep Learning Diagnosis Classical machine learning Deep
learning (no feature engineering) Feature Engineering Esteva et al, Nature Medicine, (2019) -
Deep Learning Era in Medical Imaging Diabetic eye diagnosis Gulshan,
V. et al. JAMA (2016) Skin Cancer diagnosis Esteva et al, Nature (2017) OCT diagnosis De Fauw et al, Nature Medicine (2018) Figure courtesy of X. Cao & D. Shen Image registration Image segmentation Ronneberger et al, MICCAI, 2015 -
Deep Learning for Inverse Problems Diagnosis Diagnosis & analysis Focus
of this talk: Reconstruction -
7 Wavelet transform level 2 level 1 level 3 level
4 Wavelet recomposition + Residual learning : Low-resolution image bypass High SNR band CNN (Kang, et al, Medical Physics 44(10)) -
Figures from internet Extreme Sparse View CT Stationary CT Carry
on baggage scanner Han et al, arXiv preprint arXiv:1712.10248, (2017); CT Meeting (2017) -
1st view 2nd view 3rd view 4th view 5th view
6th view 7th view 8th view 9th view -
Deep Learning Pioneers in MR Kwon et al, Medical Physics,
2017 Hammernik et al, MRM, 2018 Wang et al, ISBI, 2016 Yang et al, NIPS, 2016 Multilayer perceptron Variational network Deep learning prior ADMM-Net -
Variational Network (R=4) CG SENSE PI-CS: TGV Learning: VN PI
PI-CS Learning Hammernik MRM 2018 Courtesy of Florian Knoll -
Hybrid Domain Learning Deep Cascade of CNNs for MRI Reconstruction
Schlemper et al. IEEE TMI 2017 Courtesy of D. Rueckert (a) 11x Undersampled (b) CNN reconstruction (c) Ground Truth -
Sensor-domain Learning CNN k-space deep learning Han et al, IEEE
TMI; Lee et al, MRM (in press, 2019) -
Why so popular this time ? q Accuracy: high quality
recon > CS q Fast reconstruction time q Business model: vendor-driven training q Interpretable models q Flexibility: more than recon Imaging time Reconstruction time Conventional Compressed Sensing Machine Learning -
Too Simple to Analyze..? Convolution & pooling à stone age
tools of signal processing What do they do ? -
WHY DEEP LEARNING WORKS FOR RECON ? DOES IT CREATE
ANY ARTIFICIAL FEATURES ? -
Ye et al, SIAM J. Imaging Sciences, 2018; Ye et
al, ICML, 2019 Understanding Geometry of CNN -
Classical Methods for Inverse Problems Synthesis frame Analysis frame coefficients
Step 1: Signal Representation x = X i hbi, xi˜ bi <latexit sha1_base64="1xOFabKTay95z7c6ZUXCSJWXZAE=">AAACE3icbVDLSsNAFJ3UV62vqEs3g0UQkZKooBuh4MZlC/YBTQiTyaQdOpmEmYm0hIKf4MZfceNCEbdu3Pk3TpMutPXAhTPn3Mvce/yEUaks69soLS2vrK6V1ysbm1vbO+buXlvGqcCkhWMWi66PJGGUk5aiipFuIgiKfEY6/vBm6nfuiZA05ndqnBA3Qn1OQ4qR0pJnnoyuHZlGHoXQYYj3GYG+R0/hyBHFy1GUBbnomVWrZuWAi8SekSqYoeGZX04Q4zQiXGGGpOzZVqLcDAlFMSOTipNKkiA8RH3S05SjiEg3y2+awCOtBDCMhS6uYK7+nshQJOU48nVnhNRAzntT8T+vl6rwys0oT1JFOC4+ClMGVQynAcGACoIVG2uCsKB6V4gHSCCsdIwVHYI9f/IiaZ/V7POa1byo1psPRRxlcAAOwTGwwSWog1vQAC2AwSN4Bq/gzXgyXox346NoLRmzCPfBHxifP6mnndg=</latexit> -
Eg. Compressed Sensing Classical Methods for Inverse Problems Step 2:
Basis Search by Optimization x = X i ˜ bi hbi, xi <latexit sha1_base64="dRyoeK2luEuIb3X8ywuvlvflFPU=">AAACEnicbZBNS8MwGMdTX+d8q3r0EhyCgoxWBb0IQy8eJ7gXWEtJ03QLS9OSpLJR9hm8+FW8eFDEqydvfhvTrgfd/EPgl//zPCTP308Ylcqyvo2FxaXlldXKWnV9Y3Nr29zZbcs4FZi0cMxi0fWRJIxy0lJUMdJNBEGRz0jHH97k9c4DEZLG/F6NE+JGqM9pSDFS2vLM49GVI9PIo9BRlAUE+jkyxPus4BM4ckRx88yaVbcKwXmwS6iBUk3P/HKCGKcR4QozJGXPthLlZkgoihmZVJ1UkgThIeqTnkaOIiLdrFhpAg+1E8AwFvpwBQv390SGIinHka87I6QGcraWm//VeqkKL92M8iRVhOPpQ2HKoIphng8MqCBYsbEGhAXVf4V4gATCSqdY1SHYsyvPQ/u0bp/VrbvzWuO6jKMC9sEBOAI2uAANcAuaoAUweATP4BW8GU/Gi/FufExbF4xyZg/8kfH5AyCNnR8=</latexit> -
Our Theoretical Findings y = X i hbi(x), xi˜ bi(x)
<latexit sha1_base64="DaaFmbtzayW3V2tBvW3rbADydJY=">AAACGXicbZDLSsNAFIYnXmu9RV26GSxCBSmJCroRim5cVrAXaEKYTCbt0MkkzEykIfQ13Pgqblwo4lJXvo3TNoK2/jDw851zOHN+P2FUKsv6MhYWl5ZXVktr5fWNza1tc2e3JeNUYNLEMYtFx0eSMMpJU1HFSCcRBEU+I21/cD2ut++JkDTmdypLiBuhHqchxUhp5JlWdunINPIohA5DvMcI9D1aHR4dw6EjpsBRlAU/3DMrVs2aCM4buzAVUKjhmR9OEOM0IlxhhqTs2lai3BwJRTEjo7KTSpIgPEA90tWWo4hIN59cNoKHmgQwjIV+XMEJ/T2Ro0jKLPJ1Z4RUX87WxvC/WjdV4YWbU56kinA8XRSmDKoYjmOCARUEK5Zpg7Cg+q8Q95FAWOkwyzoEe/bkedM6qdmnNev2rFK/KuIogX1wAKrABuegDm5AAzQBBg/gCbyAV+PReDbejPdp64JRzOyBPzI+vwEaXJ8Y</latexit> Ye et al, SIIMS, 2018; Ye et al, ICML, 2019 -
y = X i hbi(x), xi˜ bi(x) <latexit sha1_base64="DaaFmbtzayW3V2tBvW3rbADydJY=">AAACGXicbZDLSsNAFIYnXmu9RV26GSxCBSmJCroRim5cVrAXaEKYTCbt0MkkzEykIfQ13Pgqblwo4lJXvo3TNoK2/jDw851zOHN+P2FUKsv6MhYWl5ZXVktr5fWNza1tc2e3JeNUYNLEMYtFx0eSMMpJU1HFSCcRBEU+I21/cD2ut++JkDTmdypLiBuhHqchxUhp5JlWdunINPIohA5DvMcI9D1aHR4dw6EjpsBRlAU/3DMrVs2aCM4buzAVUKjhmR9OEOM0IlxhhqTs2lai3BwJRTEjo7KTSpIgPEA90tWWo4hIN59cNoKHmgQwjIV+XMEJ/T2Ro0jKLPJ1Z4RUX87WxvC/WjdV4YWbU56kinA8XRSmDKoYjmOCARUEK5Zpg7Cg+q8Q95FAWOkwyzoEe/bkedM6qdmnNev2rFK/KuIogX1wAKrABuegDm5AAzQBBg/gCbyAV+PReDbejPdp64JRzOyBPzI+vwEaXJ8Y</latexit> Our
Theoretical Findings Ye et al, SIIMS, 2018; Ye et al, ICML, 2019 -
y = X i hbi(x), xi˜ bi(x) <latexit sha1_base64="DaaFmbtzayW3V2tBvW3rbADydJY=">AAACGXicbZDLSsNAFIYnXmu9RV26GSxCBSmJCroRim5cVrAXaEKYTCbt0MkkzEykIfQ13Pgqblwo4lJXvo3TNoK2/jDw851zOHN+P2FUKsv6MhYWl5ZXVktr5fWNza1tc2e3JeNUYNLEMYtFx0eSMMpJU1HFSCcRBEU+I21/cD2ut++JkDTmdypLiBuhHqchxUhp5JlWdunINPIohA5DvMcI9D1aHR4dw6EjpsBRlAU/3DMrVs2aCM4buzAVUKjhmR9OEOM0IlxhhqTs2lai3BwJRTEjo7KTSpIgPEA90tWWo4hIN59cNoKHmgQwjIV+XMEJ/T2Ro0jKLPJ1Z4RUX87WxvC/WjdV4YWbU56kinA8XRSmDKoYjmOCARUEK5Zpg7Cg+q8Q95FAWOkwyzoEe/bkedM6qdmnNev2rFK/KuIogX1wAKrABuegDm5AAzQBBg/gCbyAV+PReDbejPdp64JRzOyBPzI+vwEaXJ8Y</latexit> Our
Theoretical Findings Ye et al, SIIMS, 2018; Ye et al, ICML, 2019 -
analysis basis y = X i hbi(x), xi˜ bi(x) <latexit
sha1_base64="DaaFmbtzayW3V2tBvW3rbADydJY=">AAACGXicbZDLSsNAFIYnXmu9RV26GSxCBSmJCroRim5cVrAXaEKYTCbt0MkkzEykIfQ13Pgqblwo4lJXvo3TNoK2/jDw851zOHN+P2FUKsv6MhYWl5ZXVktr5fWNza1tc2e3JeNUYNLEMYtFx0eSMMpJU1HFSCcRBEU+I21/cD2ut++JkDTmdypLiBuhHqchxUhp5JlWdunINPIohA5DvMcI9D1aHR4dw6EjpsBRlAU/3DMrVs2aCM4buzAVUKjhmR9OEOM0IlxhhqTs2lai3BwJRTEjo7KTSpIgPEA90tWWo4hIN59cNoKHmgQwjIV+XMEJ/T2Ro0jKLPJ1Z4RUX87WxvC/WjdV4YWbU56kinA8XRSmDKoYjmOCARUEK5Zpg7Cg+q8Q95FAWOkwyzoEe/bkedM6qdmnNev2rFK/KuIogX1wAKrABuegDm5AAzQBBg/gCbyAV+PReDbejPdp64JRzOyBPzI+vwEaXJ8Y</latexit> Encoder Our Theoretical Findings Ye et al, SIIMS, 2018; Ye et al, ICML, 2019 -
analysis basis synthesis basis y = X i hbi(x), xi˜
bi(x) <latexit sha1_base64="DaaFmbtzayW3V2tBvW3rbADydJY=">AAACGXicbZDLSsNAFIYnXmu9RV26GSxCBSmJCroRim5cVrAXaEKYTCbt0MkkzEykIfQ13Pgqblwo4lJXvo3TNoK2/jDw851zOHN+P2FUKsv6MhYWl5ZXVktr5fWNza1tc2e3JeNUYNLEMYtFx0eSMMpJU1HFSCcRBEU+I21/cD2ut++JkDTmdypLiBuhHqchxUhp5JlWdunINPIohA5DvMcI9D1aHR4dw6EjpsBRlAU/3DMrVs2aCM4buzAVUKjhmR9OEOM0IlxhhqTs2lai3BwJRTEjo7KTSpIgPEA90tWWo4hIN59cNoKHmgQwjIV+XMEJ/T2Ro0jKLPJ1Z4RUX87WxvC/WjdV4YWbU56kinA8XRSmDKoYjmOCARUEK5Zpg7Cg+q8Q95FAWOkwyzoEe/bkedM6qdmnNev2rFK/KuIogX1wAKrABuegDm5AAzQBBg/gCbyAV+PReDbejPdp64JRzOyBPzI+vwEaXJ8Y</latexit> Encoder Decoder Our Theoretical Findings Ye et al, SIIMS, 2018; Ye et al, ICML, 2019 -
Linear Encoder-Decoder (ED) CNN Learned filters y = ˜ BB>x
= X i hx, bi i˜ bi <latexit sha1_base64="bo3reUJLRRRgiLys4OrWvNpVArY=">AAACJ3icbVBNSwMxEM36bf2qevQSLIIHKbsq6KUievGoaK3QrSWbzrah2eySzIpl8d948a94EVREj/4T03YP2joQePPePCbzgkQKg6775UxMTk3PzM7NFxYWl5ZXiqtr1yZONYcqj2WsbwJmQAoFVRQo4SbRwKJAQi3onvb12h1oI2J1hb0EGhFrKxEKztBSzeJRr+KjkC2gJ/Tk1sc4ofe0Qn2TRk1BfclUWwK936FBv9XDNndYqlksuWV3UHQceDkokbzOm8VXvxXzNAKFXDJj6p6bYCNjGgWX8FDwUwMJ413WhrqFikVgGtngzge6ZZkWDWNtn0I6YH87MhYZ04sCOxkx7JhRrU/+p9VTDA8bmVBJiqD4cFGYSoox7YdGW0IDR9mzgHEt7F8p7zDNONpoCzYEb/TkcXC9W/b2yu7Ffun4Mo9jjmyQTbJNPHJAjskZOSdVwskjeSZv5N15cl6cD+dzODrh5J518qec7x9ZY6R3</latexit> pooling un-pooling -
Linear E-D CNN w/ Skipped Connection more redundant expression Learned
filters y = ˜ BB>x = X i hx, bi i˜ bi <latexit sha1_base64="bo3reUJLRRRgiLys4OrWvNpVArY=">AAACJ3icbVBNSwMxEM36bf2qevQSLIIHKbsq6KUievGoaK3QrSWbzrah2eySzIpl8d948a94EVREj/4T03YP2joQePPePCbzgkQKg6775UxMTk3PzM7NFxYWl5ZXiqtr1yZONYcqj2WsbwJmQAoFVRQo4SbRwKJAQi3onvb12h1oI2J1hb0EGhFrKxEKztBSzeJRr+KjkC2gJ/Tk1sc4ofe0Qn2TRk1BfclUWwK936FBv9XDNndYqlksuWV3UHQceDkokbzOm8VXvxXzNAKFXDJj6p6bYCNjGgWX8FDwUwMJ413WhrqFikVgGtngzge6ZZkWDWNtn0I6YH87MhYZ04sCOxkx7JhRrU/+p9VTDA8bmVBJiqD4cFGYSoox7YdGW0IDR9mzgHEt7F8p7zDNONpoCzYEb/TkcXC9W/b2yu7Ffun4Mo9jjmyQTbJNPHJAjskZOSdVwskjeSZv5N15cl6cD+dzODrh5J518qec7x9ZY6R3</latexit> -
Deep Convolutional Framelets x = ˜ BB>x = X i
hx, bi i˜ bi <latexit sha1_base64="9EuOyjKGC2x9hAgBpajvIdywLlA=">AAACJ3icbVBNSwMxEM36bf2qevQSLIIHKbsq6EWRevGoaK3QXZdsOq2h2eySzEpL6b/x4l/xIqiIHv0npu0etDoQePPePCbzolQKg6776UxMTk3PzM7NFxYWl5ZXiqtr1ybJNIcqT2SibyJmQAoFVRQo4SbVwOJIQi1qnw702j1oIxJ1hd0Ugpi1lGgKztBSYfG4c+SjkA2gFVq59TFJaYceUd9kcSioL5lqSaCdHRoNWj1qc4elwmLJLbvDon+Bl4MSyes8LL74jYRnMSjkkhlT99wUgx7TKLiEfsHPDKSMt1kL6hYqFoMJesM7+3TLMg3aTLR9CumQ/enosdiYbhzZyZjhnRnXBuR/Wj3D5mHQEyrNEBQfLWpmkmJCB6HRhtDAUXYtYFwL+1fK75hmHG20BRuCN37yX3C9W/b2yu7FfunkMo9jjmyQTbJNPHJATsgZOSdVwskDeSKv5M15dJ6dd+djNDrh5J518qucr29XoqR2</latexit> Perfect reconstruction Ye et al, SIIMS 2018; Ye et al, ICML 2019 Frame conditions w skipped connection w/o skipped connection -
Deep Convolutional Framelets x = ˜ BB>x = X i
hx, bi i˜ bi <latexit sha1_base64="9EuOyjKGC2x9hAgBpajvIdywLlA=">AAACJ3icbVBNSwMxEM36bf2qevQSLIIHKbsq6EWRevGoaK3QXZdsOq2h2eySzEpL6b/x4l/xIqiIHv0npu0etDoQePPePCbzolQKg6776UxMTk3PzM7NFxYWl5ZXiqtr1ybJNIcqT2SibyJmQAoFVRQo4SbVwOJIQi1qnw702j1oIxJ1hd0Ugpi1lGgKztBSYfG4c+SjkA2gFVq59TFJaYceUd9kcSioL5lqSaCdHRoNWj1qc4elwmLJLbvDon+Bl4MSyes8LL74jYRnMSjkkhlT99wUgx7TKLiEfsHPDKSMt1kL6hYqFoMJesM7+3TLMg3aTLR9CumQ/enosdiYbhzZyZjhnRnXBuR/Wj3D5mHQEyrNEBQfLWpmkmJCB6HRhtDAUXYtYFwL+1fK75hmHG20BRuCN37yX3C9W/b2yu7FfunkMo9jjmyQTbJNPHJATsgZOSdVwskDeSKv5M15dJ6dd+djNDrh5J518qucr29XoqR2</latexit> Perfect reconstruction Ye et al, SIAM J. Imaging Science, 2018 Frame conditions w skipped connection w/o skipped connection -
Role of ReLUs? Generator for Multiple Expressions y = ˜
B(x)B(x)>x = X i hx, bi(x)i˜ bi(x) <latexit sha1_base64="T/1m1u26m8O8vLHErH3u6EKQhAM=">AAACM3icbVDLSgMxFM3Ud31VXboJFkFByowKuhGKbsSVotVCpw6Z9LYNZjJDckdaSv/JjT/iQhAXirj1H0zbWaj1QsLhPEjuCRMpDLrui5ObmJyanpmdy88vLC4tF1ZWr02cag4VHstYV0NmQAoFFRQooZpoYFEo4Sa8OxnoN/egjYjVFXYTqEespURTcIaWCgpn3SMfhWwAPd7qbA+vWx/jhHboEfVNGgWC+pKplgTa2aFhIAY2X4+YLDpig0LRLbnDoePAy0CRZHMeFJ78RszTCBRyyYypeW6C9R7TKLiEft5PDSSM37EW1CxULAJT7w137tNNyzRoM9b2KKRD9meixyJjulFonRHDtvmrDcj/tFqKzcN6T6gkRVB89FAzlRRjOiiQNoQGjrJrAeNa2L9S3maacbQ1520J3t+Vx8H1bsnbK7kX+8XyZVbHLFknG2SLeOSAlMkpOScVwskDeSZv5N15dF6dD+dzZM05WWaN/Brn6xvQsKgT</latexit> ⌃l(x) = 2 6 6 6 4 1 0 · · · 0 0 2 · · · 0 . . . . . . ... . . . 0 0 · · · ml 3 7 7 7 5 <latexit sha1_base64="1HHS4n8UkvGQcnzeL2YdPrnnXeg=">AAACkXicbVHLSgMxFM2M7/qqunQTLIpuyowK6kIpuhHcKFoVmjpkMrdtMMkMSUYsQ//H73Hn35i2g4/qhQvnnnNfuYkzwY0Ngg/Pn5qemZ2bX6gsLi2vrFbX1u9NmmsGTZaKVD/G1IDgCpqWWwGPmQYqYwEP8fPFUH94AW14qu5sP4O2pF3FO5xR66io+kZueVfSJ7H7uodPMYmhy1URS2o1fx1gYoZqFOIdHDgnLEmtGQWEjJlxwv6kWCEvZfQNkt+MSwom+5btChkJNxxU8rVKVK0F9WBk+C8IS1BDpV1H1XeSpCyXoCwT1JhWGGS2XVBtORMwqJDcQEbZM+1Cy0FFJZh2MbroAG87JsGdVDtXFo/YnxUFlcb0Zewy3X49M6kNyf+0Vm47x+2Cqyy3oNh4UCcX2KZ4+D044RqYFX0HKNPc7YpZj2rKrPvEijtCOPnkv+B+vx4e1IObw1rjvDzHPNpEW2gXhegINdAlukZNxLwV79A79c78Df/Eb/hlru+VNRvol/lXn1/Lv9k=</latexit> Input dependent {0,1} matrix --> Input adaptivity -
Input Space Partitioning for Multiple Expressions A CNN performs automatic
assignment of distinct linear representation depending on input -
Expressivity of E-D CNN # of representation # of network
elements # of channel -
Expressivity of E-D CNN # of representation # of network
elements # of channel Network depth -
Expressivity of E-D CNN # of representation # of network
elements # of channel Network depth Skipped connection -
Lipschitz Continuity K = max p Kp, Kp = k
˜ B(zp)B(zp)>k2 <latexit sha1_base64="zV0QFc8bcwR20HLOVcDQeQMOtmY=">AAACIHicbZDLSgMxFIYz9V5voy7dBItQQcpMFeqmUOpGcKNgbaFTh0wmtaGZmZicEWv1Udz4Km5cKKI7fRrTy0KtBxI+/v8ckvMHUnANjvNpZaamZ2bn5heyi0vLK6v22vq5TlJFWY0mIlGNgGgmeMxqwEGwhlSMRIFg9aB7OPDr10xpnsRn0JOsFZHLmLc5JWAk3y4dl72I3PgSH/ty17tKSTggXMbenQdchAxX87e+3BndFx4k0rvzi76dcwrOsPAkuGPIoXGd+PaHFyY0jVgMVBCtm64jodUnCjgV7D7rpZpJQrvkkjUNxiRiutUfLniPt40S4naizIkBD9WfE30Sad2LAtMZEejov95A/M9rptA+aPV5LFNgMR091E4FhgQP0sIhV4yC6BkgVHHzV0w7RBEKJtOsCcH9u/IknBcL7l7BOd3PVarjOObRJtpCeeSiEqqgI3SCaoiiB/SEXtCr9Wg9W2/W+6g1Y41nNtCvsr6+AcwXoYQ=</latexit> z1 <latexit sha1_base64="Ob3+IEXFhF5uWyRIGKNYQ89lNRY=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbTbt0swm7E6GG/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvSKQw6LrfTmFldW19o7hZ2tre2d0r7x80TZxqxhsslrFuB9RwKRRvoEDJ24nmNAokbwWjm6nfeuTaiFg94DjhfkQHSoSCUbTS/VPP65UrbtWdgSwTLycVyFHvlb+6/ZilEVfIJDWm47kJ+hnVKJjkk1I3NTyhbEQHvGOpohE3fjY7dUJOrNInYaxtKSQz9fdERiNjxlFgOyOKQ7PoTcX/vE6K4ZWfCZWkyBWbLwpTSTAm079JX2jOUI4toUwLeythQ6opQ5tOyYbgLb68TJpnVe+86t5dVGrXeRxFOIJjOAUPLqEGt1CHBjAYwDO8wpsjnRfn3fmYtxacfOYQ/sD5/AEPZo2k</latexit> zp <latexit sha1_base64="Q3WIlMLDjf+qfP58xUVUuKL5KD4=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbSbt0swm7G6GG/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvSATXxnW/ncLK6tr6RnGztLW9s7tX3j9o6jhVDBssFrFqB1Sj4BIbhhuB7UQhjQKBrWB0M/Vbj6g0j+WDGSfoR3QgecgZNVa6f+olvXLFrbozkGXi5aQCOeq98le3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9mpE3JilT4JY2VLGjJTf09kNNJ6HAW2M6JmqBe9qfif10lNeOVnXCapQcnmi8JUEBOT6d+kzxUyI8aWUKa4vZWwIVWUGZtOyYbgLb68TJpnVe+86t5dVGrXeRxFOIJjOAUPLqEGt1CHBjAYwDO8wpsjnBfn3fmYtxacfOYQ/sD5/AFu4o3j</latexit> Related to the generalizability Dependent on the Local Lipschitz -
Optimization Issue • Optimization Landscape • All local minimizers are
global minimizers • Ex) overparameterized network • Implicit bias • Specific global minimizers are determine d by the optimization algorithms Q) Nonconvex optimization problem à Can we prove convergence ? o <latexit sha1_base64="4DjPcWEGh5PQIKL0eIUnsOTwihE=">AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwdZiO5RMmmlD8xiSjFCG/oUbF4q49W/c+Tdm2llo64HA4Zx7ybknSjgz1ve/vdLK6tr6RnmzsrW9s7tX3T9oG5VqQltEcaU7ETaUM0lblllOO4mmWEScPkTjm9x/eKLaMCXv7SShocBDyWJGsHXSY09gO4riTE371Zpf92dAyyQoSA0KNPvVr95AkVRQaQnHxnQDP7FhhrVlhNNppZcammAyxkPadVRiQU2YzRJP0YlTBihW2j1p0Uz9vZFhYcxERG4yT2gWvVz8z+umNr4KMyaT1FJJ5h/FKUdWofx8NGCaEssnjmCimcuKyAhrTKwrqeJKCBZPXibts3pwXvfvLmqN66KOMhzBMZxCAJfQgFtoQgsISHiGV3jzjPfivXsf89GSV+wcwh94nz/vApER</latexit> x <latexit sha1_base64="hUTvfx5dXOHFo5uS6vfKoPsIiQ8=">AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg==</latexit> o <latexit sha1_base64="4DjPcWEGh5PQIKL0eIUnsOTwihE=">AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwdZiO5RMmmlD8xiSjFCG/oUbF4q49W/c+Tdm2llo64HA4Zx7ybknSjgz1ve/vdLK6tr6RnmzsrW9s7tX3T9oG5VqQltEcaU7ETaUM0lblllOO4mmWEScPkTjm9x/eKLaMCXv7SShocBDyWJGsHXSY09gO4riTE371Zpf92dAyyQoSA0KNPvVr95AkVRQaQnHxnQDP7FhhrVlhNNppZcammAyxkPadVRiQU2YzRJP0YlTBihW2j1p0Uz9vZFhYcxERG4yT2gWvVz8z+umNr4KMyaT1FJJ5h/FKUdWofx8NGCaEssnjmCimcuKyAhrTKwrqeJKCBZPXibts3pwXvfvLmqN66KOMhzBMZxCAJfQgFtoQgsISHiGV3jzjPfivXsf89GSV+wcwh94nz/vApER</latexit> o <latexit sha1_base64="4DjPcWEGh5PQIKL0eIUnsOTwihE=">AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwdZiO5RMmmlD8xiSjFCG/oUbF4q49W/c+Tdm2llo64HA4Zx7ybknSjgz1ve/vdLK6tr6RnmzsrW9s7tX3T9oG5VqQltEcaU7ETaUM0lblllOO4mmWEScPkTjm9x/eKLaMCXv7SShocBDyWJGsHXSY09gO4riTE371Zpf92dAyyQoSA0KNPvVr95AkVRQaQnHxnQDP7FhhrVlhNNppZcammAyxkPadVRiQU2YzRJP0YlTBihW2j1p0Uz9vZFhYcxERG4yT2gWvVz8z+umNr4KMyaT1FJJ5h/FKUdWofx8NGCaEssnjmCimcuKyAhrTKwrqeJKCBZPXibts3pwXvfvLmqN66KOMhzBMZxCAJfQgFtoQgsISHiGV3jzjPfivXsf89GSV+wcwh94nz/vApER</latexit> x <latexit sha1_base64="hUTvfx5dXOHFo5uS6vfKoPsIiQ8=">AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg==</latexit> o <latexit sha1_base64="4DjPcWEGh5PQIKL0eIUnsOTwihE=">AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwdZiO5RMmmlD8xiSjFCG/oUbF4q49W/c+Tdm2llo64HA4Zx7ybknSjgz1ve/vdLK6tr6RnmzsrW9s7tX3T9oG5VqQltEcaU7ETaUM0lblllOO4mmWEScPkTjm9x/eKLaMCXv7SShocBDyWJGsHXSY09gO4riTE371Zpf92dAyyQoSA0KNPvVr95AkVRQaQnHxnQDP7FhhrVlhNNppZcammAyxkPadVRiQU2YzRJP0YlTBihW2j1p0Uz9vZFhYcxERG4yT2gWvVz8z+umNr4KMyaT1FJJ5h/FKUdWofx8NGCaEssnjmCimcuKyAhrTKwrqeJKCBZPXibts3pwXvfvLmqN66KOMhzBMZxCAJfQgFtoQgsISHiGV3jzjPfivXsf89GSV+wcwh94nz/vApER</latexit> x <latexit sha1_base64="hUTvfx5dXOHFo5uS6vfKoPsIiQ8=">AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg==</latexit> o <latexit sha1_base64="4DjPcWEGh5PQIKL0eIUnsOTwihE=">AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwdZiO5RMmmlD8xiSjFCG/oUbF4q49W/c+Tdm2llo64HA4Zx7ybknSjgz1ve/vdLK6tr6RnmzsrW9s7tX3T9oG5VqQltEcaU7ETaUM0lblllOO4mmWEScPkTjm9x/eKLaMCXv7SShocBDyWJGsHXSY09gO4riTE371Zpf92dAyyQoSA0KNPvVr95AkVRQaQnHxnQDP7FhhrVlhNNppZcammAyxkPadVRiQU2YzRJP0YlTBihW2j1p0Uz9vZFhYcxERG4yT2gWvVz8z+umNr4KMyaT1FJJ5h/FKUdWofx8NGCaEssnjmCimcuKyAhrTKwrqeJKCBZPXibts3pwXvfvLmqN66KOMhzBMZxCAJfQgFtoQgsISHiGV3jzjPfivXsf89GSV+wcwh94nz/vApER</latexit> x <latexit sha1_base64="hUTvfx5dXOHFo5uS6vfKoPsIiQ8=">AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg==</latexit> x <latexit sha1_base64="hUTvfx5dXOHFo5uS6vfKoPsIiQ8=">AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg==</latexit> x <latexit sha1_base64="hUTvfx5dXOHFo5uS6vfKoPsIiQ8=">AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg==</latexit> -
Benign Optimization Landscape full-rank condition Independent features Independent features full-rank
condition Nguyen, et al, ICML, 2018 Ye et al, ICML, 2019 -
Regularized Recon vs. Deep Recon Diagnosis Classical Regularized Recon (basis
engineering) Deep Recon (no basis engineering) Basis Engineering -
Focused / Plane Wave Ultrasound Imaging Focused Imaging Plane Wave
Imaging Couade M, JVDI, 2015 Yoon et al, TMI, 2018 -
Adaptive Beamformer Conventional Beamforming Pipeline I Q =
zl[n] H(zl)[n] <latexit sha1_base64="Mv7jnJQUjhxiLpE3LDXNSJ47/J8=">AAACSnicbZBPSwMxEMWztWpd/6169BIsQr2UXRX0IhS91JuCtYXuUrLptA3NZpckK9aln8+LJ29+CC8eFPFiWovY1oHA4zdvmMkLE86Udt0XK7eQX1xaLqzYq2vrG5vO1vatilNJoUZjHstGSBRwJqCmmebQSCSQKORQD/sXo379DqRisbjRgwSCiHQF6zBKtEEth/ghdJnIwohoye6Hl9j38TX2QbR/GT7DMy77ocWbIjBe2zeoRwnPqsOSoQcG21PTLafolt1x4XnhTUQRTeqq5Tz77ZimEQhNOVGq6bmJDjIiNaMchrafKkgI7ZMuNI0UJAIVZOMohnjfkDbuxNI8ofGY/p3ISKTUIAqNc3S4mu2N4H+9Zqo7p0HGRJJqEPRnUSflWMd4lCtuMwlU84ERhEpmbsW0RySh2qRvmxC82S/Pi9vDsndUdq+Pi5XzSRwFtIv2UAl56ARVUBVdoRqi6BG9onf0YT1Zb9an9fVjzVmTmR00Vbn8N6l4s94=</latexit> Data specific weights -
Deep Beamformer Khan et al, MICCAI (accepted), 2019 Adaptive and
Compressive Deep Beamformer Conventional Beamforming Pipeline Input partitioning à Data specific representation -
k-Space Deep Learning Han et al, IEEE Trans. Medical Imaging
(in press), 2019 Lee et al, MRM (in press), 2019 -
Duality between Sparsity and Low-Rankness Ye et al, IEEE Trans.
Information Theory, 2017;Jin et al, IEEE TCI, 2017; Lee et al, MRM, 2017 -
Annihilating Filter-based Low-rank Hankel Matrix Approach (ALOHA) Jin KH et
al IEEE TCI, 2016, Lee et al, MRM, 2015; Ye et al, IEEE TIT, 2016 -
Single coil MRI Jin KH et al IEEE TCI, 2016
Lee et al, MRM, 2015 -
Parallel MRI Jin KH et al IEEE TCI, 2016 Lee
et al, MRM, 2015 -
Optimization Problem Equivalent to search for frame basis to make
the nonzero-coefficents sparse -
: Non-local basis : Local basis Convolution Framelets (Yin et
al, SIIMS, 2017) > = I > = I -
: Non-local basis : Local basis Convolution Framelets (Yin et
al, SIIMS, 2017) > = I > = I -
: Non-local basis : Local basis Convolution Framelets (Yin et
al, SIIMS, 2017) > = I > = I -
˜ T = I : Non-local basis : Local basis
:Nonlocal basis : Local basis Deep Convolutional Framelets (Y, Han, Cha, SIIMS, 2018) ˜ T = I <latexit sha1_base64="nsw6gd0e+4ZO1ci14Ht6KPpot+8=">AAAB/XicbZDLSsNAFIZPvNZ6i5edm8EiuCqJCroRim50V6E3aGKZTCbt0MmFmYlQQ/FV3LhQxK3v4c63cdJmoa0/DHz85xzOmd9LOJPKsr6NhcWl5ZXV0lp5fWNza9vc2W3JOBWENknMY9HxsKScRbSpmOK0kwiKQ4/Ttje8zuvtByoki6OGGiXUDXE/YgEjWGmrZ+47dckcxbhPc7pvoEt02zMrVtWaCM2DXUAFCtV75pfjxyQNaaQIx1J2bStRboaFYoTTcdlJJU0wGeI+7WqMcEilm02uH6Mj7fgoiIV+kUIT9/dEhkMpR6GnO0OsBnK2lpv/1bqpCi7cjEVJqmhEpouClCMVozwK5DNBieIjDZgIpm9FZIAFJkoHVtYh2LNfnofWSdU+rVp3Z5XaVRFHCQ7gEI7BhnOowQ3UoQkEHuEZXuHNeDJejHfjY9q6YBQze/BHxucPPbyUaw==</latexit> -
˜ T = I : Non-local basis : Local basis
: Pooling : Convolution filters Deep Convolutional Framelets (Y, Han, Cha, SIIMS, 2018) ˜ T = I <latexit sha1_base64="nsw6gd0e+4ZO1ci14Ht6KPpot+8=">AAAB/XicbZDLSsNAFIZPvNZ6i5edm8EiuCqJCroRim50V6E3aGKZTCbt0MmFmYlQQ/FV3LhQxK3v4c63cdJmoa0/DHz85xzOmd9LOJPKsr6NhcWl5ZXV0lp5fWNza9vc2W3JOBWENknMY9HxsKScRbSpmOK0kwiKQ4/Ttje8zuvtByoki6OGGiXUDXE/YgEjWGmrZ+47dckcxbhPc7pvoEt02zMrVtWaCM2DXUAFCtV75pfjxyQNaaQIx1J2bStRboaFYoTTcdlJJU0wGeI+7WqMcEilm02uH6Mj7fgoiIV+kUIT9/dEhkMpR6GnO0OsBnK2lpv/1bqpCi7cjEVJqmhEpouClCMVozwK5DNBieIjDZgIpm9FZIAFJkoHVtYh2LNfnofWSdU+rVp3Z5XaVRFHCQ7gEI7BhnOowQ3UoQkEHuEZXuHNeDJejHfjY9q6YBQze/BHxucPPbyUaw==</latexit> Hd(f) Hd(f) = ˜ T ˜ T C C = T Hd(f) C = T (f ~ ) Encoder: Unlifting: f = (˜C) ~ ⌧(˜ ) convolution pooling un-pooling convolution Decoder: -
K-space Deep Learning (Radial R=6) Ground-truth Acceleration Image learning CS
K-space learning Han et al, IEEE TMI (in press) -
K-space Deep Learning (Radial R=6) Ground-truth Acceleration Image learning CS
K-space learning Han et al, IEEE TMI (in press) -
EPI Ghost Artifact Correction Gx RO G y PE G
z SS R F Ghost artifact image In EPI, Gradient is distorted by eddy currents and this causes phase shift Distorted gradient FT Even and odd echo mismatch causes ghost artifact! Phase shift -
k-space Deep Learning for EPI Ghost Correction Image domain loss
L2 loss is calculated on the image domain k-space (with Ghost) e eo o … e e o o … ALOHA IFT e e o o … Coil 1 … coil P Coil 1 … coil P Coil 1 … coil P Neural network k-space (with Ghost) e e o o … e e o o … IFT e e o o … Coil 1 … coil P Coil 1 … coil P Coil 1 … coil P k-space learning Network Input Network Label 34 Lee et al, MRM (in press), 2019 -
7T EPI result (R=2) ALOHA Ghost image Half ROI learning
With Reference PEC-SENSE Proposed (Full ROI) GSR : 10.48% GSR : 9.71% GSR : 15.04% GSR : 8.80% GSR : 4.92% 49 Lee et al, MRM (in press), 2019 -
Improving U-Net Ye et al, SIAM J. Imaging Science, 2018
Han et al, IEEE Trans. Medical Imaging, 2018 Yoo et al, SIAM J. Applied Math, 2019 -
Problem of U-net Pooling does NOT satisfy the frame condition
JC Ye et al, SIAM Journal Imaging Sciences, 2018 Y. Han et al, TMI, 2018. ext > ext = I + > 6= I -
Improving U-net using Deep Conv Framelets • Dual Frame U-net
• Tight Frame U-net JC Ye et al, SIAM Journal Imaging Sciences, 2018 Y. Han and J. C. Ye, TMI, 2018 -
U-Net versus Dual Frame U-Net Y. Han and J. C.
Ye, TMI, 2018; Yoo et al, SIJAM, 2018 -
Summary • Deep learning is a novel signal representation using
combinatorial framelets • ReLUs generate multiple linear representation by partitioning the input space • Local Lipschitz controls the global Liptschiz continuity • Skipped connection improves the optimization landscape • Black-box nature of neural networks have been being unveiled. -
Outlooks q End-to-End AI for radiological imaging Ø From AI-powered
image acquisition to diagnosis for clear and rapid radiological imaging Existing AI Solutions: Diagnosis Our future: from acquisition to diagnosis -
Acknowledgement • Daniel Rueckert (Imperial College) • Florian Knoll (NYU)
• Fang Liu (Univ. of Wisconsin) • Mehmet Akcakaya (Univ. of Minnesota) • Dong Liang (SIAT, China) • Dinggang Shen (UNC) • Peder Larson (UCSF) • Grant – NRF of Korea – Ministry of Trade Industry and Energy • Hyunwook Park (KAIST) • Sung-hong Park (KAIST) • Jongho Lee (SNU) • Doshik Hwang (Yonsei Univ) • Won-Jin Moon (KonkukUniv Medical Center) • Eungyeop Kim (Gachon Univ. Medical Center) • Leonard Sunwoo (SNUBH) • Kyuhwan Jung (Vuno) -
References 1.Ye, Jong Chul, and Woon Kyoung Sung. "Understanding Geometry
of Encoder-Decoder CNNs." International Conference on Machine Learning (ICML), 2019. 2.Jong Chul Ye, Yoseob Han and Eunju Cha, "Deep convolutional framelets: a general deep learning framework for inverse problems", SIAM Journal on Imaging Sciences 11(2), 991–1048, 2018. 3. Yoseob Han and Jong Chul Ye,"Framing U-Net via Deep Convolutional Framelets: Application to Sparse-view CT", IEEE Trans. on Medical Imaging, vol. 37, no. 6, pp. 1418-1429, June, 2018. 4. Yoseob Han, Leonard Sunwoo, and Jong Chul Ye, "k-Space Deep Learning for Accelerated MRI", IEEE Trans. on Medical Imaging (in press), 2019 5. Juyoung Lee, Yoseob Han, Jae-Kyun Ryu, Jang-Yeon Park and Jong Chul Ye, "k-Space Deep Learning for Reference-free EPI Ghost Correction", Magnetic Resonance in Medicine (in press), 2019 6. Eunhee Kang, Won Chang, Jaejun Yoo, and Jong Chul Ye,"Deep Convolutional Framelet Denosing for Low-Dose CT via Wavelet Residual Network", IEEE Trans. on Medical Imaging, vol. 37, no.6, pp. 1358-1369, 2018. 7. Eunhee Kang, Junhong Min and Jong Chul Ye, " A Deep Convolutional Neural Network using Directional Wavelets for Low-dose X- ray CT Reconstruction", Medical Physics 44.10 (2017). 8.Jong Chul Ye, Jong Min Kim, Kyong Hwan Jin and Kiryung Lee, "Compressive sampling using annihilating filter-based low-rank interpolation", IEEE Trans. on Information Theory, vol. 63, no. 2, pp.777-801, Feb. 2017. 9. Kyong Hwan Jin, Dongwook Lee, and Jong Chul Ye. "A general framework for compressed sensing and parallel MRI using annihilating filter based low-rank Hankel matrix," IEEE Trans. on Computational Imaging, vol 2, no. 4, pp. 480 - 495, Dec. 2016.