Tweet
Tweet

Slide 1

Slide 1 text

Understanding Geometry of Encoder Decoder CNN for Inverse Problems Jong Chul Ye, Ph.D Endowed Chair Professor BISPL - BioImaging, Signal Processing, and Learning lab. Dept. Bio & Brain Engineering Dept. Mathematical Sciences KAIST, Korea

Slide 2

Slide 2 text

Deep Learning Revolution • Deep learning has been successfully used for classification, low- level computer vision, etc • Even outperforms human observers Figure modified from Kaiming He’s presentation

Slide 3

Slide 3 text

Classical Learning vs Deep Learning Diagnosis Classical machine learning Deep learning (no feature engineering) Feature Engineering Esteva et al, Nature Medicine, (2019)

Slide 4

Slide 4 text

Deep Learning Era in Medical Imaging Diabetic eye diagnosis Gulshan, V. et al. JAMA (2016) Skin Cancer diagnosis Esteva et al, Nature (2017) OCT diagnosis De Fauw et al, Nature Medicine (2018) Figure courtesy of X. Cao & D. Shen Image registration Image segmentation Ronneberger et al, MICCAI, 2015

Slide 5

Slide 5 text

Deep Learning for Inverse Problems Diagnosis Diagnosis & analysis Focus of this talk: Reconstruction

Slide 6

Slide 6 text

6

Slide 7

Slide 7 text

7 Wavelet transform level 2 level 1 level 3 level 4 Wavelet recomposition + Residual learning : Low-resolution image bypass High SNR band CNN (Kang, et al, Medical Physics 44(10))

Slide 8

Slide 8 text

Full dose Quarter dose

Slide 9

Slide 9 text

Full dose Ours from Quarter dose

Slide 10

Slide 10 text

Ours (Kang et al, TMI, 2018)

Slide 11

Slide 11 text

Ours (Kang et al, TMI, 2018)

Slide 12

Slide 12 text

MBIR C D Ours MBIR (Kang et al, TMI, 2018)

Slide 13

Slide 13 text

Figures from internet Extreme Sparse View CT Stationary CT Carry on baggage scanner Han et al, arXiv preprint arXiv:1712.10248, (2017); CT Meeting (2017)

Slide 14

Slide 14 text

Source/detector configuration Han et al, arXiv preprint arXiv:1712.10248, (2017); CT Meeting (2017)

Slide 15

Slide 15 text

1st view 2nd view 3rd view 4th view 5th view 6th view 7th view 8th view 9th view

Slide 16

Slide 16 text

FBP

Slide 17

Slide 17 text

TV

Slide 18

Slide 18 text

Deep Learning Han et al, CT meetings, 2018

Slide 19

Slide 19 text

Deep Learning Pioneers in MR Kwon et al, Medical Physics, 2017 Hammernik et al, MRM, 2018 Wang et al, ISBI, 2016 Yang et al, NIPS, 2016 Multilayer perceptron Variational network Deep learning prior ADMM-Net

Slide 20

Slide 20 text

Variational Network (R=4) CG SENSE PI-CS: TGV Learning: VN PI PI-CS Learning Hammernik MRM 2018 Courtesy of Florian Knoll

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

Image Domain Learning FBPConvNet Jin et al. TIP 2017

Slide 23

Slide 23 text

Hybrid Domain Learning Deep Cascade of CNNs for MRI Reconstruction Schlemper et al. IEEE TMI 2017 Courtesy of D. Rueckert (a) 11x Undersampled (b) CNN reconstruction (c) Ground Truth

Slide 24

Slide 24 text

Hybrid Domain Learning Learned Primal and Dual Adler et al, IEEE TMI 2018

Slide 25

Slide 25 text

Domain-transform Learning AUTOMAP Zhu et al, Nature, 2018

Slide 26

Slide 26 text

Sensor-domain Learning CNN k-space deep learning Han et al, IEEE TMI; Lee et al, MRM (in press, 2019)

Slide 27

Slide 27 text

Why so popular this time ? q Accuracy: high quality recon > CS q Fast reconstruction time q Business model: vendor-driven training q Interpretable models q Flexibility: more than recon Imaging time Reconstruction time Conventional Compressed Sensing Machine Learning

Slide 28

Slide 28 text

INTERPRETATION OF DEEP LEARNING FOR RECON

Slide 29

Slide 29 text

Too Simple to Analyze..? Convolution & pooling à stone age tools of signal processing What do they do ?

Slide 30

Slide 30 text

Dark Age of Applied Mathematics ?

Slide 31

Slide 31 text

WHY DEEP LEARNING WORKS FOR RECON ? DOES IT CREATE ANY ARTIFICIAL FEATURES ?

Slide 32

Slide 32 text

Ye et al, SIAM J. Imaging Sciences, 2018; Ye et al, ICML, 2019 Understanding Geometry of CNN

Slide 33

Slide 33 text

CNN Encoder-Decoder CNN for Inverse Problems

Slide 34

Slide 34 text

CNN Encoder-Decoder CNN for Inverse Problems

Slide 35

Slide 35 text

CNN Successful applications to various inverse problems Encoder-Decoder CNN for Inverse Problems

Slide 36

Slide 36 text

Why Same Architecture Works for Different Inverse Problems ?

Slide 37

Slide 37 text

Classical Methods for Inverse Problems Synthesis frame Analysis frame coefficients Step 1: Signal Representation x = X i hbi, xi˜ bi AAACE3icbVDLSsNAFJ3UV62vqEs3g0UQkZKooBuh4MZlC/YBTQiTyaQdOpmEmYm0hIKf4MZfceNCEbdu3Pk3TpMutPXAhTPn3Mvce/yEUaks69soLS2vrK6V1ysbm1vbO+buXlvGqcCkhWMWi66PJGGUk5aiipFuIgiKfEY6/vBm6nfuiZA05ndqnBA3Qn1OQ4qR0pJnnoyuHZlGHoXQYYj3GYG+R0/hyBHFy1GUBbnomVWrZuWAi8SekSqYoeGZX04Q4zQiXGGGpOzZVqLcDAlFMSOTipNKkiA8RH3S05SjiEg3y2+awCOtBDCMhS6uYK7+nshQJOU48nVnhNRAzntT8T+vl6rwys0oT1JFOC4+ClMGVQynAcGACoIVG2uCsKB6V4gHSCCsdIwVHYI9f/IiaZ/V7POa1byo1psPRRxlcAAOwTGwwSWog1vQAC2AwSN4Bq/gzXgyXox346NoLRmzCPfBHxifP6mnndg=

Slide 38

Slide 38 text

Eg. Compressed Sensing Classical Methods for Inverse Problems Step 2: Basis Search by Optimization x = X i ˜ bi hbi, xi AAACEnicbZBNS8MwGMdTX+d8q3r0EhyCgoxWBb0IQy8eJ7gXWEtJ03QLS9OSpLJR9hm8+FW8eFDEqydvfhvTrgfd/EPgl//zPCTP308Ylcqyvo2FxaXlldXKWnV9Y3Nr29zZbcs4FZi0cMxi0fWRJIxy0lJUMdJNBEGRz0jHH97k9c4DEZLG/F6NE+JGqM9pSDFS2vLM49GVI9PIo9BRlAUE+jkyxPus4BM4ckRx88yaVbcKwXmwS6iBUk3P/HKCGKcR4QozJGXPthLlZkgoihmZVJ1UkgThIeqTnkaOIiLdrFhpAg+1E8AwFvpwBQv390SGIinHka87I6QGcraWm//VeqkKL92M8iRVhOPpQ2HKoIphng8MqCBYsbEGhAXVf4V4gATCSqdY1SHYsyvPQ/u0bp/VrbvzWuO6jKMC9sEBOAI2uAANcAuaoAUweATP4BW8GU/Gi/FufExbF4xyZg/8kfH5AyCNnR8=

Slide 39

Slide 39 text

Why do They Look so Different ? Any Link between Them ?

Slide 40

Slide 40 text

Our Theoretical Findings y = X i hbi(x), xi˜ bi(x) AAACGXicbZDLSsNAFIYnXmu9RV26GSxCBSmJCroRim5cVrAXaEKYTCbt0MkkzEykIfQ13Pgqblwo4lJXvo3TNoK2/jDw851zOHN+P2FUKsv6MhYWl5ZXVktr5fWNza1tc2e3JeNUYNLEMYtFx0eSMMpJU1HFSCcRBEU+I21/cD2ut++JkDTmdypLiBuhHqchxUhp5JlWdunINPIohA5DvMcI9D1aHR4dw6EjpsBRlAU/3DMrVs2aCM4buzAVUKjhmR9OEOM0IlxhhqTs2lai3BwJRTEjo7KTSpIgPEA90tWWo4hIN59cNoKHmgQwjIV+XMEJ/T2Ro0jKLPJ1Z4RUX87WxvC/WjdV4YWbU56kinA8XRSmDKoYjmOCARUEK5Zpg7Cg+q8Q95FAWOkwyzoEe/bkedM6qdmnNev2rFK/KuIogX1wAKrABuegDm5AAzQBBg/gCbyAV+PReDbejPdp64JRzOyBPzI+vwEaXJ8Y Ye et al, SIIMS, 2018; Ye et al, ICML, 2019

Slide 41

Slide 41 text

y = X i hbi(x), xi˜ bi(x) AAACGXicbZDLSsNAFIYnXmu9RV26GSxCBSmJCroRim5cVrAXaEKYTCbt0MkkzEykIfQ13Pgqblwo4lJXvo3TNoK2/jDw851zOHN+P2FUKsv6MhYWl5ZXVktr5fWNza1tc2e3JeNUYNLEMYtFx0eSMMpJU1HFSCcRBEU+I21/cD2ut++JkDTmdypLiBuhHqchxUhp5JlWdunINPIohA5DvMcI9D1aHR4dw6EjpsBRlAU/3DMrVs2aCM4buzAVUKjhmR9OEOM0IlxhhqTs2lai3BwJRTEjo7KTSpIgPEA90tWWo4hIN59cNoKHmgQwjIV+XMEJ/T2Ro0jKLPJ1Z4RUX87WxvC/WjdV4YWbU56kinA8XRSmDKoYjmOCARUEK5Zpg7Cg+q8Q95FAWOkwyzoEe/bkedM6qdmnNev2rFK/KuIogX1wAKrABuegDm5AAzQBBg/gCbyAV+PReDbejPdp64JRzOyBPzI+vwEaXJ8Y Our Theoretical Findings Ye et al, SIIMS, 2018; Ye et al, ICML, 2019

Slide 42

Slide 42 text

y = X i hbi(x), xi˜ bi(x) AAACGXicbZDLSsNAFIYnXmu9RV26GSxCBSmJCroRim5cVrAXaEKYTCbt0MkkzEykIfQ13Pgqblwo4lJXvo3TNoK2/jDw851zOHN+P2FUKsv6MhYWl5ZXVktr5fWNza1tc2e3JeNUYNLEMYtFx0eSMMpJU1HFSCcRBEU+I21/cD2ut++JkDTmdypLiBuhHqchxUhp5JlWdunINPIohA5DvMcI9D1aHR4dw6EjpsBRlAU/3DMrVs2aCM4buzAVUKjhmR9OEOM0IlxhhqTs2lai3BwJRTEjo7KTSpIgPEA90tWWo4hIN59cNoKHmgQwjIV+XMEJ/T2Ro0jKLPJ1Z4RUX87WxvC/WjdV4YWbU56kinA8XRSmDKoYjmOCARUEK5Zpg7Cg+q8Q95FAWOkwyzoEe/bkedM6qdmnNev2rFK/KuIogX1wAKrABuegDm5AAzQBBg/gCbyAV+PReDbejPdp64JRzOyBPzI+vwEaXJ8Y Our Theoretical Findings Ye et al, SIIMS, 2018; Ye et al, ICML, 2019

Slide 43

Slide 43 text

analysis basis y = X i hbi(x), xi˜ bi(x) AAACGXicbZDLSsNAFIYnXmu9RV26GSxCBSmJCroRim5cVrAXaEKYTCbt0MkkzEykIfQ13Pgqblwo4lJXvo3TNoK2/jDw851zOHN+P2FUKsv6MhYWl5ZXVktr5fWNza1tc2e3JeNUYNLEMYtFx0eSMMpJU1HFSCcRBEU+I21/cD2ut++JkDTmdypLiBuhHqchxUhp5JlWdunINPIohA5DvMcI9D1aHR4dw6EjpsBRlAU/3DMrVs2aCM4buzAVUKjhmR9OEOM0IlxhhqTs2lai3BwJRTEjo7KTSpIgPEA90tWWo4hIN59cNoKHmgQwjIV+XMEJ/T2Ro0jKLPJ1Z4RUX87WxvC/WjdV4YWbU56kinA8XRSmDKoYjmOCARUEK5Zpg7Cg+q8Q95FAWOkwyzoEe/bkedM6qdmnNev2rFK/KuIogX1wAKrABuegDm5AAzQBBg/gCbyAV+PReDbejPdp64JRzOyBPzI+vwEaXJ8Y Encoder Our Theoretical Findings Ye et al, SIIMS, 2018; Ye et al, ICML, 2019

Slide 44

Slide 44 text

analysis basis synthesis basis y = X i hbi(x), xi˜ bi(x) AAACGXicbZDLSsNAFIYnXmu9RV26GSxCBSmJCroRim5cVrAXaEKYTCbt0MkkzEykIfQ13Pgqblwo4lJXvo3TNoK2/jDw851zOHN+P2FUKsv6MhYWl5ZXVktr5fWNza1tc2e3JeNUYNLEMYtFx0eSMMpJU1HFSCcRBEU+I21/cD2ut++JkDTmdypLiBuhHqchxUhp5JlWdunINPIohA5DvMcI9D1aHR4dw6EjpsBRlAU/3DMrVs2aCM4buzAVUKjhmR9OEOM0IlxhhqTs2lai3BwJRTEjo7KTSpIgPEA90tWWo4hIN59cNoKHmgQwjIV+XMEJ/T2Ro0jKLPJ1Z4RUX87WxvC/WjdV4YWbU56kinA8XRSmDKoYjmOCARUEK5Zpg7Cg+q8Q95FAWOkwyzoEe/bkedM6qdmnNev2rFK/KuIogX1wAKrABuegDm5AAzQBBg/gCbyAV+PReDbejPdp64JRzOyBPzI+vwEaXJ8Y Encoder Decoder Our Theoretical Findings Ye et al, SIIMS, 2018; Ye et al, ICML, 2019

Slide 45

Slide 45 text

Linear Encoder-Decoder (ED) CNN Learned filters y = ˜ BB>x = X i hx, bi i˜ bi AAACJ3icbVBNSwMxEM36bf2qevQSLIIHKbsq6KUievGoaK3QrSWbzrah2eySzIpl8d948a94EVREj/4T03YP2joQePPePCbzgkQKg6775UxMTk3PzM7NFxYWl5ZXiqtr1yZONYcqj2WsbwJmQAoFVRQo4SbRwKJAQi3onvb12h1oI2J1hb0EGhFrKxEKztBSzeJRr+KjkC2gJ/Tk1sc4ofe0Qn2TRk1BfclUWwK936FBv9XDNndYqlksuWV3UHQceDkokbzOm8VXvxXzNAKFXDJj6p6bYCNjGgWX8FDwUwMJ413WhrqFikVgGtngzge6ZZkWDWNtn0I6YH87MhYZ04sCOxkx7JhRrU/+p9VTDA8bmVBJiqD4cFGYSoox7YdGW0IDR9mzgHEt7F8p7zDNONpoCzYEb/TkcXC9W/b2yu7Ffun4Mo9jjmyQTbJNPHJAjskZOSdVwskjeSZv5N15cl6cD+dzODrh5J518qec7x9ZY6R3 pooling un-pooling

Slide 46

Slide 46 text

Linear E-D CNN w/ Skipped Connection more redundant expression Learned filters y = ˜ BB>x = X i hx, bi i˜ bi AAACJ3icbVBNSwMxEM36bf2qevQSLIIHKbsq6KUievGoaK3QrSWbzrah2eySzIpl8d948a94EVREj/4T03YP2joQePPePCbzgkQKg6775UxMTk3PzM7NFxYWl5ZXiqtr1yZONYcqj2WsbwJmQAoFVRQo4SbRwKJAQi3onvb12h1oI2J1hb0EGhFrKxEKztBSzeJRr+KjkC2gJ/Tk1sc4ofe0Qn2TRk1BfclUWwK936FBv9XDNndYqlksuWV3UHQceDkokbzOm8VXvxXzNAKFXDJj6p6bYCNjGgWX8FDwUwMJ413WhrqFikVgGtngzge6ZZkWDWNtn0I6YH87MhYZ04sCOxkx7JhRrU/+p9VTDA8bmVBJiqD4cFGYSoox7YdGW0IDR9mzgHEt7F8p7zDNONpoCzYEb/TkcXC9W/b2yu7Ffun4Mo9jjmyQTbJNPHJAjskZOSdVwskjeSZv5N15cl6cD+dzODrh5J518qec7x9ZY6R3

Slide 47

Slide 47 text

Deep Convolutional Framelets x = ˜ BB>x = X i hx, bi i˜ bi AAACJ3icbVBNSwMxEM36bf2qevQSLIIHKbsq6EWRevGoaK3QXZdsOq2h2eySzEpL6b/x4l/xIqiIHv0npu0etDoQePPePCbzolQKg6776UxMTk3PzM7NFxYWl5ZXiqtr1ybJNIcqT2SibyJmQAoFVRQo4SbVwOJIQi1qnw702j1oIxJ1hd0Ugpi1lGgKztBSYfG4c+SjkA2gFVq59TFJaYceUd9kcSioL5lqSaCdHRoNWj1qc4elwmLJLbvDon+Bl4MSyes8LL74jYRnMSjkkhlT99wUgx7TKLiEfsHPDKSMt1kL6hYqFoMJesM7+3TLMg3aTLR9CumQ/enosdiYbhzZyZjhnRnXBuR/Wj3D5mHQEyrNEBQfLWpmkmJCB6HRhtDAUXYtYFwL+1fK75hmHG20BRuCN37yX3C9W/b2yu7FfunkMo9jjmyQTbJNPHJATsgZOSdVwskDeSKv5M15dJ6dd+djNDrh5J518qucr29XoqR2 Perfect reconstruction Ye et al, SIIMS 2018; Ye et al, ICML 2019 Frame conditions w skipped connection w/o skipped connection

Slide 48

Slide 48 text

Deep Convolutional Framelets x = ˜ BB>x = X i hx, bi i˜ bi AAACJ3icbVBNSwMxEM36bf2qevQSLIIHKbsq6EWRevGoaK3QXZdsOq2h2eySzEpL6b/x4l/xIqiIHv0npu0etDoQePPePCbzolQKg6776UxMTk3PzM7NFxYWl5ZXiqtr1ybJNIcqT2SibyJmQAoFVRQo4SbVwOJIQi1qnw702j1oIxJ1hd0Ugpi1lGgKztBSYfG4c+SjkA2gFVq59TFJaYceUd9kcSioL5lqSaCdHRoNWj1qc4elwmLJLbvDon+Bl4MSyes8LL74jYRnMSjkkhlT99wUgx7TKLiEfsHPDKSMt1kL6hYqFoMJesM7+3TLMg3aTLR9CumQ/enosdiYbhzZyZjhnRnXBuR/Wj3D5mHQEyrNEBQfLWpmkmJCB6HRhtDAUXYtYFwL+1fK75hmHG20BRuCN37yX3C9W/b2yu7FfunkMo9jjmyQTbJNPHJATsgZOSdVwskDeSKv5M15dJ6dd+djNDrh5J518qucr29XoqR2 Perfect reconstruction Ye et al, SIAM J. Imaging Science, 2018 Frame conditions w skipped connection w/o skipped connection

Slide 49

Slide 49 text

Role of ReLUs? Generator for Multiple Expressions y = ˜ B(x)B(x)>x = X i hx, bi(x)i˜ bi(x) AAACM3icbVDLSgMxFM3Ud31VXboJFkFByowKuhGKbsSVotVCpw6Z9LYNZjJDckdaSv/JjT/iQhAXirj1H0zbWaj1QsLhPEjuCRMpDLrui5ObmJyanpmdy88vLC4tF1ZWr02cag4VHstYV0NmQAoFFRQooZpoYFEo4Sa8OxnoN/egjYjVFXYTqEespURTcIaWCgpn3SMfhWwAPd7qbA+vWx/jhHboEfVNGgWC+pKplgTa2aFhIAY2X4+YLDpig0LRLbnDoePAy0CRZHMeFJ78RszTCBRyyYypeW6C9R7TKLiEft5PDSSM37EW1CxULAJT7w137tNNyzRoM9b2KKRD9meixyJjulFonRHDtvmrDcj/tFqKzcN6T6gkRVB89FAzlRRjOiiQNoQGjrJrAeNa2L9S3maacbQ1520J3t+Vx8H1bsnbK7kX+8XyZVbHLFknG2SLeOSAlMkpOScVwskDeSZv5N15dF6dD+dzZM05WWaN/Brn6xvQsKgT ⌃l(x) = 2 6 6 6 4 1 0 · · · 0 0 2 · · · 0 . . . . . . ... . . . 0 0 · · · ml 3 7 7 7 5 AAACkXicbVHLSgMxFM2M7/qqunQTLIpuyowK6kIpuhHcKFoVmjpkMrdtMMkMSUYsQ//H73Hn35i2g4/qhQvnnnNfuYkzwY0Ngg/Pn5qemZ2bX6gsLi2vrFbX1u9NmmsGTZaKVD/G1IDgCpqWWwGPmQYqYwEP8fPFUH94AW14qu5sP4O2pF3FO5xR66io+kZueVfSJ7H7uodPMYmhy1URS2o1fx1gYoZqFOIdHDgnLEmtGQWEjJlxwv6kWCEvZfQNkt+MSwom+5btChkJNxxU8rVKVK0F9WBk+C8IS1BDpV1H1XeSpCyXoCwT1JhWGGS2XVBtORMwqJDcQEbZM+1Cy0FFJZh2MbroAG87JsGdVDtXFo/YnxUFlcb0Zewy3X49M6kNyf+0Vm47x+2Cqyy3oNh4UCcX2KZ4+D044RqYFX0HKNPc7YpZj2rKrPvEijtCOPnkv+B+vx4e1IObw1rjvDzHPNpEW2gXhegINdAlukZNxLwV79A79c78Df/Eb/hlru+VNRvol/lXn1/Lv9k= Input dependent {0,1} matrix --> Input adaptivity

Slide 50

Slide 50 text

Input Space Partitioning for Multiple Expressions

Slide 51

Slide 51 text

Input Space Partitioning for Multiple Expressions A CNN performs automatic assignment of distinct linear representation depending on input

Slide 52

Slide 52 text

Expressivity of E-D CNN # of representation # of network elements

Slide 53

Slide 53 text

Expressivity of E-D CNN # of representation # of network elements # of channel

Slide 54

Slide 54 text

Expressivity of E-D CNN # of representation # of network elements # of channel Network depth

Slide 55

Slide 55 text

Expressivity of E-D CNN # of representation # of network elements # of channel Network depth Skipped connection

Slide 56

Slide 56 text

Lipschitz Continuity K = max p Kp, Kp = k ˜ B(zp)B(zp)>k2 AAACIHicbZDLSgMxFIYz9V5voy7dBItQQcpMFeqmUOpGcKNgbaFTh0wmtaGZmZicEWv1Udz4Km5cKKI7fRrTy0KtBxI+/v8ckvMHUnANjvNpZaamZ2bn5heyi0vLK6v22vq5TlJFWY0mIlGNgGgmeMxqwEGwhlSMRIFg9aB7OPDr10xpnsRn0JOsFZHLmLc5JWAk3y4dl72I3PgSH/ty17tKSTggXMbenQdchAxX87e+3BndFx4k0rvzi76dcwrOsPAkuGPIoXGd+PaHFyY0jVgMVBCtm64jodUnCjgV7D7rpZpJQrvkkjUNxiRiutUfLniPt40S4naizIkBD9WfE30Sad2LAtMZEejov95A/M9rptA+aPV5LFNgMR091E4FhgQP0sIhV4yC6BkgVHHzV0w7RBEKJtOsCcH9u/IknBcL7l7BOd3PVarjOObRJtpCeeSiEqqgI3SCaoiiB/SEXtCr9Wg9W2/W+6g1Y41nNtCvsr6+AcwXoYQ= z1 AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbTbt0swm7E6GG/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvSKQw6LrfTmFldW19o7hZ2tre2d0r7x80TZxqxhsslrFuB9RwKRRvoEDJ24nmNAokbwWjm6nfeuTaiFg94DjhfkQHSoSCUbTS/VPP65UrbtWdgSwTLycVyFHvlb+6/ZilEVfIJDWm47kJ+hnVKJjkk1I3NTyhbEQHvGOpohE3fjY7dUJOrNInYaxtKSQz9fdERiNjxlFgOyOKQ7PoTcX/vE6K4ZWfCZWkyBWbLwpTSTAm079JX2jOUI4toUwLeythQ6opQ5tOyYbgLb68TJpnVe+86t5dVGrXeRxFOIJjOAUPLqEGt1CHBjAYwDO8wpsjnRfn3fmYtxacfOYQ/sD5/AEPZo2k zp AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lU0GPRi8eK9gPaUDbbSbt0swm7G6GG/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvSATXxnW/ncLK6tr6RnGztLW9s7tX3j9o6jhVDBssFrFqB1Sj4BIbhhuB7UQhjQKBrWB0M/Vbj6g0j+WDGSfoR3QgecgZNVa6f+olvXLFrbozkGXi5aQCOeq98le3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9mpE3JilT4JY2VLGjJTf09kNNJ6HAW2M6JmqBe9qfif10lNeOVnXCapQcnmi8JUEBOT6d+kzxUyI8aWUKa4vZWwIVWUGZtOyYbgLb68TJpnVe+86t5dVGrXeRxFOIJjOAUPLqEGt1CHBjAYwDO8wpsjnBfn3fmYtxacfOYQ/sD5/AFu4o3j Related to the generalizability Dependent on the Local Lipschitz

Slide 57

Slide 57 text

Optimization Issue • Optimization Landscape • All local minimizers are global minimizers • Ex) overparameterized network • Implicit bias • Specific global minimizers are determine d by the optimization algorithms Q) Nonconvex optimization problem à Can we prove convergence ? o AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwdZiO5RMmmlD8xiSjFCG/oUbF4q49W/c+Tdm2llo64HA4Zx7ybknSjgz1ve/vdLK6tr6RnmzsrW9s7tX3T9oG5VqQltEcaU7ETaUM0lblllOO4mmWEScPkTjm9x/eKLaMCXv7SShocBDyWJGsHXSY09gO4riTE371Zpf92dAyyQoSA0KNPvVr95AkVRQaQnHxnQDP7FhhrVlhNNppZcammAyxkPadVRiQU2YzRJP0YlTBihW2j1p0Uz9vZFhYcxERG4yT2gWvVz8z+umNr4KMyaT1FJJ5h/FKUdWofx8NGCaEssnjmCimcuKyAhrTKwrqeJKCBZPXibts3pwXvfvLmqN66KOMhzBMZxCAJfQgFtoQgsISHiGV3jzjPfivXsf89GSV+wcwh94nz/vApER x AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg== o AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwdZiO5RMmmlD8xiSjFCG/oUbF4q49W/c+Tdm2llo64HA4Zx7ybknSjgz1ve/vdLK6tr6RnmzsrW9s7tX3T9oG5VqQltEcaU7ETaUM0lblllOO4mmWEScPkTjm9x/eKLaMCXv7SShocBDyWJGsHXSY09gO4riTE371Zpf92dAyyQoSA0KNPvVr95AkVRQaQnHxnQDP7FhhrVlhNNppZcammAyxkPadVRiQU2YzRJP0YlTBihW2j1p0Uz9vZFhYcxERG4yT2gWvVz8z+umNr4KMyaT1FJJ5h/FKUdWofx8NGCaEssnjmCimcuKyAhrTKwrqeJKCBZPXibts3pwXvfvLmqN66KOMhzBMZxCAJfQgFtoQgsISHiGV3jzjPfivXsf89GSV+wcwh94nz/vApER o AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwdZiO5RMmmlD8xiSjFCG/oUbF4q49W/c+Tdm2llo64HA4Zx7ybknSjgz1ve/vdLK6tr6RnmzsrW9s7tX3T9oG5VqQltEcaU7ETaUM0lblllOO4mmWEScPkTjm9x/eKLaMCXv7SShocBDyWJGsHXSY09gO4riTE371Zpf92dAyyQoSA0KNPvVr95AkVRQaQnHxnQDP7FhhrVlhNNppZcammAyxkPadVRiQU2YzRJP0YlTBihW2j1p0Uz9vZFhYcxERG4yT2gWvVz8z+umNr4KMyaT1FJJ5h/FKUdWofx8NGCaEssnjmCimcuKyAhrTKwrqeJKCBZPXibts3pwXvfvLmqN66KOMhzBMZxCAJfQgFtoQgsISHiGV3jzjPfivXsf89GSV+wcwh94nz/vApER x AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg== o AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwdZiO5RMmmlD8xiSjFCG/oUbF4q49W/c+Tdm2llo64HA4Zx7ybknSjgz1ve/vdLK6tr6RnmzsrW9s7tX3T9oG5VqQltEcaU7ETaUM0lblllOO4mmWEScPkTjm9x/eKLaMCXv7SShocBDyWJGsHXSY09gO4riTE371Zpf92dAyyQoSA0KNPvVr95AkVRQaQnHxnQDP7FhhrVlhNNppZcammAyxkPadVRiQU2YzRJP0YlTBihW2j1p0Uz9vZFhYcxERG4yT2gWvVz8z+umNr4KMyaT1FJJ5h/FKUdWofx8NGCaEssnjmCimcuKyAhrTKwrqeJKCBZPXibts3pwXvfvLmqN66KOMhzBMZxCAJfQgFtoQgsISHiGV3jzjPfivXsf89GSV+wcwh94nz/vApER x AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg== o AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwdZiO5RMmmlD8xiSjFCG/oUbF4q49W/c+Tdm2llo64HA4Zx7ybknSjgz1ve/vdLK6tr6RnmzsrW9s7tX3T9oG5VqQltEcaU7ETaUM0lblllOO4mmWEScPkTjm9x/eKLaMCXv7SShocBDyWJGsHXSY09gO4riTE371Zpf92dAyyQoSA0KNPvVr95AkVRQaQnHxnQDP7FhhrVlhNNppZcammAyxkPadVRiQU2YzRJP0YlTBihW2j1p0Uz9vZFhYcxERG4yT2gWvVz8z+umNr4KMyaT1FJJ5h/FKUdWofx8NGCaEssnjmCimcuKyAhrTKwrqeJKCBZPXibts3pwXvfvLmqN66KOMhzBMZxCAJfQgFtoQgsISHiGV3jzjPfivXsf89GSV+wcwh94nz/vApER x AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg== x AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg== x AAAB8XicbVDLSgMxFL1TX7W+qi7dBIvgqsyooCspuHFZwT6wLSWT3mlDM5khyYhl6F+4caGIW//GnX9jpp2Fth4IHM65l5x7/FhwbVz32ymsrK6tbxQ3S1vbO7t75f2Dpo4SxbDBIhGptk81Ci6xYbgR2I4V0tAX2PLHN5nfekSleSTvzSTGXkiHkgecUWOlh25IzcgP0qdpv1xxq+4MZJl4OalAjnq//NUdRCwJURomqNYdz41NL6XKcCZwWuomGmPKxnSIHUslDVH30lniKTmxyoAEkbJPGjJTf2+kNNR6Evp2MkuoF71M/M/rJCa46qVcxolByeYfBYkgJiLZ+WTAFTIjJpZQprjNStiIKsqMLalkS/AWT14mzbOqd1517y4qteu8jiIcwTGcggeXUINbqEMDGEh4hld4c7Tz4rw7H/PRgpPvHMIfOJ8//K+RGg==

Slide 58

Slide 58 text

Benign Optimization Landscape full-rank condition Independent features Nguyen, et al, ICML, 2018

Slide 59

Slide 59 text

Benign Optimization Landscape full-rank condition Independent features Independent features full-rank condition Nguyen, et al, ICML, 2018 Ye et al, ICML, 2019

Slide 60

Slide 60 text

Regularized Recon vs. Deep Recon Diagnosis Classical Regularized Recon (basis engineering) Deep Recon (no basis engineering) Basis Engineering

Slide 61

Slide 61 text

THEORY-DRIVEN CNN DESIGN :some snapshots

Slide 62

Slide 62 text

Deep Beamformer Khan et al, MICCAI, 2019

Slide 63

Slide 63 text

Focused / Plane Wave Ultrasound Imaging Focused Imaging Plane Wave Imaging Couade M, JVDI, 2015 Yoon et al, TMI, 2018

Slide 64

Slide 64 text

Adaptive Beamformer Conventional Beamforming Pipeline  I Q =  zl[n] H(zl)[n] AAACSnicbZBPSwMxEMWztWpd/6169BIsQr2UXRX0IhS91JuCtYXuUrLptA3NZpckK9aln8+LJ29+CC8eFPFiWovY1oHA4zdvmMkLE86Udt0XK7eQX1xaLqzYq2vrG5vO1vatilNJoUZjHstGSBRwJqCmmebQSCSQKORQD/sXo379DqRisbjRgwSCiHQF6zBKtEEth/ghdJnIwohoye6Hl9j38TX2QbR/GT7DMy77ocWbIjBe2zeoRwnPqsOSoQcG21PTLafolt1x4XnhTUQRTeqq5Tz77ZimEQhNOVGq6bmJDjIiNaMchrafKkgI7ZMuNI0UJAIVZOMohnjfkDbuxNI8ofGY/p3ISKTUIAqNc3S4mu2N4H+9Zqo7p0HGRJJqEPRnUSflWMd4lCtuMwlU84ERhEpmbsW0RySh2qRvmxC82S/Pi9vDsndUdq+Pi5XzSRwFtIv2UAl56ARVUBVdoRqi6BG9onf0YT1Zb9an9fVjzVmTmR00Vbn8N6l4s94= Data specific weights

Slide 65

Slide 65 text

Deep Beamformer Khan et al, MICCAI (accepted), 2019 Adaptive and Compressive Deep Beamformer Conventional Beamforming Pipeline Input partitioning à Data specific representation

Slide 66

Slide 66 text

B-mode Focused Mode Deep Beamformer Khan et al, MICCAI (accepted), 2019

Slide 67

Slide 67 text

Plane Wave Deep Beamformer Khan et al, MICCAI (accepted), 2019

Slide 68

Slide 68 text

k-Space Deep Learning Han et al, IEEE Trans. Medical Imaging (in press), 2019 Lee et al, MRM (in press), 2019

Slide 69

Slide 69 text

Duality between Sparsity and Low-Rankness Ye et al, IEEE Trans. Information Theory, 2017;Jin et al, IEEE TCI, 2017; Lee et al, MRM, 2017

Slide 70

Slide 70 text

Annihilating Filter-based Low-rank Hankel Matrix Approach (ALOHA) Jin KH et al IEEE TCI, 2016, Lee et al, MRM, 2015; Ye et al, IEEE TIT, 2016

Slide 71

Slide 71 text

Single coil MRI Jin KH et al IEEE TCI, 2016 Lee et al, MRM, 2015

Slide 72

Slide 72 text

Parallel MRI Jin KH et al IEEE TCI, 2016 Lee et al, MRM, 2015

Slide 73

Slide 73 text

Optimization Problem Equivalent to search for frame basis to make the nonzero-coefficents sparse

Slide 74

Slide 74 text

: Non-local basis : Local basis Convolution Framelets (Yin et al, SIIMS, 2017) > = I > = I

Slide 75

Slide 75 text

: Non-local basis : Local basis Convolution Framelets (Yin et al, SIIMS, 2017) > = I > = I

Slide 76

Slide 76 text

: Non-local basis : Local basis Convolution Framelets (Yin et al, SIIMS, 2017) > = I > = I

Slide 77

Slide 77 text

˜ T = I : Non-local basis : Local basis :Nonlocal basis : Local basis Deep Convolutional Framelets (Y, Han, Cha, SIIMS, 2018) ˜ T = I AAAB/XicbZDLSsNAFIZPvNZ6i5edm8EiuCqJCroRim50V6E3aGKZTCbt0MmFmYlQQ/FV3LhQxK3v4c63cdJmoa0/DHz85xzOmd9LOJPKsr6NhcWl5ZXV0lp5fWNza9vc2W3JOBWENknMY9HxsKScRbSpmOK0kwiKQ4/Ttje8zuvtByoki6OGGiXUDXE/YgEjWGmrZ+47dckcxbhPc7pvoEt02zMrVtWaCM2DXUAFCtV75pfjxyQNaaQIx1J2bStRboaFYoTTcdlJJU0wGeI+7WqMcEilm02uH6Mj7fgoiIV+kUIT9/dEhkMpR6GnO0OsBnK2lpv/1bqpCi7cjEVJqmhEpouClCMVozwK5DNBieIjDZgIpm9FZIAFJkoHVtYh2LNfnofWSdU+rVp3Z5XaVRFHCQ7gEI7BhnOowQ3UoQkEHuEZXuHNeDJejHfjY9q6YBQze/BHxucPPbyUaw==

Slide 78

Slide 78 text

˜ T = I : Non-local basis : Local basis : Pooling : Convolution filters Deep Convolutional Framelets (Y, Han, Cha, SIIMS, 2018) ˜ T = I AAAB/XicbZDLSsNAFIZPvNZ6i5edm8EiuCqJCroRim50V6E3aGKZTCbt0MmFmYlQQ/FV3LhQxK3v4c63cdJmoa0/DHz85xzOmd9LOJPKsr6NhcWl5ZXV0lp5fWNza9vc2W3JOBWENknMY9HxsKScRbSpmOK0kwiKQ4/Ttje8zuvtByoki6OGGiXUDXE/YgEjWGmrZ+47dckcxbhPc7pvoEt02zMrVtWaCM2DXUAFCtV75pfjxyQNaaQIx1J2bStRboaFYoTTcdlJJU0wGeI+7WqMcEilm02uH6Mj7fgoiIV+kUIT9/dEhkMpR6GnO0OsBnK2lpv/1bqpCi7cjEVJqmhEpouClCMVozwK5DNBieIjDZgIpm9FZIAFJkoHVtYh2LNfnofWSdU+rVp3Z5XaVRFHCQ7gEI7BhnOowQ3UoQkEHuEZXuHNeDJejHfjY9q6YBQze/BHxucPPbyUaw== Hd(f) Hd(f) = ˜ T ˜ T C C = T Hd(f) C = T (f ~ ) Encoder: Unlifting: f = (˜C) ~ ⌧(˜ ) convolution pooling un-pooling convolution Decoder:

Slide 79

Slide 79 text

Single Resolution Network Architecture

Slide 80

Slide 80 text

Multi-Resolution Network Architecture

Slide 81

Slide 81 text

ALOHA CNN k-Space Deep Learning Han et al, IEEE TMI,2019

Slide 82

Slide 82 text

K-space Deep Learning (Radial R=6) Ground-truth Acceleration Image learning CS K-space learning Han et al, IEEE TMI (in press)

Slide 83

Slide 83 text

K-space Deep Learning (Radial R=6) Ground-truth Acceleration Image learning CS K-space learning Han et al, IEEE TMI (in press)

Slide 84

Slide 84 text

EPI Ghost Artifact Correction Gx RO G y PE G z SS R F Ghost artifact image In EPI, Gradient is distorted by eddy currents and this causes phase shift Distorted gradient FT Even and odd echo mismatch causes ghost artifact! Phase shift

Slide 85

Slide 85 text

k-space Deep Learning for EPI Ghost Correction Image domain loss L2 loss is calculated on the image domain k-space (with Ghost) e eo o … e e o o … ALOHA IFT e e o o … Coil 1 … coil P Coil 1 … coil P Coil 1 … coil P Neural network k-space (with Ghost) e e o o … e e o o … IFT e e o o … Coil 1 … coil P Coil 1 … coil P Coil 1 … coil P k-space learning Network Input Network Label 34 Lee et al, MRM (in press), 2019

Slide 86

Slide 86 text

7T EPI result (R=2) ALOHA Ghost image Half ROI learning With Reference PEC-SENSE Proposed (Full ROI) GSR : 10.48% GSR : 9.71% GSR : 15.04% GSR : 8.80% GSR : 4.92% 49 Lee et al, MRM (in press), 2019

Slide 87

Slide 87 text

Improving U-Net Ye et al, SIAM J. Imaging Science, 2018 Han et al, IEEE Trans. Medical Imaging, 2018 Yoo et al, SIAM J. Applied Math, 2019

Slide 88

Slide 88 text

Problem of U-net Pooling does NOT satisfy the frame condition JC Ye et al, SIAM Journal Imaging Sciences, 2018 Y. Han et al, TMI, 2018. ext > ext = I + > 6= I

Slide 89

Slide 89 text

Improving U-net using Deep Conv Framelets • Dual Frame U-net • Tight Frame U-net JC Ye et al, SIAM Journal Imaging Sciences, 2018 Y. Han and J. C. Ye, TMI, 2018

Slide 90

Slide 90 text

U-Net versus Dual Frame U-Net Y. Han and J. C. Ye, TMI, 2018; Yoo et al, SIJAM, 2018

Slide 91

Slide 91 text

Tight-Frame U-Net JC Ye et al, SIAM Journal Imaging Sciences, 2018

Slide 92

Slide 92 text

Denoising: U-Net vs. Tight-Frame U-Net

Slide 93

Slide 93 text

Inpainting: U-Net vs. Tight-Frame U-Net

Slide 94

Slide 94 text

Style Transfer : Power of Tight Frame U-net

Slide 95

Slide 95 text

No content

Slide 96

Slide 96 text

No content

Slide 97

Slide 97 text

No content

Slide 98

Slide 98 text

No content

Slide 99

Slide 99 text

Summary • Deep learning is a novel signal representation using combinatorial framelets • ReLUs generate multiple linear representation by partitioning the input space • Local Lipschitz controls the global Liptschiz continuity • Skipped connection improves the optimization landscape • Black-box nature of neural networks have been being unveiled.

Slide 100

Slide 100 text

Outlooks q End-to-End AI for radiological imaging Ø From AI-powered image acquisition to diagnosis for clear and rapid radiological imaging Existing AI Solutions: Diagnosis Our future: from acquisition to diagnosis

Slide 101

Slide 101 text

Acknowledgement • Daniel Rueckert (Imperial College) • Florian Knoll (NYU) • Fang Liu (Univ. of Wisconsin) • Mehmet Akcakaya (Univ. of Minnesota) • Dong Liang (SIAT, China) • Dinggang Shen (UNC) • Peder Larson (UCSF) • Grant – NRF of Korea – Ministry of Trade Industry and Energy • Hyunwook Park (KAIST) • Sung-hong Park (KAIST) • Jongho Lee (SNU) • Doshik Hwang (Yonsei Univ) • Won-Jin Moon (KonkukUniv Medical Center) • Eungyeop Kim (Gachon Univ. Medical Center) • Leonard Sunwoo (SNUBH) • Kyuhwan Jung (Vuno)

Slide 102

Slide 102 text

References 1.Ye, Jong Chul, and Woon Kyoung Sung. "Understanding Geometry of Encoder-Decoder CNNs." International Conference on Machine Learning (ICML), 2019. 2.Jong Chul Ye, Yoseob Han and Eunju Cha, "Deep convolutional framelets: a general deep learning framework for inverse problems", SIAM Journal on Imaging Sciences 11(2), 991–1048, 2018. 3. Yoseob Han and Jong Chul Ye,"Framing U-Net via Deep Convolutional Framelets: Application to Sparse-view CT", IEEE Trans. on Medical Imaging, vol. 37, no. 6, pp. 1418-1429, June, 2018. 4. Yoseob Han, Leonard Sunwoo, and Jong Chul Ye, "k-Space Deep Learning for Accelerated MRI", IEEE Trans. on Medical Imaging (in press), 2019 5. Juyoung Lee, Yoseob Han, Jae-Kyun Ryu, Jang-Yeon Park and Jong Chul Ye, "k-Space Deep Learning for Reference-free EPI Ghost Correction", Magnetic Resonance in Medicine (in press), 2019 6. Eunhee Kang, Won Chang, Jaejun Yoo, and Jong Chul Ye,"Deep Convolutional Framelet Denosing for Low-Dose CT via Wavelet Residual Network", IEEE Trans. on Medical Imaging, vol. 37, no.6, pp. 1358-1369, 2018. 7. Eunhee Kang, Junhong Min and Jong Chul Ye, " A Deep Convolutional Neural Network using Directional Wavelets for Low-dose X- ray CT Reconstruction", Medical Physics 44.10 (2017). 8.Jong Chul Ye, Jong Min Kim, Kyong Hwan Jin and Kiryung Lee, "Compressive sampling using annihilating filter-based low-rank interpolation", IEEE Trans. on Information Theory, vol. 63, no. 2, pp.777-801, Feb. 2017. 9. Kyong Hwan Jin, Dongwook Lee, and Jong Chul Ye. "A general framework for compressed sensing and parallel MRI using annihilating filter based low-rank Hankel matrix," IEEE Trans. on Computational Imaging, vol 2, no. 4, pp. 480 - 495, Dec. 2016.