Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Deep Learning for CT Reconstruction

Jong Chul Ye
February 24, 2018

Deep Learning for CT Reconstruction

Keynote Talk by Jong Chul Ye, SPIE Medical Imaging 2018, Feb, 24, 2018, Houston, USA

Jong Chul Ye

February 24, 2018
Tweet

More Decks by Jong Chul Ye

Other Decks in Research

Transcript

  1. Jong Chul Ye KAIST Bio-Imaging & Signal Processing Lab. Dept.

    Bio & Brain Engineering KAIST, Korea D De ee ep p L Le ea ar rn ni in ng g f fo or r C CT T R Re ec co on ns st tr ru uc ct ti io on n
  2. 2

  3. * NMSE is wri9en at the corner. 2 2. .1

    12 20 03 3e e- -1 1 X : Input (48) Ground truth (380th) Han and Ye, arxiv:1611.06391,2016, Jin et al, IEEE TIP, 2017 U-Net for Sparse View CT 6 6. .2 22 27 77 7e e- -3 3 U-Net
  4. •  Successful demonstraMon of deep learning for various image reconstrucMon

    problems –  Low-dose x-ray CT (Kang et al, Chen et al, Wolterink et al) –  Sparse view CT (Jin et al, Han et al, Adler et al) –  Interior tomography (Han et al) –  Stationary CT for baggage inspection (Han et al) –  CS-MRI (Hammernik et al, Yang et al, Lee et al, Zhu et al) –  US imaging (Yoon et al ) –  Diffuse optical tomography (Yoo et al) •  Advantages –  Very fast reconstruction time –  Significantly improved results Other works
  5. Why Deep Learning works for recon ? Existing views 1:

    unfolded iteration •  Most prevailing views •  Direct connecMon to sparse recovery –  Cannot explain the filter channels Jin, arXiv:1611.03679
  6. Why Deep Learning works for recon ? Existing views 2:

    generative model •  Image reconstruc5on as a distribu5on matching –  However, difficult to explain the role of black-box network Bora et al, Compressed Sensing using Generative Models, arXiv:1703.03208
  7. Still Unresolved Problems.. •  What is the role of the

    nonlinearity such as rectified linear unit (ReLU) ? •  What is the role of the filter channels in convolutional layer ? •  Why do we need a pooling and unpooling in some architectures ? •  Why do some networks need fully connected layers whereas the others do not ? •  What is the role of by-pass connection or residual network ?
  8. Our Proposal: Deep Learning == Deep Convolutional Framelets •  Ye

    et al, “Deep convolutional framelets: A general deep learning framework for inverse problems”, SIAM Journal Imaging Sciences (in press), 2018.
  9. Why we are excited about Hankel matrix ? T -T

    0 n1 -n1 0 * FRI Sampling theory (Ve9erlie et al) and compressed sensing
  10. ︙ ︙ 1 2 3 4 5 6 7 8

    9 -1 0 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 12 2 3 4 5 6 7 8 9 12 13 3 4 5 6 7 8 9 10 10 10 10 0 11 11 11 1 2 3 4 5 Finite length convolution Matrix Representation * ALOHA : Annihilating filter based LOw rank Hankel matrix Approach * Jin KH et al. IEEE TCI, 2016 * Jin KH et al.,IEEE TIP, 2015 * Ye JC et al. IEEE TIT, 2016 Annihilating filter-based low-rank Hankel matrix
  11. Missing elements can be found by low rank Hankel structured

    matrix compleMon Nuclear norm Projec5on on sampling posi5ons min m kH ( m ) k⇤ subject to P⌦(b) = P⌦( f ) Rank H ( f ) = k * Jin KH et al IEEE TCI, 2016 * Jin KH et al.,IEEE TIP, 2015 * Ye JC et al., IEEE TIT, 2016 m Annihilating filter-based low-rank Hankel matrix
  12. Deep Convolutional Framelets (Y, Han, Cha; 2018) H d( f

    ) H d( f ) = ˜ T ˜ T C C = Φ T H d( f ) C = Φ T ( f ~ ) Encoder: ˜ T = I ˜ = PR(V ) H d( f ) = U ⌃ V T Decoder: f = (˜ Φ C ) ~ ⌧ (˜ ) : Non-local basis : Local basis : Generalized pooling : CNN filters : Frame condition : Projection condition
  13. Low-Dose CT •  To reduce the radiation exposure, sparse-view CT,

    low-dose CT and interior tomography. Sparse-view CT (Down-sampled View) Low-dose CT (Reduced X-ray dose) Interior Tomography (Truncated FOV)
  14. WavResNet for low-dose CT (Kang, Yoo, Y; 2017) E. Kang

    et al, Medical Physics, vol. 44, no. 10, 2017. E. Kang et al, arXiv preprint arXiv:1707.09938
  15. Sparse-View CT •  To reduce the radiation exposure, sparse-view CT,

    low-dose CT and interior tomography. Sparse-view CT (Down-sampled View) Low-dose CT (Reduced X-ray dose) Interior Tomography (Truncated FOV)
  16. Problem of U-net Pooling does NOT satisfy the frame condition

    JC Ye et al, SIAM Journal Imaging Sciences, 2018 Y. Han and J. C. Ye, arXiv preprint arXiv:1708.08333, 2017. ext > ext = I + > 6 = I
  17. Improving U-net using Deep Conv Framelets •  Dual Frame U-net

    •  Tight Frame U-net JC Ye et al, SIAM Journal Imaging Sciences, 2018 Y. Han and J. C. Ye, arXiv preprint arXiv:1708.08333, 2017.
  18. 90 view recon U-Net vs. Tight-Frame U-Net •  JC Ye

    et al, SIAM Journal Imaging Sciences, 2018 •  Y. Han and J. C. Ye, arXiv preprint arXiv:1708.08333, 2017.
  19. 1st view 2nd view 3rd view 4th view 5th view

    6th view 7th view 8th view 9th view
  20. FBP

  21. TV

  22. Interior Tomography for ROI Reconstruction •  To reduce the radiation

    exposure, sparse-view CT, low-dose CT and interior tomography. Sparse-view CT (Down-sampled View) Low-dose CT (Reduced X-ray dose) Interior Tomography (Truncated FOV)
  23. Conclusions: •  Deep learning has demonstrated significant performance improvement in

    CT reconstruction problems •  Deep convolutional framelets is a mathematical framework to understand deep learning for inverse problems –  Novel data-driven signal representation –  Structured low-rank shrinkage by controlling the no. of filter channels –  Frame condition is important •  Our analysis shows that “correctly designed” deep network is NOT a black-box and is SAFE for medical imaging reconstruction