1996 and 1998, respectively Ph.D. degree from Osaka University, Japan, in 2001 Was with Nara Institute of Science and Technology, Japan, 2001-2007 Was with Osaka University, Japan, 2007-2011 Have been with Shizuoka University, Japan, since 2011 My interests are in applications and development of CMOS image sensors for biomedical imaging and computational imaging Email: kagawa@idl.rie.shizuoka.ac.jp Self Introduction
+ Rich digital signal processing circuits Pixel array with in-pixel rich processing circuits Low-level focal-plane processing without in- pixel circuits Simple and small pixel Complicated and large pixel Post-processing Reduced data Results Results Results Conventional Charge-domain signal processing Smart pixel
Hyper-spectrum camera A lot of wavelength spectral bands Lensless camera Ultra-thin X-ray CT Lower X-ray exposure or shorter acquisition time … Examples of CS applications
Complete charge transfer Floating diffusion amplifier Low dark current Noiseless charge handling High conversion gain p n n + Depletion region p + SiO2
al. (2016)] Add the address line +High sensitivity High resolution - Temporal constraint Share the control line +High sensitivity High resolution -Spatial constraint [Luo et al. (2019)] In-pixel shutter memory +No constraint -Lower sensitivity Lower resolution
Observation period Multi-aperture compressive A1 A2 A3 A4 Observation period Conventional burst readout Observation period F1 F2 F3 F4 Time Efficient sampling
maximum frame rate. More frames or extended depth range is obtained in single-shot filming and time-of-flight depth imaging, respectively. Benefits of CS in Ultra-High-Speed Filming
camera Developed CMOS image sensor PD for stop trigger 10x objective lens Pulse laser (λ=1064nm) Specimen (SPCC) Plasma image by the RGB camera Motorized stage
width of 7ns) Mirror Developed CMOS image sensor Aperture Light source and camera Captured scene Panel (13.0m) Doll (8.0m) Pylon (6.0m) Pillar (4.0m) Lens Diffuser
something with something Moving something down Real Input Model Top 1 Top 3 Top 5 Video (upper bound) C3D 71.0 88.0 88.0 Single image Coded exposure (Proposed) SVC2D 72.0 84.0 88.0 Long exposure C2D 20.0 40.0 52.0 Short exposure C2D 21.0 47.0 60.0 Courtesy of Prof. Nagahara, Osaka Univ. Video Compressed Covering something with something Moving something down
processing is introduced to compress temporal optical signals. Temporally compressive ultra-high-speed CMOS image sensor based on the inner-product operation has been fabricated. Single-shot filming and multi-exposure-based transient imaging have been demonstrated at a burst frame rate of 303Mfps. Extended depth range and separation of dual-path components have been demonstrated. Applications of DNN such as one-path image reproduction and motion recognition from a compressed image were introduced. Summary
Wakin, “An introduction to compressive sampling,” IEEE Signal Processing Magazine, Vol. 25, pp. 21-30 (2008). (compressive video1) Y. Hitomi, J. Gu, M. Gupta, T. Mitsunaga, and S. K. Nayar, “Video from a single coded exposure photograph using a learned over-complete dictionary,” ICCV (2011). (compressive video2) T. Sonoda, H. Nagahara, K. Endo, Y. Sugiyama, and R. Taniguchi, “High-speed imaging using CMOS image sensor with quasi pixel-wise exposure,” ICCP (2016). (compressive video3) Y. Luo, J. Jiang, M. Cai, and S. Mirabbasi, “CMOS computational camera with a two-tap coded exposure image sensor for single-shot spatial-temporal compressive sensing,” Optics Express, Vol. 27, pp. 31475-31489 (2019). (spatial CS) Y. Oike and A. E. Gamal, “A 256x256 CMOS image sensors with ΔΣ-based single-shot compressed sensing,” IEEE ISSCC Dig. Tech. Papers, pp. 386-387(2012). (UHS1) T. Arai, J. Yonai, T. Hayashida, H. Ohtake, H. van Kujik, and T. G. Etoh, “A 252-V/lux・s, 16.7-million- frames-per-second 312-kpixel back-side-illuminated ultrahigh-speed charge-coupled device,” IEEE Electron Devices, Vol. 60, pp. 3450-3458(2013). (UHS2) Y. Tochigi, et al., “A global-shutter CMOS image sensor with readout speed of 1Tpixel/s burst and 780Mpixel/s continuous,” ISSCC Dig. Tech. Papers, pp. 382-383 (2012). (UHS3) M. Suzuki, et al., “Over 100 Million Frames per Second 368 Frames Global Shutter Burst CMOS Image Sensor with In-Pixel Trench Capacitor Memory Array,” Proc. IISW, pp. 266-269 (2019). (UHS4) T. Etoh, T. Okinaka, Y. Takano, K. Takehara, H. Nakano, K. Shimonomura, T. Ando, N. Ngo, Y. Kamakura, V. Dao, A. Nguyen, E. Charbon, C. Zhang, P. Moor, P. Goeschalckx, and L. Haspeslagh, “Light-in- flight imaging by a silicon image sensor: toward the theoretical highest frame rate,” MDPI Sensors, Vol. 19, Article 2247 (2019).
S. Han, M. Seo, K. Yasutomi, and K. Kagawa, “CMOS lock-in pixel image sensors with lateral electric field control for time-resolved imaging,” Int'l Image Sensor Workshop, pp. 361-364 (2013). (LEFM2) M. Seo, K. Kagawa, K. Yasutomi, T. Takasawa, Y. Kawata, N. Teranishi, Z. Li, I.A. Halin, and S. Kawahito, “10.8ps-time-resolution 256x512 image sensor with 2-tap true-CDS lock-in pixels for fluorescence lifetime imaging,” IEEE ISSCC Dig. Tech. Papers, pp. 198-199 (2015). (LEFM3) M. Seo, Y. Shirakawa, Y. Masuda, Y. Kawata, K. Kagawa, K. Yasutomi, S. Kawahito, “A programmable sub-nanosecond time-gated 4-tap lock-in pixel CMOS image sensor for real-time fluorescence lifetime imaging microscopy,” ISSCC Dig. Tech. Papers, pp. 70-71 (2017). (LEFM4) Y. Shirakawa, K. Yasutomi, K. Kagawa, S. Aoyama, and S. Kawahito, “An 8-tap CMOS lock-in pixel image sensor for short-pulse time-of-flight measurements,” MDPI Sensors, Vol. 20, Article 1040 (2020). (Our work1) F. Mochizuki, K. Kagawa, S. Okihara, M. Seo, Z. Bo, T. Takasawa, K. Yasutomi, and S. Kawahito, “Single-shot 200Mfps 5x3-aperture compressive CMOS imager,” IEEE ISSCC Dig. Tech. Papers, pp. 116- 117(2015). (Our work2) K. Kagawa, T. Kokado, Y. Sato, F. Mochizuki, H. Nagahara, T. Takasawa, K. Yasutomi, and S. Kawahito, “Multi-tap Macro-Pixel Based Compressive Ultra-High-Speed CMOS Image Sensor,” Proc. IISW, pp. 270-273 (2019). (CS action recognition) T. Okawara, M. Yoshida, H. Nagahara, and Y. Yagi, “Action recognition from a single coded image,” ICCP (2020). (DNN compressive video) M. Yoshida, A. Torii, M. Okutomi, K. Endo, Y. Sugiyama, R. Taniguchi, and H. Nagahara, “Joint optimization for compressive video sensing and reconstruction under hardware constraints,” ECCV (2018).