Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gradient-based Optimization of Time-Multiplexed Binary Computer-Generated Holograms by Digital Mirror Device - Digital Holography and Three-Dimensional Imaging at OSA Imaging and Applied Optics Congress (oral presentation by Kenta Yamamoto)

Gradient-based Optimization of Time-Multiplexed Binary Computer-Generated Holograms by Digital Mirror Device - Digital Holography and Three-Dimensional Imaging at OSA Imaging and Applied Optics Congress (oral presentation by Kenta Yamamoto)

This slide was presented in "Computer Generated Holograms III (DTh7C)" at t OSA Imaging and Applied Optics Congress.
https://www.osa.org/en-us/meetings/osa_meetings/osa_imaging_and_applied_optics_congress/

【Publication】
Yamamoto, Kenta and Ochiai, Yoichi. “Gradient-based Optimization of Time-Multiplexed Binary Computer-Generated Holograms by Digital Mirror Device.” Digital Holography and Three-Dimensional Imaging. Optical Society of America, 2021. (to appear)
https://digitalnature.slis.tsukuba.ac.jp/2021/07/multiple-binary-hologram-optimization-publication/

【Project page】
https://digitalnature.slis.tsukuba.ac.jp/2021/07/multiple-binary-hologram-optimization/

【Presenter】
Kenta Yamamoto (山本健太)
University of Tsukuba
Graduate School of Human Comprehensive Science.
Digital Nature Group (Yoichi Ochiai)

【Abstract】
We propose a gradient-based optimization method for time-multiplexed binary holograms displayed on a digital mirror device. Optimized binary holograms can be used to reconstruct high-quality images.

6d667e747a4a1cda7d201c3358424257?s=128

Digital Nature Group

July 27, 2021
Tweet

Transcript

  1. Gradient-based Optimization of Time-Multiplexed Binary Computer-Generated Holograms by Digital Mirror

    Device Kenta Yamamoto1, Yoichi Ochiai1 1University of Tsukuba, Digital Nature Group
  2. 2 Overview of Our Work After Aberration Correction Process summary

    diagram of the proposed method. "Gradient-based Optimization Method of Time-Multiplexed
 Binary Computer-Generated Holograms".
  3. Background

  4. 4 Background After Aberration Correction For Displaying 3D Images, Computer-Generated

    Holography (CGH) is important. Holographic display, holographic projector, holographic near-eye display are based on CGH. Holographic Display [Yaras 2010] Holographic Projector [Buckley 2011] Holographic Near-Eye Display [Maimone 2017]
  5. 5 Background After Aberration Correction For full-colorization, high refresh rate

    equipment is essential. Phase SLMs are applied for current holographic displays due to the diffraction efficiency. However, the refresh rate is still slow (regularly up to 60Hz). LCOS SLM High Diffraction Efficiency Low Refresh Rate Digital Mirror Device Low Diffraction Efficiency High Refresh Rate
  6. 6 Background After Aberration Correction Therefore, DMDs have the potential

    for full-color holographic display. However, the basic problem is how to obtain a high-definition reproduced image with binary holograms because DMD can only display in binary. LCOS SLM High Diffraction Efficiency Low Refresh Rate Digital Mirror Device Low Diffraction Efficiency High Refresh Rate
  7. 7 Background After Aberration Correction Hologram Binarization Method 1. Sign

    Thresholding 2. Error Diffusion 3. Direct Binary Search 4. Iterative Method 5. Down Sampling 6. Advanced Down Sampling and Iterative Method
  8. 8 Background After Aberration Correction Sign Thresholding This is a

    very simple technique. If the real part of the original hologram Hc is 0 or more, it is 1, otherwise it is 0. Hc(m,n): originam hologram Hb(m,n): binarized hologram
  9. 9 Background After Aberration Correction Error Diffusion A method of

    sharing the error with surrounding pixels. <simple algorithm> 1. E0 = 0 2. H'0 = H0 - E0 (repeat 3.-5.) 3. Hout0 = binarize(H'0) 4. E = Hout0 - H0 5. H'1 = H1 - E ( binarize(): ex. Sign Thresholding ) Reiner Eschbach. "Comparison of error diffusion methods for computer-generated holograms". 1991.
  10. 10 Background After Aberration Correction Direct Binary Search A method

    of updating a binary hologram according to the change in MSE when each pixel is inverted. After verifying all pixels, the process ends. Seldowitz, et al. "Synthesis of digital holograms by direct binary search". 1987.
  11. 11 Background After Aberration Correction Piestun et al. "On-axis binary-amplitude

    computer generated holograms". 1997. Iterative Method A method of reaching the optimum hologram while repeating propagation by applying restrictions before and after propagation.
  12. 12 Background After Aberration Correction Down Sampling A method of

    thresholding after down-sampling the target image. In "Computer generation of binary Fresnel holography", downsampling is performed periodically. Tsang et al. "Computer generation of binary Fresnel holography". 2011.
  13. 13 Background After Aberration Correction Localized Random Down-Sampling and Adaptive

    Intensity Accumulation For downsampling, a method of randomly selecting pixels is adopted. Better hologram optimization is achieved by generating an adaptive mask according to the accumulated intensity. Liu et al. "3D display by binary computer-generated holograms with localized random down-sampling and adaptive intensity accumulation". 2020.
  14. 14 Background After Aberration Correction Gradient-based hologram optimization In recent

    years, gradient-based hologram optimization has achieved high accuracy. "3D computer-generated holography by non-convex optimization" (2017) "High resolution étendue expansion for holographic displays" (2020) "Wirtinger holography for near-eye displays" (2019)
  15. 15 Background After Aberration Correction Methods using automatic differentiation have

    incredible high accuracy algorithms. For example: "Neural Holography" and "Diff-PAT". Peng et al. "Neural Holography with Camera-in-the-loop Training". 2020. Fushimi and Yamamoto et al. "Acoustic hologram optimisation using automatic differentiation". 2021. Input: φ Differentiable Propagation Calculation: ̂ f Output: ̂ f(ϕ) Neural Holography Diff-PAT
  16. 16 Background After Aberration Correction We aim to realize a

    high-definition reproduced image with a small number of holograms 
 by combining time-multiplexed binary holograms and gradient-based optimization using automatic differentiation. Time-Multiplexed Binary Holograms Gradient-based Optimization using Automatic Differentiation
  17. Method

  18. 18 Method After Aberration Correction Hologram optimization method using automatic

    differentiation. 1. Propagation Calculation from Initial Random Hologram 2. Calculate Difference (Loss) between Propagated Image and Target Image 3. Gradient for each pixel is derived 4. Repeat
  19. 19 Method After Aberration Correction Impossibility of gradient derivation in

    binary hologram. When the gradient of the binarize step function is derived, it becomes 0 (at t=0, ∞).
  20. 20 Method After Aberration Correction Tensorflow was used for the

    automatic differentiation package
 (because it supports 2D Fourier Transform). Tensorflow has a function called "custom gradient", which allows to specify the gradient. When the gradient of the binarize function is fixed to 1, the optimization came to converge.
  21. 21 Method After Aberration Correction This is the process for

    time-multiplexed.
  22. Results

  23. 23 Simulation After Aberration Correction Simulation Results (c) original image

    (d) N=2, PSNR: 22.55 (e) N=3, PSNR: 25.75 (f) N=5 , PSNR: 28.85 (g) N=10, PSNR: 29.25
  24. 24 Simulation After Aberration Correction Results for each iteration of

    Loss and PSNR. (a) loss and iteration (b) PSNR and iteration N=10 N=5 N=3 N=2 N=10 N=5 N=3 N=2
  25. 25 Experiment After Aberration Correction Real-environment experiment to confirm optimized

    hologram. Texas Instruments DLP 9000 was used for DMD. (g) N=10, PSNR: 29.25 (h) captured image (N=10) (c) original image
  26. Additional Experiment

  27. 27 Additional Experiment After Aberration Correction When preparing this presentation,

    we noticed that a paper about "Differentiable Binarization" was published in 2020 in AAAI. Pi,j: Input, Ti,j: Threshold, Bi,j: Binary Map, k: 50 (empirically derived) DB: Differentiable Binarization SB: Standard Binarization Liao et al. "Real-time Scene Text Detection with Differentiable Binarization". 2020.
  28. 28 Additional Experiment After Aberration Correction Simulation results optimized using

    this function. Even if this function is used, the binzary hologram is optimized, but the result is better when the custom gradient is used. PSNR: 21.95 Five optimized binary holograms.
  29. Summary and Future Work

  30. 30 Summary After Aberration Correction Here is the summary of

    our work. 1. A gradient-based optimization method using automatic differentiation was applied to the derivation of time-multiplexed binary holograms. 2. We conducted simulations and real-environment experiments and showed that the optimization results are useful. (g) N=10, PSNR: 29.25 (h) captured image (N=10) (c) original image
  31. 31 Future Work After Aberration Correction The following three issues

    should be addressed in the future. 1. Because we have not compared it with the existing methods, we will compare it and verify whether it has an advantage. 2. We will investigate the cause of poor results when using the differentiable binary function. 3. We will apply to optimization of full-color holograms and build optical systems.
  32. Gradient-based Optimization of Time-Multiplexed Binary Computer-Generated Holograms by Digital Mirror

    Device Kenta Yamamoto1, Yoichi Ochiai1 1University of Tsukuba, Digital Nature Group