Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gradient-based Optimization of Time-Multiplexed Binary Computer-Generated Holograms by Digital Mirror Device - Digital Holography and Three-Dimensional Imaging at OSA Imaging and Applied Optics Congress (oral presentation by Kenta Yamamoto)

Gradient-based Optimization of Time-Multiplexed Binary Computer-Generated Holograms by Digital Mirror Device - Digital Holography and Three-Dimensional Imaging at OSA Imaging and Applied Optics Congress (oral presentation by Kenta Yamamoto)

This slide was presented in "Computer Generated Holograms III (DTh7C)" at t OSA Imaging and Applied Optics Congress.
https://www.osa.org/en-us/meetings/osa_meetings/osa_imaging_and_applied_optics_congress/

【Publication】
Yamamoto, Kenta and Ochiai, Yoichi. “Gradient-based Optimization of Time-Multiplexed Binary Computer-Generated Holograms by Digital Mirror Device.” Digital Holography and Three-Dimensional Imaging. Optical Society of America, 2021. (to appear)
https://digitalnature.slis.tsukuba.ac.jp/2021/07/multiple-binary-hologram-optimization-publication/

【Project page】
https://digitalnature.slis.tsukuba.ac.jp/2021/07/multiple-binary-hologram-optimization/

【Presenter】
Kenta Yamamoto (山本健太)
University of Tsukuba
Graduate School of Human Comprehensive Science.
Digital Nature Group (Yoichi Ochiai)

【Abstract】
We propose a gradient-based optimization method for time-multiplexed binary holograms displayed on a digital mirror device. Optimized binary holograms can be used to reconstruct high-quality images.

Digital Nature Group

July 27, 2021
Tweet

More Decks by Digital Nature Group

Other Decks in Research

Transcript

  1. Gradient-based Optimization of Time-Multiplexed Binary
    Computer-Generated Holograms by Digital Mirror Device
    Kenta Yamamoto1, Yoichi Ochiai1
    1University of Tsukuba, Digital Nature Group

    View full-size slide

  2. 2
    Overview of Our Work
    After Aberration Correction
    Process summary diagram of the proposed method.
    "Gradient-based Optimization Method of Time-Multiplexed

    Binary Computer-Generated Holograms".

    View full-size slide

  3. 4
    Background
    After Aberration Correction
    For Displaying 3D Images, Computer-Generated Holography (CGH) is important.
    Holographic display, holographic projector, holographic near-eye display are based on CGH.
    Holographic Display [Yaras 2010] Holographic Projector [Buckley 2011] Holographic Near-Eye Display
    [Maimone 2017]

    View full-size slide

  4. 5
    Background
    After Aberration Correction
    For full-colorization, high refresh rate equipment is essential.
    Phase SLMs are applied for current holographic displays due to the diffraction efficiency.
    However, the refresh rate is still slow (regularly up to 60Hz).
    LCOS SLM
    High Diffraction Efficiency
    Low Refresh Rate
    Digital Mirror Device
    Low Diffraction Efficiency
    High Refresh Rate

    View full-size slide

  5. 6
    Background
    After Aberration Correction
    Therefore, DMDs have the potential for full-color holographic display.
    However, the basic problem is how to obtain a high-definition reproduced image with binary
    holograms because DMD can only display in binary.
    LCOS SLM
    High Diffraction Efficiency
    Low Refresh Rate
    Digital Mirror Device
    Low Diffraction Efficiency
    High Refresh Rate

    View full-size slide

  6. 7
    Background
    After Aberration Correction
    Hologram Binarization Method
    1. Sign Thresholding
    2. Error Diffusion
    3. Direct Binary Search
    4. Iterative Method
    5. Down Sampling
    6. Advanced Down Sampling and Iterative Method

    View full-size slide

  7. 8
    Background
    After Aberration Correction
    Sign Thresholding
    This is a very simple technique.
    If the real part of the original hologram Hc is 0 or more, it is 1, otherwise it is 0.
    Hc(m,n): originam hologram
    Hb(m,n): binarized hologram

    View full-size slide

  8. 9
    Background
    After Aberration Correction
    Error Diffusion
    A method of sharing the error with surrounding pixels.

    1. E0 = 0
    2. H'0 = H0 - E0
    (repeat 3.-5.)
    3. Hout0 = binarize(H'0)
    4. E = Hout0 - H0
    5. H'1 = H1 - E
    ( binarize(): ex. Sign Thresholding )
    Reiner Eschbach. "Comparison of error diffusion methods for computer-generated holograms". 1991.

    View full-size slide

  9. 10
    Background
    After Aberration Correction
    Direct Binary Search
    A method of updating a binary hologram according to the change in MSE
    when each pixel is inverted.
    After verifying all pixels, the process ends.
    Seldowitz, et al. "Synthesis of digital holograms by direct binary search". 1987.

    View full-size slide

  10. 11
    Background
    After Aberration Correction
    Piestun et al. "On-axis binary-amplitude computer generated holograms". 1997.
    Iterative Method
    A method of reaching the optimum hologram while repeating
    propagation by applying restrictions before and after propagation.

    View full-size slide

  11. 12
    Background
    After Aberration Correction
    Down Sampling
    A method of thresholding after down-sampling the target image.
    In "Computer generation of binary Fresnel holography", downsampling is performed periodically.
    Tsang et al. "Computer generation of binary Fresnel holography". 2011.

    View full-size slide

  12. 13
    Background
    After Aberration Correction
    Localized Random Down-Sampling and Adaptive Intensity Accumulation
    For downsampling, a method of randomly selecting pixels is adopted.
    Better hologram optimization is achieved by generating an adaptive mask according to the accumulated intensity.
    Liu et al. "3D display by binary computer-generated holograms with localized random down-sampling and adaptive intensity accumulation". 2020.

    View full-size slide

  13. 14
    Background
    After Aberration Correction
    Gradient-based hologram optimization
    In recent years, gradient-based hologram optimization has achieved high accuracy.
    "3D computer-generated holography
    by non-convex optimization" (2017)
    "High resolution étendue expansion
    for holographic displays" (2020)
    "Wirtinger holography for near-eye displays" (2019)

    View full-size slide

  14. 15
    Background
    After Aberration Correction
    Methods using automatic differentiation have incredible high accuracy algorithms.
    For example: "Neural Holography" and "Diff-PAT".
    Peng et al. "Neural Holography with Camera-in-the-loop Training". 2020.
    Fushimi and Yamamoto et al. "Acoustic hologram optimisation using automatic differentiation". 2021.
    Input:
    φ
    Differentiable
    Propagation
    Calculation: ̂
    f
    Output:
    ̂
    f(ϕ)
    Neural Holography Diff-PAT

    View full-size slide

  15. 16
    Background
    After Aberration Correction
    We aim to realize a high-definition reproduced image with a small number of holograms 

    by combining time-multiplexed binary holograms and gradient-based optimization using
    automatic differentiation.
    Time-Multiplexed Binary Holograms
    Gradient-based Optimization
    using Automatic Differentiation

    View full-size slide

  16. 18
    Method
    After Aberration Correction
    Hologram optimization method using automatic differentiation.
    1. Propagation Calculation from Initial Random Hologram
    2. Calculate Difference (Loss) between Propagated Image and Target Image
    3. Gradient for each pixel is derived
    4. Repeat

    View full-size slide

  17. 19
    Method
    After Aberration Correction
    Impossibility of gradient derivation in binary hologram.
    When the gradient of the binarize step function is derived, it becomes 0 (at t=0, ∞).

    View full-size slide

  18. 20
    Method
    After Aberration Correction
    Tensorflow was used for the automatic differentiation package

    (because it supports 2D Fourier Transform).
    Tensorflow has a function called "custom gradient", which allows to specify the gradient.
    When the gradient of the binarize function is fixed to 1,
    the optimization came to converge.

    View full-size slide

  19. 21
    Method
    After Aberration Correction
    This is the process for time-multiplexed.

    View full-size slide

  20. 23
    Simulation
    After Aberration Correction
    Simulation Results
    (c) original image (d) N=2, PSNR: 22.55 (e) N=3, PSNR: 25.75 (f) N=5 , PSNR: 28.85 (g) N=10, PSNR: 29.25

    View full-size slide

  21. 24
    Simulation
    After Aberration Correction
    Results for each iteration of Loss and PSNR.
    (a) loss and iteration (b) PSNR and iteration
    N=10
    N=5
    N=3
    N=2
    N=10
    N=5
    N=3
    N=2

    View full-size slide

  22. 25
    Experiment
    After Aberration Correction
    Real-environment experiment to confirm optimized hologram.
    Texas Instruments DLP 9000 was used for DMD.
    (g) N=10, PSNR: 29.25 (h) captured image (N=10)
    (c) original image

    View full-size slide

  23. Additional Experiment

    View full-size slide

  24. 27
    Additional Experiment
    After Aberration Correction
    When preparing this presentation, we noticed that a paper about "Differentiable Binarization"
    was published in 2020 in AAAI.
    Pi,j: Input, Ti,j: Threshold,
    Bi,j: Binary Map, k: 50 (empirically derived)
    DB: Differentiable Binarization
    SB: Standard Binarization
    Liao et al. "Real-time Scene Text Detection with Differentiable Binarization". 2020.

    View full-size slide

  25. 28
    Additional Experiment
    After Aberration Correction
    Simulation results optimized using this function.
    Even if this function is used, the binzary hologram is optimized, but the result is better when
    the custom gradient is used.
    PSNR: 21.95
    Five optimized binary holograms.

    View full-size slide

  26. Summary and Future Work

    View full-size slide

  27. 30
    Summary
    After Aberration Correction
    Here is the summary of our work.
    1. A gradient-based optimization method using automatic differentiation was applied to the derivation of time-multiplexed
    binary holograms.
    2. We conducted simulations and real-environment experiments and showed that the optimization results are useful.
    (g) N=10, PSNR: 29.25 (h) captured image (N=10)
    (c) original image

    View full-size slide

  28. 31
    Future Work
    After Aberration Correction
    The following three issues should be addressed in the future.
    1. Because we have not compared it with the existing methods, we will compare it and verify whether it has an advantage.
    2. We will investigate the cause of poor results when using the differentiable binary function.
    3. We will apply to optimization of full-color holograms and build optical systems.

    View full-size slide

  29. Gradient-based Optimization of Time-Multiplexed Binary
    Computer-Generated Holograms by Digital Mirror Device
    Kenta Yamamoto1, Yoichi Ochiai1
    1University of Tsukuba, Digital Nature Group

    View full-size slide