Interactive Eye Aberration Correction for Holographic Near-Eye Display - Augmented Humans 2021 (Oral presentation by Kenta Yamamoto)

This slide was presented in Session 6 "Augmented Vision" at the Augmented Humans (AHs) International Conference 2021.
https://augmented-humans.org/

【Publication】
Kenta Yamamoto, Ippei Suzuki, Kosaku Namikawa, Kaisei Sato, and Yoichi Ochiai. 2021 Interactive Eye Aberration Correction for Holographic Near-Eye Display. In Proceedings of the Augmented Humans (AHs) International Conference 2021. (to appear)
https://digitalnature.slis.tsukuba.ac.jp/2021/02/interactive-eye-aberration-correction-for-hned_ahs2021/

【Project page】
https://digitalnature.slis.tsukuba.ac.jp/2021/02/interactive-eye-aberration-correction-for-hned/

【Project movie】
https://www.youtube.com/watch?v=lVYIwx6J4Rg

【Presenter】
Kenta Yamamoto (山本健太)
University of Tsukuba
Graduate School of Library, Information and Media Studies
Digital Nature Group (Yoichi Ochiai)

【Abstract】
Distortions of observed images have been a long-standing problem in near-eye displays. Although many correction methods for optical system-dependent aberrations have been proposed, the image distortions caused by eye aberrations have not been studied thoroughly. In addition to the problem, eye aberrations are individual specific. Therefore, a system capable of correcting the aberration irrespective of the individual is necessary. In this study, we propose an aberration-correctable holographic near-eye display (HNED) that can be used to interactively compensate for image distortions caused by eye aberrations. We formulate a propagation equation that includes eye aberrations in the HNED and developed a GUI that enables a user to correct eye aberrations on their own. In this system, the image displayed on the HNED is updated based on the correction coefficients specified by the user. We performed experiments on human subjects to verify the effectiveness of the proposed method. Our results indicate that the minimum identifiable size in our HNED can be reduced by the aberration correction using our interface, and especially our aberration correction method is useful for the visibility of low visual-acuity users.

Transcript

1. Interactive Eye Aberration Correction

   for Holographic Near-Eye Display
   Kenta Yamamoto1, Ippei Suzuki1, Kosaku Namikawa1, Kaisei Sato1, Yoichi Ochiai1,2
   1University of Tsukuba, Digital Nature Group
   2Pixie Dust Technologies, Inc.
   

   View Slide

2. 2
   Overview of This Work
   Before Aberration Correction
   Aberration Correction using User Interface
   After Aberration Correction
   

   View Slide

3. Introduction
   

   View Slide

4. 4
   Distorted Image by Wave Aberration
   with Wave Aberration
   Clear Image Distorted Image
   Problems of Near-Eye Displays:
   Image Distortion, Focus Cues, 3D Image, Parallax, Occlusion, ... etc.
   

   View Slide

5. 5
   Cause of Wave Aberration
   Optical Hardware Human Eye
   This study
   

   View Slide

6. 6
   Previous Work
   [Maimone et al. 2017] [Takaki et al. 2018] [Kim et al. 2019, 2020] [Ours. 2021]
   No GUI with GUI
   

   View Slide

7. 7
   Previous Work
   [Maimone et al. 2017] [Takaki et al. 2018] [Kim et al. 2019, 2020] [Ours. 2021]
   No GUI with GUI
   

   View Slide

8. 8
   Previous Work
   [Maimone et al. 2017] [Takaki et al. 2018] [Kim et al. 2019, 2020] [Ours. 2021]
   No GUI with GUI
   

   View Slide

9. 9
   Previous Work
   [Maimone et al. 2017] [Takaki et al. 2018] [Kim et al. 2019, 2020] [Ours. 2021]
   No GUI with GUI
   

   View Slide

10. Proposed Method
    

    View Slide

11. 11
    Optical Design of Our Holographic Near-Eye Display
    Eyepiece is widening viewing angle
    

    View Slide

12. 12
    Optical Design of Our Holographic Near-Eye Display
    Ureal is propagated from SLM
    

    View Slide

13. 13
    Optical Design of Our Holographic Near-Eye Display
    Uobject plane propagates to SLM
    

    View Slide

14. 14
    Aberration Correction Formulation
    Eye Aberration
    Opposite Aberration
    Calculate Opposite Aberration
    

    View Slide

15. 15
    Aberration Correction Formulation
    𝑢𝑠𝑙𝑚
    = 𝑃𝑟𝑜𝑝𝑎
    (𝑢𝑜𝑏𝑗𝑒𝑐𝑡
    ) ∗ 𝑡(𝑥,𝑦) ∗ P𝑟𝑒𝑣
    (𝑥,𝑦)
    propagation
    from uobject to uslm
    lens phase
    distribution
    general pupil function
    with opposite aberration
    

    View Slide

16. 16
    Aberration Correction Simulation
    7FSUJDBM"TUJHNBUJTN
    XJUI$PNQFOTBUJPO
    
    0CMJRVF"TUJHNBUJTN
    XJUIPVU$PNQFOTBUJPO
    
    0CMJRVF"TUJHNBUJTN
    XJUI$PNQFOTBUJPO
    
    7FSUJDBM"TUJHNBUJTN
    XJUI$PNQFOTBUJPO
    
    0CMJRVF"TUJHNBUJTN
    XJUIPVU$PNQFOTBUJPO
    
    0CMJRVF"TUJHNBUJTN
    XJUI$PNQFOTBUJPO
    
    uslm ureal uretina
    Aberration Correction
    Distorted
    Corrected
    Opposite Distortion
    

    View Slide

17. 17
    Optical Setup & Schematic Diagram
    AL: Aberration reproduction Lens
    

    View Slide

18. 18
    Display Result
    

    View Slide

19. User Interface & User Study
    

    View Slide

20. 20
    User Interface for Aberration Correction
    Oblique Astigmatism Hyperopia or Myopia Vertical or Horizontal
    Astigmatism
    Detail of User Interface
    User Interface with Optical Setup
    

    View Slide

21. 21
    Workflow of Aberration Correction
    B
    "EKVTUJOH1BSBNFUFST
    1IBTF
    *OUFSGBDF
    %JTQMBZ3FTVMUT
    C
    $IFDLJOH*NBHF
    *OUFSGBDF
    Checking Displayed Image
    B
    "EKVTUJOH1BSBNFUFST
    1IBTF
    *OUFSGBDF
    %JTQMBZ3FTVMUT
    C
    $IFDLJOH*NBHF
    

    View Slide

22. 22
    Workflow of Aberration Correction
    B
    "EKVTUJOH1BSBNFUFST
    1IBTF
    *OUFSGBDF
    %JTQMBZ3FTVMUT
    C
    $IFDLJOH*NBHF
    *OUFSGBDF
    B
    "EKVTUJOH1BSBNFUFST
    1IBTF
    *OUFSGBDF
    %JTQMBZ3FTVMUT
    C
    $IFDLJOH*NBHF
    *OUFSGBDF
    B
    "EKVTUJOH1BSBNFUFST
    1IBTF
    *OUFSGBDF
    %JTQMBZ3FTVMUT
    C
    $IFDLJOH*NBHF
    Adjusting Parameters for Aberration Correction
    

    View Slide

23. 23
    Workflow of Aberration Correction
    B
    "EKVTUJOH1BSBNFUFST
    1IBTF
    *OUFSGBDF
    %JTQMBZ3FTVMUT
    C
    $IFDLJOH*NBHF
    *OUFSGBDF
    B
    "EKVTUJOH1BSBNFUFST
    1IBTF
    *OUFSGBDF
    %JTQMBZ3FTVMUT
    C
    $IFDLJOH*NBHF
    Checking Displayed Image
    

    View Slide

24. 24
    Workflow of Aberration Correction
    B
    "EKVTUJOH1BSBNFUFST
    1IBTF
    *OUFSGBDF
    %JTQMBZ3FTVMUT
    C
    $IFDLJOH*NBHF
    *OUFSGBDF
    B
    "EKVTUJOH1BSBNFUFST
    1IBTF
    *OUFSGBDF
    %JTQMBZ3FTVMUT
    C
    $IFDLJOH*NBHF
    *OUFSGBDF
    B
    "EKVTUJOH1BSBNFUFST
    1IBTF
    *OUFSGBDF
    %JTQMBZ3FTVMUT
    C
    $IFDLJOH*NBHF
    Adjusting Parameters for Aberration Correction
    

    View Slide

25. 25
    Workflow of Aberration Correction
    B
    "EKVTUJOH1BSBNFUFST
    1IBTF
    *OUFSGBDF
    %JTQMBZ3FTVMUT
    C
    $IFDLJOH*NBHF
    *OUFSGBDF
    Until the user can get clear image, the user repeats this cycle.
    

    View Slide

26. 26
    User Experiment
    Minimum identifiable gap
    size is measured before
    and after the aberration
    correction.
    18 people participate this user experiment.
    Gap size
    

    View Slide

27. 27
    Result of User Experiment
    Condition 1 Condition 2 Condition 3 Condition 4
    Not Improved Improved Improved Improved
    

    View Slide

28. 28
    Summary of Contributions
    Aberration Correctable Holographic
    Near-Eye Display
    User Interface for Aberration
    Correction Adjustment
    User Study
    

    View Slide

29. Interactive Eye Aberration Correction

    for Holographic Near-Eye Display
    Kenta Yamamoto1, Ippei Suzuki1, Kosaku Namikawa1, Kaisei Sato1, Yoichi Ochiai1,2
    1University of Tsukuba, Digital Nature Group
    2Pixie Dust Technologies, Inc.
    

    View Slide