ExPixel-PixelShader for multiplex-image hiding in consumer 3D flat panels

SIGGRAPH2014のStudent Research Competitionで発表した資料

Transcript

1. ExPixel: PixelShader for multiplex-image hiding in consumer 3D flat panels Hisataka Suzuki 1 – Rex HSIEH 2 – Akihiko Shirai 1 Affiliation: 1ShiraiLab, Kanagawa Institute of Technology 2Carnegie Mellon University http://www.shirai.la/project/expixel
2.  I’ve 3 brothers and 1 sister.  My father is Pakistani, and mother is Japanese.  I’m studying Information Media in Shirai Lab. Introduce myself
3. Concept: Common issue in living room
4.  ExPixel is a Pixel-Shader Technology that can show multiple images on one screen at same time.  ExPixel  Extended Pixel  X-Pol (FPR for 3D)  Extra Image… What is ExPixel
5. Related Works: Multiplex display  Direction based separation  “Bois-Clair” (Gaspar Antoine, 1692)  “Vari-Vue” (Victor Anderson, 1962)  “Lumisight Table”(2005), ”UlteriorScape” (2008) by Kakehi  Time/Shutter based technologies  “Snared illumination” (fakespace lab, 2004)  “ThirdEye” (Pranav Mistry, 2009)  “SimulView” (SONY, 2010-)  Polarization based technologies  “FairyFinder” (Hachiya, 2005)  “Mysterious Pond”, “A visibility control system for collaborative digital table” (Kitamura, Sakurai. 2009)  Hidden but not shown for naked eyes  Patent: JPA2005-092227 (Semiconductor Energy Lab, 2005)
6. Adding a new value of 3D devices. Realizing multiplex-hidden imagery on a consumer 3D flat panel without any hardware modification. Providing content creation environment for wider creators. Challenge
7. Scritter to ExPixel: Scritter NAGANO, K., UTSUGI, T., HIRANO, M., HAMADA, T., SHIRAI,A., AND NAKAJIMA, M. 2010. A new ”multiplex content” displaying system compatible with current 3d projection technology. In ACM SIGGRAPH 2010 Posters, ACM, New York,NY, USA, SIGGRAPH ’10, 79:1–79:1. Multiplex by two projectors.
8. Scritter to ExPixel: ScritterH Wataru FUJIMURA, Yukua KOIDE, Robert SONGER, Takahiro HAYAKAWA, Akihiko SHIRAI, Kazuhisa YANAKA, "2x3D: Real-Time Shader for Simultaneous 2D/3D Hybrid Theater", ACM SIGGRAPH ASIA 2012 Emerging Technologies, Singapore, 2012 Hidden imagery with two projectors.
9. ScritterH, Projector based hidden imagery  Merge an arbitrary image A for naked eyes and an image B that you would like to hide to generate a different image C and multiple- project B and C.  Image A visible to naked eyes while image B with glasses. * Figure from ScritterH project
10.  Naked eye: Left + Filter: Right.  It is Implemented as a GPU based shader.  Good for theater but not suitable for consumer usage. 2x3D: Hybrid 2D + 3D display Wataru FUJIMURA, Yukua KOIDE, Robert SONGER, Takahiro HAYAKAWA, Akihiko SHIRAI, Kazuhisa YANAKA, "2x3D: Real-Time Shader for Simultaneous 2D/3D Hybrid Theater", ACM SIGGRAPH ASIA 2012 Emerging Technologies, Singapore, 2012
11.  Scritter/ScritterH doesn’t need synchronized devices. Only needs polarization filter to separate two channels.  However, it still needs specific products such as DMD projector, polarization supported screen.  It is difficult to prepare two projectors as a stuck. ScritterH, Projector based hidden imagery
12. Algorithm of ExPixel (1/3) With a circular polarization filter Contrast compression
13. Contrast compression 0 255 0 255 input output 0 255 0 255 input output 0 255 0 255 input output a’.rgb = a.rgb × (1.0 - ) + b’.rgb = b.rgb ×
14. Algorithm of ExPixel (2/3) With a circular polarization filter Inversed gamma correction
15.  Obtain image C with considering display gamma. Inversed gamma correction c.rgb = (γ − γ) 1 γ gamma correction
16. Algorithm of ExPixel (3/3) With a circular polarization filter Even odd selection
17. Line-by-line: Even odd selection R L Image A Image B R L R L R L Display P polarization S polarization P polarization S polarization P polarization S polarization Normal Use: Parallax images ExPixe Use: Arbitrary images
18. Line-by-line: Even odd selection Only even lines Only odd lines
19.  Everybody can make multiplex hidden content with Unity 3D Content Creation tool 1) Import Package 2) Set Images 3) Run!!
20. Multi-language presentation Naked Channel (Janpanese) Hidden Channel (English)
21. UbiCode Naked Channel (Advertisement) Hidden Channel (QR code)
22. POLAR: Physical Optical Layered AR AR marker on the main content
23. POLAR: Physical Optical Layered AR AR marker on the hidden channel
24. Various applications Using Scene Naked Eye Hidden Eye Experience Presentation Listener Presenter Only presenter can check scenario. Classroom No hint Hint available Show hints for student who needs hint. Museum Children Adult Explications for different age. Shop Non member Member Show special price for registered member. Sightseeing Children Adult Explications for different interests. Medical Patient Doctor Supplementary information which should not be shown to patients. Municipal services Local residents People from abroad Communication tool between domestic people and foreigner.
25.  Converter  XNA  Static image  Video  Unity  Static image  Video  Game  AR  Presentation  Player Shader model 3.0 Converter  Processing  PHP  Static image  FPGA Hardware Content creation tools
26. Realizing multiplex-hidden imagery on a consumer 3D flat panel without any hardware modification. Providing content creation environment for wider creators. Adding a new value of 3D devices. Conclusion
27. Give me feedback please! ExPixel: PixelShader for multiplex-image hiding in consumer 3D flat panels Hisataka Suzuki 1 – Rex HSIEH 2 – Akihiko Shirai 1 Affiliation: 1ShiraiLab, Kanagawa Institute of Technology 2Carnegie Mellon University http://www.shirai.la/project/expixel
28. Why it cancel the hidden image while canceling image and hidden image are not on the same line? Resolution of display is very high that people can’t recognize pixels. It is not recognized as a pixel, (because it’s same size as pixels!!) ExPixel = Contrast compression + sub pixel
29. Spatial Frequency (Landolt ring)
30. Contrast compression Before After
31.  Contrast  It needs to reduce each image’s contrast according to current algorithm.  Resolution  Vertical resolution became half, but horizontal resolution can be kept actual amount.  Image with black background for Naked Image  After pass contrast compression step, black color level 0 is pushed up to gray color level 127. Weak Points