Interaction with Objects and Humans Based on Visualized Flow Using a Background-Oriented Schlieren Method - HCII2021 Paper Presentation

This presentation was created for the poster session at the HCI International 2021.
http://2021.hci.international/

【Publication】
Shieru Suzuki, Shun Sasaguri, Yoichi Ochiai. (2021) Interaction with Objects and Humans Based on Visualized Flow Using a Background-Oriented Schlieren Method. In: Kurosu M. (eds) Human-Computer Interaction. Design and User Experience Case Studies. HCII 2021. Lecture Notes in Computer Science, vol 12764. Springer, Cham.
https://doi.org/10.1007/978-3-030-78468-3_9
https://digitalnature.slis.tsukuba.ac.jp/2021/07/interaction-based-on-bos-method-hcii2021/

【Project page】
https://digitalnature.slis.tsukuba.ac.jp/2021/03/interaction-based-on-bos-method/

【Presenter】
Shieru Suzuki (鈴木紫琉)
University of Tsukuba
School of Informatics, College of Media Arts, Science and Technology
Digital Nature Group (Yoichi Ochiai)

【Abstract】
Air flow is a ubiquitous phenomenon and is generated under the influences of various objects and phenomena on earth. In this project, we explored interaction methods using a visualization method of the flow field in case studies. A background-oriented schlieren (BOS) method, which visualizes the density field of the flow, was used in this study. Visualizing the air flow around a human body or object demonstrated the ability to provide meaningful information about the target person or object. In addition, by prototyping and testing systems, we discussed the use of the visualized flow to sense the fine airflow and the use of the flow as an input interface.

Transcript

1. © R&D Center for Digital Nature, University of Tsukuba Interaction with Objects and Humans based on Visualized Flow using a Background- oriented Schlieren Method Shieru Suzuki1, Shun Sasaguri1, Yoichi Ochiai1,2 1DIgital Nature Group, University of Tsukuba, 2Pixie Dust Technologies, Inc. 1
2. © R&D Center for Digital Nature, University of Tsukuba 2 Background
3. © R&D Center for Digital Nature, University of Tsukuba 3 Why visualized airflow?
4. © R&D Center for Digital Nature, University of Tsukuba 4 Tactile sensation [Sodhi et al. 2013] Fog display [Alakärppä et al. 2017] Smell display [Seah et al. 2014] Temperature modulation [Han et al. 2017] Levitating spheres [Alrøe et al. 2012] HCI They are not focused on airflow visualization. Why visualized airflow?
5. © R&D Center for Digital Nature, University of Tsukuba Objective Explore the potential for interaction methods with a flow visualization method in the form of case studies 5
6. © R&D Center for Digital Nature, University of Tsukuba Methodology Case studies 1 2 Objects ⁶ Humans Visualized flow ⁶ Objects 6
7. © R&D Center for Digital Nature, University of Tsukuba 7 We use Background Oriented Schlieren as the airflow visualization method (BOS) 1. Why we use BOS? 2. What does BOS actually visualizes? 3. How to perform BOS?
8. © R&D Center for Digital Nature, University of Tsukuba Why BOS (Background-Oriented Schlieren)? 8 LDV, PIV, PTV Physical visualization • Harmful particles • Laser irradiation • Block our view • Non-uniform
 visualization Schlieren • Complex optics Kaessinger, J. C., Kors, K. C., Lum, J. S., Dillon, H. E., & Mayer, S. K. (2014, November). Utilizing Schlieren imaging to visualize heat transfer studies. In ASME International Mechanical Engineering Congress and Exposition (Vol. 46507, p. V005T05A033). American Society of Mechanical Engineers. BOS • No visual obstacle • Uniform visualization • No particles in the air • No Laser use • Simple implementation
9. © R&D Center for Digital Nature, University of Tsukuba Why BOS (Background-Oriented Schlieren)? 9 BOS • No visual obstacle • Uniform visualization • No harmful particles in the air • No Laser use • Simple implementation LDV, PIV, PTV Physical visualization • Harmful particles • Laser irradiation • Block our view • Non-uniform
 visualization Schlieren • Complex optics Kaessinger, J. C., Kors, K. C., Lum, J. S., Dillon, H. E., & Mayer, S. K. (2014, November). Utilizing Schlieren imaging to visualize heat transfer studies. In ASME International Mechanical Engineering Congress and Exposition (Vol. 46507, p. V005T05A033). American Society of Mechanical Engineers.
10. © R&D Center for Digital Nature, University of Tsukuba What is visualized with BOS? 10 Humidity Types and constituents of the gas Temperature Pressure Density Gradient
11. © R&D Center for Digital Nature, University of Tsukuba 11 What is visualized with BOS? In this study Humidity Types and constituents of the gas Temperature Pressure Density Gradient
12. © R&D Center for Digital Nature, University of Tsukuba How to Perform BOS? 12
13. © R&D Center for Digital Nature, University of Tsukuba How to Perform BOS? 13 Initial Image
14. © R&D Center for Digital Nature, University of Tsukuba How to Perform BOS? 14 Initial Image
15. © R&D Center for Digital Nature, University of Tsukuba How to Perform BOS? 15 Initial Image Each picture
16. © R&D Center for Digital Nature, University of Tsukuba How to Perform BOS? 16
17. © R&D Center for Digital Nature, University of Tsukuba How to Perform BOS? 17 Visualized Image
18. © R&D Center for Digital Nature, University of Tsukuba How to Perform BOS? 18 Background Image of Simplified BOS* * Akatsuka, J., Nagai, S.: Flow visualization by a simplified bos technique. In: 29th AIAA Applied Aerodynamics Conference. p. 3653 (2011). S-BOS
19. © R&D Center for Digital Nature, University of Tsukuba 19 Case Studies 1 2 Subtle wind Human eye Visualized flow Meaning Object Objects ⁶ Humans Visualized flow ⁶ Objects
20. © R&D Center for Digital Nature, University of Tsukuba 20 Case1: Result1 Case1: Result2 Case1: Result3 Case1: Result4 Case2: Results & Discussion Conclusion & Future Work Discussion1 Discussion2 Discussion3 Discussion4 Topics
21. © R&D Center for Digital Nature, University of Tsukuba 21 Case1 Visualize Information Human eye
22. © R&D Center for Digital Nature, University of Tsukuba 22 Affordance? “affordances define what actions are possible” “affordances are relationships between object and user” We see human bodies as objects as well. [Norman, 1988]
23. © R&D Center for Digital Nature, University of Tsukuba Case1: Set-up 23
24. © R&D Center for Digital Nature, University of Tsukuba Case1: Results & Discussions 24 Raw Processed Facial Profile Hot Cup Hot & Cold Can Paint
25. © R&D Center for Digital Nature, University of Tsukuba Case1: Results & Discussions (Facial Profile) 25 Raw Processed
26. © R&D Center for Digital Nature, University of Tsukuba Case1: Results & Discussions (Hot Cup) 26 Raw
27. © R&D Center for Digital Nature, University of Tsukuba Case1: Results & Discussions (Hot & Cold Can) 27 Raw
28. © R&D Center for Digital Nature, University of Tsukuba Case1: Results & Discussions (Paint) 28 Raw
29. © R&D Center for Digital Nature, University of Tsukuba Case1: Results & Discussions (Paint) 29 0s Time Variation of Sprayed Paper
30. © R&D Center for Digital Nature, University of Tsukuba Case1: Results & Discussions (Paint) 30 File Paper Paint
31. © R&D Center for Digital Nature, University of Tsukuba 31 Case2 Visualization Subtle wind Marionette
32. © R&D Center for Digital Nature, University of Tsukuba Case2: Set-up 32 Marionette
33. © R&D Center for Digital Nature, University of Tsukuba Case2: Results & Discussions 33
34. © R&D Center for Digital Nature, University of Tsukuba 34 Conclusion and Future Work
35. © R&D Center for Digital Nature, University of Tsukuba 35 Visualized flow based on BOS • gives us meaningful information • extends affordances • The hot plume had some degree of turbulence • The source of the surrounding fine airflow was not limited We conducted several experiments in two case studies to explore the potential for interactions based on S-BOS visualization. Conclusion Objects ⁶ Humans Visualized flow ⁶ Objects
36. © R&D Center for Digital Nature, University of Tsukuba 36 For the development of a sensing method for detecting objects’ movements, • Stabilization of the hot air plumes • A better experimental environment in which undesired disturbances are removed are the challenges. Raw video Processed video Future Work
37. © R&D Center for Digital Nature, University of Tsukuba 37 More detailed emotion recognition than physiological sensors Future Work
38. © R&D Center for Digital Nature, University of Tsukuba 38 Thank you. Email: wasdkey@digitalnature.slis.tsukuba.ac.jp
39. © R&D Center for Digital Nature, University of Tsukuba 39 Appendix
40. © R&D Center for Digital Nature, University of Tsukuba How to Perform BOS? 40 d ~ a f grad(n) / g … α Sensitivity d: f : Focal length of the camera grad(n) : The gradient of refractive index of the medium
41. © R&D Center for Digital Nature, University of Tsukuba How to Perform BOS? 41 Background Image
42. © R&D Center for Digital Nature, University of Tsukuba How to Perform BOS? 42 Calculation [Akatsuka et al. 2011] Luminance value of the pixel at (i, j) Gradient of the luminance of the pixel at (i, j) Output image Measured image Reference image Median of the luminance values Measured image Reference image
43. © R&D Center for Digital Nature, University of Tsukuba How to Perform BOS? 43 Monitor
44. © R&D Center for Digital Nature, University of Tsukuba Case1: Results & Discussions (Hot Cup) 44 S-BOS Thermography
45. © R&D Center for Digital Nature, University of Tsukuba Case2: Set-up 45
46. © R&D Center for Digital Nature, University of Tsukuba Results & Discussions 46 d ~ a f grad(n) / g … α Sensitivity d: • f : Focal length of the camera • grad(n) : The gradient of refractive index of the medium