Blind-Badminton: A Working Prototype to Recognize Position of Flying Object for Visually Impaired Users - HCII2021 Paper Presentation

This presentation was created for the poster session at the HCI International 2021.
http://2021.hci.international/

【Publication】
Masaaki Sadasue, Daichi Tagami, Sayan Sarcar, Yoichi Ochiai. (2021) Blind-Badminton. In: Antona M., Stephanidis C. (eds) Universal Access in Human-Computer Interaction. Access to Media, Learning and Assistive Environments. HCII 2021. Lecture Notes in Computer Science, vol 12769. Springer, Cham. https://doi.org/10.1007/978-3-030-78095-1_36
https://digitalnature.slis.tsukuba.ac.jp/2021/07/blind-badminton-hcii2021//

【Project page】
https://digitalnature.slis.tsukuba.ac.jp/2021/07/blind-badminton/

【Presenter】
Masaaki Sadasue (貞末真明)
University of Tsukuba
Graduate School of Comprehensive Human Science
Digital Nature Group (Yoichi Ochiai)

【Abstract】
This paper proposes a system for recognizing flying objects during a ball game in blind sports. “Blind sports” is a term that refers to a sport for visually impaired people. There are various types of blind sports, and several of these sports, such as goalball and blind soccer, are registered in the Paralympic Games. This study specifically aimed at realizing games similar to badminton. Various user experiments were conducted to verify the requirements for playing sports that are similar to badminton without visual stimulus. Additionally, we developed a system that provides users with auxiliary information, including height, depth, left and right directions, and swing delay by adopting sound feedback via a binaural sound source, as well as haptic feedback via a handheld device.This study evaluated several conditions including that of a balloon owing to its slow falling speed and adopted a UAV drone that generates flight sounds on its own and adjusts its speed and trajectory during the course of the games. To evaluate playability, this study focused on three points: a questionnaire following the experiment, the error in the drone’s traveling direction, and the racket’s swing direction. From the play results and answers of the questionnaire, it was determined that the users were able to recognize the right and left directions, as well as the depth of the drone using the noise generated by the drone, and that this approach is playable in these situations.

Transcript

1. Blind-Badminton A Working Prototype to Recognize Position of Flying Object for Visually Impaired Users Masaaki Sadasue, Daichi Tagami, Sayan Sarcar, and Yoichi Ochiai
2. Outline: 1.Introduction 2.Related Work 3.Implementation 4.Experiment 1.User Study1 2.User Study2 5.Discussion / Future Work 6.Conclusion
3. Introduction
4. Introduction: Ordinary sports Conventional Idea： • Sound source into ball • Increase the number of bounce • Alternative act for visually impaired people So, what about sports like badminton? 1
5. Introduction: Problem The range of sports that visually impaired people can participate in is narrower than that of sighted. 2
6. Introduction: Purpose 3 By developing a feedback system using motion capture, we will verify to what extent people with limited visual information were able to guess using the racket. We believe we can contribute to the development of future racquet sports for the visually impaired.
7. Introduction: Model sports is badminton 4 • It is an indoor sport, so it is not affected by the weather. • One-on-one matches are less likely to get hurt. • No need for bouncing.
8. Related Work
9. Related Work: Most of the research on sports for visually impaired people to date has involved analysis of movement.[1] In addition, there is research to make competitions like badminton possible by using virtual space.[2] 6 [2] Research to realize virtual badminton by calculating the trajectory of the shuttle from the swing on a PC using a unique racket Shin Kim, Kun-pyo Lee, and Tek-Jin Nam. 2016. Sonic-Badminton: Audio-Augmented Badminton Game for Blind People. In <i>Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems</i> (<i>CHI EA '16</i>). Association for Computing Machinery, New York, NY, USA, 1922–1929. DOI:https://doi.org/10.1145/2851581.2892510 [1] Research to enable motion learning by presenting the movements of other players through tactile and auditory feedback. Makoto Kobayashi and Hisayuki Tatsumi. 2020. Floor-Volleyball Motion Feedback System for Visually Impaired Players. In <i>2020 12th International Conference on Education Technology and Computers</i> (<i>ICETC'20</i>). Association for Computing Machinery, New York, NY, USA, 46–50. DOI:https://doi.org/10.1145/3436756.3437020 However, in the real world, there is no research that can recognize a flying ball and make it possible to rally.
10. Related Work : Exergame Exergame： Exergame, a video game that allows users to control their avatars in accordance with their movements to improve their physical performance, such as Wii Sports, has been modified for the visually impaired to encourage exercise.[3] Participants don't move from in front of the TV, and it becomes less effective as a full-body workout.[4] 7 [3] Research to increase physical activity by allowing visually impaired children to experience the Wii sport of tennis. Tony Morelli, John Foley, Luis Columna, Lauren Lieberman, and Eelke Folmer. 2010. VI- Tennis: a vibrotactile/audio exergame for players who are visually impaired. In <i>Proceedings of the Fifth International Conference on the Foundations of Digital Games</i> (<i>FDG '10</i>). Association for Computing Machinery, New York, NY, USA, 147–154. DOI:https://doi.org/10.1145/1822348.1822368 [4] Exergame's Effect on Whole Body Exercise Research Anthony Morelli. 2010. Haptic/audio based exergaming for visually impaired individuals. <i>SIGACCESS Access. Comput.</i>, 96 (January 2010), 50–53. DOI:https://doi.org/10.1145/1731849.1731859
11. Related Work : Position of this study 8 To develop a real-world racquet sports for the visually impaired, investigate how well a player can recognize the position of flying objects by giving various feedback.
12. Implementation
13. Implementation: Motion captures 11 Markers on participant and flying object for motion capture. Use motion capture to track each movement.
14. Implementation: Flying objects 9 1. Shuttlecock -Using in badminton scene 2. Balloon -Slower speed than shuttlecock 3. Drone -Next slide
15. Implementation: Drone 10 Tello from DJI The Advantages are： • Small • Control of its speed and orbit • Emit flight noise
16. Implementation: Overall view 12 Syncing Motion Capture Video with Unity Use various systems of feedback to allow the player to recognize the position of flying objects.
17. Implementation: Haptic feedback 13 Strong vibrations are given when the altitude of the flying object is high, and weak vibrations are given when the altitude is low. The roll of this feedback (Joy-Con from Nintendo switch): -Depending on the height of the flying object obtained by motion capture, vibration feedback is given to the participant.
18. Implementation: Sound feedback 14 A 5kHz sinusoidal binaural sound, which varies according to the distance and direction of the flying object from the participant The roll of this feedback (a binaural sound source with earphones): -Depending on the distance and direction between the flying object and participant obtained by motion capture, sound feedback is given to the participant. When the sound localization in the horizontal direction exceeds 4 kHz, a 5kHz sine wave exceeding 4 kHz is adopted to follow the azimuth well.
19. Experiment
20. Experiment: 15 User Study1: an experiment that adopts the proposed system in a shuttlecock and balloon User Study2: an experiment that adopts this system in a drone
21. Experiment: Purpose of User Study1 16 -By using the system, we created for shuttles and balloons without flight sounds, we verified how well the participants could recognize the flying objects and hit them with their rackets. -We also conducted a similar study on drones that were not using any system.
22. Experiment: Procedure of User Study1 17 Five Patterns: 1) balloons and binaural sound sources, 2) balloons, binaural sound sources, and Joy-Con, 3) shuttlecocks and binaural sound sources, 4) shuttlecocks, binaural sound sources, and JoyCon, and 5) drones. • The order of the experiments was randomized by the participants • As a tutorial, participants practiced earphones and vibration five times. • In the experiment with the drone, the drone was moved after a voice cue.
23. Experiment: Participants of User Study1 18 Five sighted people (one female and four males), aged between 20 and 23 years, participated in the experiment. They were used blindfolded to limit their visual information.
24. Experiment: Result of User Study1 19
25. Experiment: Result of User Study1 20
26. Experiment: Purpose of User Study2 21 Most participants in User Study1 were unable to react and hit the racket because the shuttle and balloon were moving too fast. This User Study2 applies the system to a drone to support sound source localization.
27. Experiment: Procedure of User Study2 22 Five Patterns: 1) balloons and binaural sound sources, 2) balloons, binaural sound sources, and Joy-Con, 3) shuttlecocks and binaural sound sources, 4) shuttlecocks, binaural sound sources, and JoyCon, and 5) drones. • The order of the experiments was randomized by the participants • As a tutorial, participants practiced earphones and vibration five times. • In the experiment with the drone, the drone was moved after a voice cue.
28. Experiment: Participants of User Study2 23 Twelve participants (1 female and 11 males), aged between 19 and 23 years They were used blindfolded to limit their visual information.
29. Experiment: Result of User Study2 24
30. Experiment: Result of User Study2 25
31. Discussion / Future Work
32. Discussion / Future Work: 26 • it is very possible that the results will be significantly different for actual low vision or totally blind people. • This experiment is also likely to vary significantly, depending on whether the disability is intermediate or congenital. • It is possible that by having the participants perform this experiment for a long period of time Based on these possibilities, to validate the study, it is necessary to conduct multiple experiments by preparing actual subjects and altering the duration and conditions
33. Discussion / Future Work: 27 Motion Capture: these equipment are too expensive to install in actual sport venues. -Because the purpose of this project is to expand the range of sports, this challenge is considerably a critical issue.
34. Conclusion
35. Conclusion: 29 Purpose: By developing a feedback system using motion capture, we will verify to what extent people with limited visual information were able to guess using the racket. Result: This study determines that the sound source localization is most accurate when the experiment is conducted with a single drone. Discussion: -To realize this, we also need to address the challenge of using a racket to hit the drone. -These equipment are too expensive to install in actual sport venues. -It is necessary to conduct multiple experiments by preparing actual subjects and altering the duration and conditions
36. Thank you for your listening