Materialization of Motions - SIGGRAPH 2017 Experience Presentation

This slide was presented at SIGGRAPH 2017 Experience Presentation "Materialization of Motion, Levitation, and Magnetics".
http://s2017.siggraph.org/experience-presentations/sessions/materialization-motion-levitation-and-magnetics

Materialization of Motions: Tangible Representation of Dance Movements for Learning and Archiving
https://doi.org/10.1145/3084863.3084869

【Project page】
http://digitalnature.slis.tsukuba.ac.jp/2017/06/materialization-of-motions/

【Project movie】
https://www.youtube.com/watch?v=RyWszfMIUAg

【Presenter】
Mose Sakashita (坂下申世)
University of Tsukuba
School of Informatics
College of Media Arts, Science and Technology
Digital Nature Group (Yoichi Ochiai Lab.)
http://www.u.tsukuba.ac.jp/~s1411453

【Abstract】
We propose a method to learn and archive dance movements by fabricating tangible three-dimensional (3D) human forms. We analyze the pa‹ern of the tempo and rhythm of a music piece and fabricate the shape of a 3D body based on the motions of the dance performer by using a 3D printer. For the implementation, we employ a depth camera to capture 3D information of a dance movement. Appropriate movements are extracted from the €le at every constant tempo by analyzing the tempo of the music piece played while the dance is performed. ‘e 3D printer enables tangible modeling of the dance movements.

1. Materialization of Motions Tangible Representation of Dance Movements for Learning and Archiving Mose Sakashita, Kenta Suzuki, Keisuke Kawahara, Kazuki Takazawa, Yoichi Ochiai
2. Motivation and Introduction “Notate Motions”
3. Introduction
4. Laban et al. 1928Labanotation Joan and Rudolf Benesh. 1962 Benesh Movement Notation Motivation : Notation of Choreography
5. Motivation : Common Method to Learn Choreography 2D displays (e.g. laptop, tablet, smartphone) Photos, books
6. 2D displays (e.g. laptop, tablet, smartphone) Photos, books 2D Information Motivation : Common Method to Learn Choreography
7. “How to notate dance choreography as tangible 3D representation”
8. Related Work
9. Related Work: Expanding the Expression of Dance Motions Augmented Performance Lighting Choreographer Haptic Feedback Sparacino et al. 1999 Fujimoto et al. 2011 Shibasaki et al. 2016
10. Related Work: Supportive systems to train dancing A Virtual Reality Dance Training System Chan et al. 2011 A Kinect Interface for Ballet Dancers Marquardt et al. 2011 CAVE Virtual Reality Environment Muneesawang et al. 2015 A Virtual Reality Dance Training System Fujimoto et al. 2012 Teach Me to Dance Charbonneau et al. 2011 2D display VRCAVE
11. Design “Tangible Representations Beyond Displays”
12. z Tangibility of the posture during dance Ability to change the position of the 3D object 3D point of view Design: Three merits of 3D tangible choreography
13. Design: Tangibility of the posture during dance
14. Design: Ability to change the position of the 3D object
15. Design: 3D point of view
16. Static Figures Zoetrope
17. Static Figures
18. Static Figures Zoetrope
19. Zoetrope
20. Implementation “From dance movements to tangible representatins”
21. Implementation : the procedure of implementation Motion Capture Sound Analysis 3D print Tangible Motion
22. Implementation : Capturing choreography
23. Procedure : Analysis of the tempo and extraction of models
24. Implementation: Zoetrope
25. BPM of a music : 150 bpm Rotation per minute : 75 rpm Frame per rotation : 24 frames Beat per rotation : 2 beats The conditions of our zoetrope Frame per second : 30 fps Implementation: Zoetrope
26. BPM of a music : b Rotation per minute : c Frame per rotation : d Beat per rotation : a Frame per second : f RPM FPS Implementation: Zoetrope
27. Pilot Study
28. Purpose To understand how 3D-printed tangible representations affect learning dancing. Study process We asked the participants to remember choreography in 5 minutes. Participants used both 2D-display and 3D tangible figures. We asked them to give us feedback on the methods. Participants This study involved 11 participants (3 females, 8 males) including participants who have learned dancing. Pilot Study: purpose and process
29. Free description P1 “By using the tangible movements, I could understand the position of foots easily.” P3 “The tangible movements allow me to understand the rhythm, but the video was necessary to understand the movements between the rhythms” P11 “The tangible forms were easy to understand the dance movements”
30. Contribution and Future Work
31. 1. We propose a method to learn and archive dance choreography by printing tangible human forms. 2. We suggest a method to extract key frames of the dance movements by using the tempo of the music. 3. We designed two applications: Static Figures for touching and changing formations, Zoetrope for rendering 3D animation of the choreography. Contribution
32. 1. Further study on blind people to figure out how helpful the tangible human form is to learn or enjoy the tangible movements. 2. Combination between this method and the motion tracking system with a single RGB-camera. 3. Printing out computationally generated dance choreography. Future Work
33. Mose Sakashita University of Tsukuba Undergrads Kenta Suzuki University of Tsukuba Undergrads Kazuki Takazawa University of Tsukuba Master Keisuke Kawahara University of Tsukuba Master Yoichi Ochiai University of Tsukuba Assistant Professor
34. Experience at SIGGRAPH 2017
35. Experience at SIGGRAPH 2017
36. 30+ Danced! Experience at SIGGRAPH 2017 people
37. Any questions?

Transcript

1. Materialization of Motions Tangible Representation of Dance Movements for Learning and Archiving Mose Sakashita, Kenta Suzuki, Keisuke Kawahara, Kazuki Takazawa, Yoichi Ochiai
2. Motivation and Introduction “Notate Motions”
3. Introduction
4. Laban et al. 1928 Labanotation Joan and Rudolf Benesh. 1962 Benesh Movement Notation Motivation : Notation of Choreography
5. Motivation : Common Method to Learn Choreography 2D displays (e.g. laptop, tablet, smartphone) Photos, books
6. 2D displays (e.g. laptop, tablet, smartphone) Photos, books 2D Information Motivation : Common Method to Learn Choreography
7. “How to notate dance choreography as tangible 3D representation”
8. Related Work
9. Related Work: Expanding the Expression of Dance Motions Augmented Performance Lighting Choreographer Haptic Feedback Sparacino et al. 1999 Fujimoto et al. 2011 Shibasaki et al. 2016
10. Related Work: Supportive systems to train dancing A Virtual Reality Dance Training System Chan et al. 2011 A Kinect Interface for Ballet Dancers Marquardt et al. 2011 CAVE Virtual Reality Environment Muneesawang et al. 2015 A Virtual Reality Dance Training System Fujimoto et al. 2012 Teach Me to Dance Charbonneau et al. 2011 2D display VR CAVE
11. Design “Tangible Representations Beyond Displays”
12. z Tangibility of the posture during dance Ability to change the position of the 3D object 3D point of view Design: Three merits of 3D tangible choreography
13. Design: Tangibility of the posture during dance
14. Design: Ability to change the position of the 3D object
15. Design: 3D point of view
16. Static Figures Zoetrope
17. Static Figures
18. Static Figures Zoetrope
19. Zoetrope
20. Implementation “From dance movements to tangible representatins”
21. Implementation : the procedure of implementation Motion Capture Sound Analysis 3D print Tangible Motion
22. Implementation : Capturing choreography
23. Procedure : Analysis of the tempo and extraction of models
25. Implementation: Zoetrope
26. BPM of a music : 150 bpm Rotation per minute : 75 rpm Frame per rotation : 24 frames Beat per rotation : 2 beats The conditions of our zoetrope Frame per second : 30 fps Implementation: Zoetrope
27. BPM of a music : b Rotation per minute : c Frame per rotation : d Beat per rotation : a Frame per second : f RPM FPS Implementation: Zoetrope
29. Pilot Study
30. Purpose To understand how 3D-printed tangible representations affect learning dancing. Study process We asked the participants to remember choreography in 5 minutes. Participants used both 2D-display and 3D tangible figures. We asked them to give us feedback on the methods. Participants This study involved 11 participants (3 females, 8 males) including participants who have learned dancing. Pilot Study: purpose and process
31. Free description P1 “By using the tangible movements, I could understand the position of foots easily.” P3 “The tangible movements allow me to understand the rhythm, but the video was necessary to understand the movements between the rhythms” P11 “The tangible forms were easy to understand the dance movements”
32. Contribution and Future Work
33. 1. We propose a method to learn and archive dance choreography by printing tangible human forms. 2. We suggest a method to extract key frames of the dance movements by using the tempo of the music. 3. We designed two applications: Static Figures for touching and changing formations, Zoetrope for rendering 3D animation of the choreography. Contribution
34. 1. Further study on blind people to figure out how helpful the tangible human form is to learn or enjoy the tangible movements. 2. Combination between this method and the motion tracking system with a single RGB-camera. 3. Printing out computationally generated dance choreography. Future Work
35. Mose Sakashita University of Tsukuba Undergrads Kenta Suzuki University of Tsukuba Undergrads Kazuki Takazawa University of Tsukuba Master Keisuke Kawahara University of Tsukuba Master Yoichi Ochiai University of Tsukuba Assistant Professor
36. Experience at SIGGRAPH 2017
37. Experience at SIGGRAPH 2017
38. 30+ Danced! Experience at SIGGRAPH 2017 people
39. Any questions?