showing excitement. • Mapping to Buildings. • Mapping to clothing, face, notes and other familiar objects. • Artist's concert. etc… Fig 1 : Tokyo Disneyland Once upon a time. Fig 2 : Perfume Cannes Lions International Festival of Creativity
mapping, since viewers enjoy mostly by watching projected images, feeling of immersion in viewers’ contents is insufficient. • We thought that it would be possible to solve the problem if the viewer became a participant. → Propose projects to participants and propose interactive projection mapping that changes with the participant’s movements.
to synchronize real and virtual. • To paste the image material on the surface layer of the projection target. • It is being used in a wide range of fields ranging from entertainment to medical treatment.
for Windows is a peripheral device that enables operation by body movement, gesture and voice without using a controller. • Projector (NEC NP50J) Used for projection Fig 3 : Kinect for Windows. Infrared sensor RGB camera Depth image sensor Microphone
Integration : Visual Studio 2017 • Language : C++ • Library • OpenNI2 • NiTE2 • OpenCV • OpenGL Used to retrieve data from Kinect Used for image acquisition, output, drawing
Fig 10. → Pose detection. Fig 9 : “Skeleton” screen display. Fig 10 : “Skeleton” number. Center of the Head Center of the Body Neck Left waist Left shoulder Right waist Right shoulder Left knee Left elbow Right knee Right elbow Left foot Left hand Right foot Right hand
of the subject by projecting "Combination" to the subject. • Baseball mode (Pitcher’s movement) • Soccer mode (Lifting movement) We implemented baseball and football movements for right-handed people.
skeleton becomes unstable. → It may switch to an unintended mode. We pose “Attention”. • When a series of operations of the mode is completed, initialization is performed. → Continuous projection possible. • Before switching to each mode. → The screen remains black.
baseball balls and baseball stadiums are projected. • As the flight distance increases, the ball gradually decreases. • When the position of the ball becomes larger than the threshold value of the !-coordinate the ball disappears and is initialized.
of “Combination” when playing baseball (No background). 1. Hold hands around chest. 2. Roll up so that the right hand is above the head, the ball appears and starts to move.
soccer balls and soccer stadiums are projected. • when the position of the ball is lower than the !-coordinate threshold the ball will disappear and be initialized.
of “Combination” when playing soccer (No background). 1. When the right knee is kicked up so as to be as high as the right waist, the ball appears. 2. When the right knee is returned to its original position, the ball falls.
uses the following physical operation. • The !-coordinate upward is positive and the center of the screen of the projection mapping is set to ! = 0. Fig 13 : Physical operation used to implement the movement of lifting. y ! = $% & − 1 2 *&+ ! = $′% & − 1 2 *&+ take ! = 25 ∗ & − / + *&+ for $% = 25 $% , $′% : Initial velocity & : Time * : Gravitational acceleration (* = 9.8)
of the ball at the time of kicking up the ball. ∴ "# = 2&ℎ ∴ "′# = 4&ℎ In order to perform a continuous lifting operation, the highest point of the ball at and the height of the ball kicked up again are made equal. "* − "# * = 2 −& ℎ The speed of the highest point is 0 − "# * = 2 −& ℎ from " = 0 from "* − "- # * = 2 −& (2ℎ) The speed of the highest point is 0 − "- # * = 2 −& (2ℎ) from " = 0 from 0 = 35.4 ∗ 5 − 1 2 &5* "′# = 2"# "′# = 25 ∗ 2 ≈ 35.4 from the above y Fig 13 : Physical operation used to implement the movement of lifting.
to understand? Q2, The person who answered that it was hard to understand. → What point was hard to understand? Q3, Is there a feeling of immersion? Q4, Do you have any opinions or improvements? Q1 and Q3 were evaluated in 5 grades. Good evaluation is 5, bad evaluation is 1.
to understand? → The average score is 4.75 Q3, Is there a feeling of immersion? → The average score is 3.67 0 1 2 3 4 5 operation method Immersion feeling Average score of the questionnaire Fig 15 : Average score of the questionnaire.
or improvements? Improvement • Please change the speed of the ball according to the speed of operation. • I want you to add sound. • The movement of the ball is slightly behind. • I wish I could play more than one person in this system. • I want the image to change with not tired action.
simple operation, high evaluation was obtained. → Many people can enjoy this projection mapping. • Regarding the feeling of immersion, We got the result that it is roughly the case, We also learned that there is still room for improvement.
those looking at projection mapping, but not for those who do it." → We think that because the subject cannot actually see the projection mapping. • “The movement of the ball is slightly behind.” → It is thought that there is a time lag between when Kinect recognizes the skeleton and reflects it in the projection mapping.
left-handed people. • Addition of other movements of baseball / football such as batting and shooting. • Make it available for multiple people. • Add other events. • Texture mapping tailored to the event. • Play the sound when the ball is hit, change the speed.
the movement of people increases like this in this way .... More realistic stage production. • In the future, We would like to realize the projection mapping which further increased the immersion feeling based on the improvement point. Thank you very much for kind attention.
takashinoda
nwiizo
grimoh
ufried
masakiokuda
hinakko
raybugg
o3
yutailang0119
twada
snoozer05
matsuji
shimastripe
addyosmani
bermonpainter
mraible
jacobian
techseoconnect
lara
pedronauck
nmsamuel
katiecoart
csswizardry
chrisshort
