PyKinect: Body Iteration Application Development Using Python

Introduction to the PyKinect library and the Kinect for Windows device.

Transcript

1. Eric ShangKuan (ericsk) Technical Evangelist Microsoft Taiwan Corporation
2. Agenda • Kinect for Windows • PyKinect: How-to • References
3. Kinect for Windows (K4W)
4. Kinect for Windows
5. How Kinect Works
6. Skeleton Tracking
7. Detailed Spec Resolution 1280 x 960 (RGB Camera) Viewing angle 43° vertical by 57° horizontal field of view Tilt range ±27° Frame rate 30fps Audio format 16KHz, 24-bit PCM Audio input A four-microphone array with 24-bit analog-to-digital converter (ADC) and Kinect-resident signal processing including acoustic echo cancellation and noise suppression # of tracking users 4 # of tracking points 20
8. Different from XBox360 Kinect • Near mode – Enables the camera to see objects as close as 40 centimeters in front of the device without losing accuracy or precision, with graceful degradation out to 3 meters. • Shortening USB cable and small dongle • Support and software updates
9. Kinect Software Development • Windows 7/8 RP • Visual Studio 2010 • C++ or C# (w/ .NET 4.0) • Kinect for Windows SDK (1.5) • (optional) DirectX • (optional) Microsoft Speech Platform Runtime • (optional) Kinect for Windows Language Pack
10. K4W HW/SW Interaction
11. K4W SDK Architecture 1. Kinect hardware 2. Kinect drivers 3. Audio and Video Components 4. DirectX Media Object (DMO) for microphone array beam-forming and audio source localization. 5. Win32 standard APIs
12. PyKinect
13. PyKinect Project • A Python bindings of K4W SDK. – K4W SDK is required. • A part of Python Tools for Visual Studio (PTVS) open source project. • http://pytools.codeplex.com/wikipage?title=PyKi nect
14. Enable PyKinect • CPython 2.7 (32-bit) • Kinect for Windows SDK (1.5 is OK) • 2 ways to install PyKinect – Install PTVS and PTVS – PyKinect Sample , then install PyKinect through PTVS menu. – Install through PyPI: http://pypi.python.org/pypi/pykinect/1.0 • (optional) PyGame
15. How to use PyKinect? from pykinect import nui # Create a kinect instance kinect = nui.Runtime() # Enable the skeleton engine kinect.skeleton_engine.enabled = True # skeleton frame ready event handling kinect.skeleton_frame_ready += skeleton_frame_ready # depth frame ready event handling kinect.depth_frame_ready += depth_frame_ready # video frame ready event handling kinect.video_frame_ready += video_frame_ready
16. Skeleton Tracking • Each joint (關節) is represented by 3-axis coordination (x, y, z) – RH axis • Use JointId to ensure where the joint is.
17. Get Skeleton Data def skeleton_frame_ready(skeleton_frame): skeletons = skeleton_frame.SkeletonData # index for users for index, data in enumerate(skeletons): # get head position head_positions = data.SkeletonPositions[JointId.Head] # handle limbs - left arm LEFT_ARM = (JointId.ShoulderCenter, JointId.ShoulderLeft, JointId.ElbowLeft, JointId.WristLeft, JointId.HandLeft) left_arm_positions = [data.SkeletonPositions[x] for x in LEFT_ARM]
18. Demo – Draw Skeletons
19. References • Kinect for Windows SDK Programming Guide & References: http://msdn.microsoft.com/en-us/library/hh855347 • Check-out the PyKinect source: http://pytools.codeplex.com/SourceControl/list/changes ets