Meet The Microsoft Kinect

[2011] A Presentation to familiarize undergraduate students with the tools available for working with the Microsoft Kinect

Transcript

1. Microsoft Kinect Matthew Roscoe mat.roscoe@unb.ca www.fb.com/matthew.s.roscoe
2. In Association With:
3. What is the “Kinect”? NUI Natural User Interface
4. Meet Kinect (Formerly: “Natal”)
5. What can it give us? ¡  320x240 IR @ 30Hz ¡  640x480 RGB @ 30Hz ¡  640x480 RGBD @ 30Hz ¡  1280x1024 RGB @ 10Hz ¡  1280x1024 RGBD @ 10Hz ¡  57° Horizontal View ¡  43° Vertical View ±28° Vertical Axis Movement ¡  Microphone Array ¡  Speaker System
6. Specifications ¡  RGB Information ¡  IR Information ¡  Depth Information ¡  Audio Array ¡  Operates at 30 Hz ¡  Cost: $130.00!!
7. How does it work? ¡  Microphone Array can capture audio information. ¡  CMOS Camera will give you RGB Photos. ¡  IR Camera will give you IR Spectrum photos (think night vision). ¡  How do we get RGBD?
8. Structured Light ¡  Display a Pattern ¡  Capture Pattern on Surface ¡  Compare capture to known projection ¡  Distortion = Distance!
9. Kinect’s IR Pattern:
10. Getting Distance Values
11. Usability ¡  Usable Distance: 0 – 5 meters ¡  Max Range: 15 meters ¡  57 degree Horizontal view ¡  43 degree Vertical view ¡  640x480 / 1024x768
12. Tools for Using Kinect
13. Kinect for Windows ¡  Produced by Microsoft. (still a beta build) ¡  Supports: ¡  Skeletal Tracking ¡  Facial Recognition ¡  Motion Detection ¡  Voice Recognition ¡  http://www.kinectforwindows.org/
14. http://channel9.msdn.com/Series/KinectSDKQuickstarts
15. Robot Operating System (ROS) ¡  Very Common tool for Robotics Development. ¡  Provides infrastructures for passing messages around between subsystems. ¡  Tends to become cluttered fast. ¡  Good for rapid prototyping and systems demonstrations. ¡  http://www.ros.org
16. ¡  Library in development by Willow Garage for dealing with Point Clouds. ¡  Uses any data captured by RGBD Cameras. ¡  Recently separated from ROS, (1.0 – Standalone PCL). ¡  C/C++/Python bindings. ¡  GPU friendly (VERY important more on this later). ¡  http://www.pointclouds.org
17. GPU Programming ¡  Parallel vs. Serial processing ¡  This means many more threads & processes at our demand! ¡  NVIDIA GeForce 9600M GT has 32 cores! ¡  High End Fermi Cards have 450+ cores. ¡  Intel i7 has 8 cores. ¡  Proven to be many times faster than CPU programs when running native (non optimized code). ¡  Intel was able to prove that GPU programs where only 5x faster than their CPU counterparts.
18. Why use a GPU? ¡  Problems that the Kinect can solve are easily parallelized. ¡  Spreading tasks across multiple cores in order to speed up processing. ¡  Cheaper than CPU’s and in most cases are faster than the CPU’s of equivalent price. ¡  Thrust now provides missing STL functionality on modern GPU’s.
19. Why use a GPU? ¡ Programming is NOT more difficult on a GPU than a CPU after CUDA 4.0 ¡  MOST CPU code compiles in NVCC ¡ No special language required simply pass c/ C++ code through NVCC. ¡ Biggest issue now: ¡  Thinking in terms of parallel processing!
20. 1 10 100 1000 1 2 3 4 5 6 7 8 9 10 Running Time (ms) Planes Detected CPU vs GPU End to End Computations CPU END-To-END GPU END-To-END
21. Real World usage
24. Sunlight
25. Outdoor Use
26. Shadows
27. Kinect & The Real World
28. Applications : Segmentation ¡  Finding objects within every day environments. ¡  Planes ¡  Spheres ¡  Cones ¡  Less Defined Clusters ¡  Many uses within service robotics to find out not just what you are working with but “how” it exists in reality.
29. Applications : Skeletal Tracking ¡  Good for determining peoples positions. ¡  Can located joints and key body part (head, feet, etc). ¡  Uses in Gaming (main use) but many more in medical fields.
30. Applications : SLAM ¡  Simultaneous Location & Mapping ¡  Building up and Environment based on recorded data ¡  Requires offloading from mobile platform.
31. Demonstrations
32. Thank you for your time!
33. @GoDevMental GoDevMental Canadian Technical Students WWW.GODEVMENTAL.CA
35. ¡  Turn your project into a company ¡  Given 20,000 and relocated to Seattle for the summer ¡  http://www.microsoft.com/bizspark/kinectaccelerator/
36. Microsoft Logo here Microsoft Logo Tagline here IMAGINE CUP First Canada, then the World! Check out GoDevMental.com for details
37. Free Registration in Marketplace for Students Dreamspark.com
38. Build 2 quality apps and get a Windows Phone 7.5, code named, “Mango”. For complete details visit microsoft.ca/mangoappchallenge