Pro Yearly is on sale from $80 to $50! »

Meet The Microsoft Kinect

Meet The Microsoft Kinect

[2011] A Presentation to familiarize undergraduate students with the tools available for working with the Microsoft Kinect

67d75baf4175b2bf5aa544dca6ea9c2b?s=128

Mat Roscoe

March 19, 2012
Tweet

Transcript

  1. Microsoft Kinect Matthew Roscoe mat.roscoe@unb.ca www.fb.com/matthew.s.roscoe

  2. In Association With:

  3. What is the “Kinect”? NUI Natural User Interface

  4. Meet Kinect (Formerly: “Natal”)

  5. What can it give us? ¡  320x240 IR @ 30Hz

    ¡  640x480 RGB @ 30Hz ¡  640x480 RGBD @ 30Hz ¡  1280x1024 RGB @ 10Hz ¡  1280x1024 RGBD @ 10Hz ¡  57° Horizontal View ¡  43° Vertical View ±28° Vertical Axis Movement ¡  Microphone Array ¡  Speaker System
  6. Specifications ¡  RGB Information ¡  IR Information ¡  Depth Information

    ¡  Audio Array ¡  Operates at 30 Hz ¡  Cost: $130.00!!
  7. How does it work? ¡  Microphone Array can capture audio

    information. ¡  CMOS Camera will give you RGB Photos. ¡  IR Camera will give you IR Spectrum photos (think night vision). ¡  How do we get RGBD?
  8. Structured Light ¡  Display a Pattern ¡  Capture Pattern on

    Surface ¡  Compare capture to known projection ¡  Distortion = Distance!
  9. Kinect’s IR Pattern:

  10. Getting Distance Values

  11. Usability ¡  Usable Distance: 0 – 5 meters ¡  Max

    Range: 15 meters ¡  57 degree Horizontal view ¡  43 degree Vertical view ¡  640x480 / 1024x768
  12. Tools for Using Kinect

  13. Kinect for Windows ¡  Produced by Microsoft. (still a beta

    build) ¡  Supports: ¡  Skeletal Tracking ¡  Facial Recognition ¡  Motion Detection ¡  Voice Recognition ¡  http://www.kinectforwindows.org/
  14. http://channel9.msdn.com/Series/KinectSDKQuickstarts

  15. Robot Operating System (ROS) ¡  Very Common tool for Robotics

    Development. ¡  Provides infrastructures for passing messages around between subsystems. ¡  Tends to become cluttered fast. ¡  Good for rapid prototyping and systems demonstrations. ¡  http://www.ros.org
  16. ¡  Library in development by Willow Garage for dealing with

    Point Clouds. ¡  Uses any data captured by RGBD Cameras. ¡  Recently separated from ROS, (1.0 – Standalone PCL). ¡  C/C++/Python bindings. ¡  GPU friendly (VERY important more on this later). ¡  http://www.pointclouds.org
  17. GPU Programming ¡  Parallel vs. Serial processing ¡  This means

    many more threads & processes at our demand! ¡  NVIDIA GeForce 9600M GT has 32 cores! ¡  High End Fermi Cards have 450+ cores. ¡  Intel i7 has 8 cores. ¡  Proven to be many times faster than CPU programs when running native (non optimized code). ¡  Intel was able to prove that GPU programs where only 5x faster than their CPU counterparts.
  18. Why use a GPU? ¡  Problems that the Kinect can

    solve are easily parallelized. ¡  Spreading tasks across multiple cores in order to speed up processing. ¡  Cheaper than CPU’s and in most cases are faster than the CPU’s of equivalent price. ¡  Thrust now provides missing STL functionality on modern GPU’s.
  19. Why use a GPU? ¡ Programming is NOT more difficult on

    a GPU than a CPU after CUDA 4.0 ¡  MOST CPU code compiles in NVCC ¡ No special language required simply pass c/ C++ code through NVCC. ¡ Biggest issue now: ¡  Thinking in terms of parallel processing!
  20. 1 10 100 1000 1 2 3 4 5 6

    7 8 9 10 Running Time (ms) Planes Detected CPU vs GPU End to End Computations CPU END-To-END GPU END-To-END
  21. Real World usage

  22. None
  23. None
  24. Sunlight

  25. Outdoor Use

  26. Shadows

  27. Kinect & The Real World

  28. Applications : Segmentation ¡  Finding objects within every day environments.

    ¡  Planes ¡  Spheres ¡  Cones ¡  Less Defined Clusters ¡  Many uses within service robotics to find out not just what you are working with but “how” it exists in reality.
  29. Applications : Skeletal Tracking ¡  Good for determining peoples positions.

    ¡  Can located joints and key body part (head, feet, etc). ¡  Uses in Gaming (main use) but many more in medical fields.
  30. Applications : SLAM ¡  Simultaneous Location & Mapping ¡  Building

    up and Environment based on recorded data ¡  Requires offloading from mobile platform.
  31. Demonstrations

  32. Thank you for your time!

  33. @GoDevMental GoDevMental Canadian Technical Students WWW.GODEVMENTAL.CA

  34. None
  35. ¡  Turn your project into a company ¡  Given 20,000

    and relocated to Seattle for the summer ¡  http://www.microsoft.com/bizspark/kinectaccelerator/
  36. Microsoft Logo here Microsoft Logo Tagline here IMAGINE CUP First

    Canada, then the World! Check out GoDevMental.com for details
  37. Free Registration in Marketplace for Students Dreamspark.com

  38. Build 2 quality apps and get a Windows Phone 7.5,

    code named, “Mango”. For complete details visit microsoft.ca/mangoappchallenge