Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Microsoft Kinect & Robotics

Microsoft Kinect & Robotics

[2011] A presentation that introduces the Microsoft Kinect & its possible uses in the field of robotics.

Mat Roscoe

March 19, 2012
Tweet

More Decks by Mat Roscoe

Other Decks in Education

Transcript

  1. Specifications ¡  RGB Information ¡  IR Information ¡  Depth Information

    ¡  Audio Array ¡  Operates at 30 Hz ¡  Cost: €130.00!!
  2. How does it work? ¡  Microphone Array can capture audio

    information. ¡  CMOS Camera will give you RGB Photos. ¡  IR Camera will give you IR Spectrum photos (think night vision). ¡  How do we get RGBD?
  3. Getting the RGBD ¡  Kinect provides the ability to capture

    Depth Information. ¡  Projected IR pattern. ¡  Objects distort pattern. ¡  Amount of distortion = distance. ¡  Only problem is that this is an assumption Microsoft and PrimeSense are very tight lipped about how this actually works!
  4. Usability ¡  Usable Distance: 0 – 5 meters ¡  Max

    Range: 15 meters ¡  57 degree Horizontal view ¡  43 degree Vertical view ¡  640x480 / 1024x768
  5. Robot Operating System (ROS) ¡  Very Common tool for Robotics

    Development. ¡  Provides infrastructures for passing messages around between subsystems. ¡  Tends to become cluttered fast. ¡  Good for rapid prototyping and systems demonstrations.
  6. ¡  Library in development by Willow Garage for dealing with

    Point Clouds. ¡  Uses any data captured by RGBD Cameras. ¡  Recently separated from ROS, (1.0 – Standalone PCL). ¡  C/C++/Python bindings. ¡  Recently started becoming GPU friendly.
  7. Applications Segmentation ¡  Finding objects within every day environments. ¡ 

    Planes ¡  Spheres ¡  Cones ¡  Less Defined Clusters ¡  Many uses within service robotics to find out not just what you are working with but “how” it exists in reality.
  8. GPU Programming ¡  Parallel vs. Serial processing ¡  This means

    many more threads & processes at our demand! ¡  NVIDIA GeForce 9600M GT has 32 cores! ¡  High End Telsa has 470+ cores. ¡  Proven to be many times faster than CPU programs when running native (non optimized code). ¡  Intel was able to prove that GPU programs where only 5x faster than their CPU counterparts.
  9. Why use a GPU? ¡  Problems that the Kinect can

    solve are easily parallelized. ¡  Spreading tasks across multiple cores in order to speed up processing. ¡  Cheaper than CPU’s and in most cases are faster than the CPU’s of equivalent price. ¡  Thrust now provides missing STL functionality on modern GPU’s.
  10. Why use a GPU? ¡ Programming is NOT more difficult on

    a GPU than a CPU after CUDA 4.0 ¡  MOST CPU code compiles in NVCC ¡ No special language required simply pass c/ C++ code through NVCC. ¡ Biggest issue now: ¡  Thinking in terms of parallel processing!
  11. PCL + Parallel Processing ¡  Currently in the dev trunk

    of PCL (scheduled Fall 2011 release). ¡  Kinect Data Capture is now written in CUDA. ¡  Normal Estimation is now done in CUDA. ¡  Segmentation (RANSAC) will be parallelized by PCL 2.0! ¡  Why should we care? …
  12. Parallel PCL: Normal Estimation Standard PCL ¡  Run time: ~1100

    ms ¡  Frame rate of ~0.8 Hz ¡  Hard to use for service robotics with such performance metrics. Parallel PCL ¡  Run Time: ~ 45 ms ¡  Frame Rate: 22 Hz ¡  Short of real time, but much more useable than CPU. ¡  24x faster
  13. Parallel PCL: RANSAC ¡  RANdom Sample And Concensus ¡  Very

    good at finding data in “dirty” data sets. ¡  Rather slow algorithm.
  14. Parallel PCL: RANSAC Standard PCL ¡  Run time: 80ms –

    1000ms ¡  Frame rate of ~13Hz – 1Hz ¡  Again poor performance. ¡  Usable but must be done intelligently. Parallel PCL ¡  Run time: ~3ms ¡  Frame rate of ~333 Hz ¡  Much Higher Performance. ¡  Runs well within the real time constraint. ¡  3% GPU Usage on 570Fermi ¡  3% CPU Usage
  15. Future Work ¡  Working on getting parallel plane segmentation functioning.

    ¡  Test the robustness with other geometric forms.
  16. Notes: Test Computer Specs CPU ¡  Intel Core 2 Duo

    ¡  Cores: 2 ¡  Clock Speed: 2.93 GHz ¡  Memory: 8GB ¡  SSD GPU ¡  NVIDIA GeForce 9600M GT ¡  Cores: 32 ¡  Clock Speed: 120 MHz ¡  Memory: 512MB VRAM ¡  SSD
  17. References ¡  http://www.adafruit.com/blog/2010/11/04/kinect-tear- down/ ¡  http://www.ifixit.com/Teardown/Microsoft-Kinect- Teardown/4066/1 ¡  http://www.primesense.com/?p=515 ¡ 

    Martin A Fishcler and Robert C Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Communications of the ACM, vol. 24, no. 6, pp. 381 - 395, 1981. ¡  http://www.pointclouds.org ¡  http://www.ros.org ¡  http://developer.nvidia.com/category/zone/cuda-zone ¡  http://nicolas.burrus.name/index.php/Research/ KinectRgbDemoV5