Programming with Kinect for Windows: From Hello World to Lightsabers in 60 minutes

Slides for Introduction to Kinect Programming using C# and WPF

Transcript

1. Programming with Kinect for Windows @latish
2. Latish Sehgal @latish latish.sehgal@ improvingenterprises.com www.dotnetsurfers.com www.dallas-csharp-sig.com
3. ~ 10 years ago
5. June 2009 Project Natal announced
6. June 2009 Project Natal announced Nov 2010 Kinect Launched
8. Feb 2012 Windows Hardware, SDK 1.0 Color, Depth & Audio streams Skeletal Tracking Speech Recognition June 2009 Project Natal announced Nov 2010 Kinect Launched
9. Feb 2012 Windows Hardware, SDK 1.0 May 2012 SDK 1.5 Kinect Studio Face Tracking Seated Skeletal Tracking June 2009 Project Natal announced Nov 2010 Kinect Launched
10. Feb 2012 Windows Hardware, SDK 1.0 May 2012 SDK 1.5 Infrared Stream Accelerometer Data Win8, VS2012 support Oct 2012 SDK 1.6 June 2009 Project Natal announced Nov 2010 Kinect Launched
11. Kinect for Windows
12. Kinect Sensors
13. Viewing Angles
14. Installation •Kinect For Windows SDK •Kinect For Windows Developer Toolkit Optional •Coding4Fun Toolkit •Kinect Toolbox http://kinectforwindows.org
15. Demo
16. Video Stream
17. Resolutions 640 x 480 @ 30fps (RGB) 1280 x 960 @ 12fps (RGB) 640 x 480 @ 15fps (YUV)
18. Demo
19. Depth Stream
20. 640 x 480 320 x 240 80 x 60 @ 30 fps Resolutions
22. depth depthPoint >> DepthImageFrame.PlayerIndexBitmaskWidth player depthPoint & DepthImageFrame.PlayerIndexBitmask 0001100100000 010 depth =800mm player 2
23. Demo
24. Skeletal Tracking
25. 20 Tracked Points
26. 2 Actively Tracked, 6 Total
27. Standing vs Seated Tracking
29. •Each joint has associated tracking state
30. •Each joint has associated tracking state •Tracked, Not tracked or Inferred
31. •Each joint has associated tracking state •Tracked, Not tracked or Inferred •Use TransformSmoothParameters to smooth joint data to reduce jitter
32. Demo
33. Audio Stream
34. Sound Source Angle vs Beam Angle
35. Grammar <!-­‐-­‐  Confirmation_Yes._value:  string   ["Yes"]  -­‐-­‐> <rule  id="Confirmation_Yes"   scope="public">    <example>  yes  </example>    <example>  yes  please  </example>    <one-­‐of>        <item>  yes  </item>        <item>  yeah  </item>        <item>  yep  </item>        <item>  ok  </item>    </one-­‐of>    <item  repeat="0-­‐1">  please  </item>    <tag>  out._value  =  "Yes";</tag> var  grammar  =  new  Choices(); grammar.Add(“yes  please"); grammar.Add(“yes"); grammar.Add(“yeah"); grammar.Add(“ok");  
36. Demo
37. Face Tracking
38. 100 tracked points
39. Head Pose Angles
41. •Real-time tracking of multiple faces
42. •Real-time tracking of multiple faces •2D/3D Mesh and Points
43. •Real-time tracking of multiple faces •2D/3D Mesh and Points •Animation Units
44. Demo
45. Gestures & Poses Install-Package KinectToolbox
46. Algorithmic vs Templated
47. Demo
48. Kinect as a Service kinectservice.codeplex.com
50. •Available via Nuget
51. •Available via Nuget •Support for Windows 8, WPF and Windows Phone
52. Demo
53. Kinect Studio
54. Demo
55. Usability
57. •Review Microsoft’s Human Interface Guidelines
58. •Review Microsoft’s Human Interface Guidelines •Don’t torture your users
59. •Review Microsoft’s Human Interface Guidelines •Don’t torture your users •Best user experiences are context aware
60. •Review Microsoft’s Human Interface Guidelines •Don’t torture your users •Best user experiences are context aware •Complement gestures with speech actions
61. •Review Microsoft’s Human Interface Guidelines •Don’t torture your users •Best user experiences are context aware •Complement gestures with speech actions •Keep a logical relationship between gestures and corresponding actions
62. Watch another Demo, you will
63. Real world Uses
64. Barbie Dream Closet mattel.com
65. Home Automation http://castleos.com/
66. Alex’s Place (Children’s clinic)
67. and many more..
68. Personal Experience
69. Pros
70. •Your non-geek family/friends will think that your work is really cool! Pros
71. •Your non-geek family/friends will think that your work is really cool! •Testing will make you move Pros
72. •Your non-geek family/friends will think that your work is really cool! •Testing will make you move •This changes everything This changes a few things, but in big ways Pros
73. Cons
74. Cons •Upfront hardware investment
75. Cons •Upfront hardware investment •Testing can make you look awkward
76. Cons •Upfront hardware investment •Testing can make you look awkward •Testing will make you move
77. Resources
78. Resources •http://channel9.msdn.com/Series/KinectQuickstart
79. Resources •http://channel9.msdn.com/Series/KinectQuickstart •SDK Samples
80. Resources •http://channel9.msdn.com/Series/KinectQuickstart •SDK Samples •Book “Programming with the Kinect for Windows Software Development Kit” by David Catuhe
81. @latish latish.sehgal@improvingenterprises.com www.dotnetsurfers.com github.com/latish