Programming with Kinect for Windows: From Hello World to Lightsabers in 60 minutes

Transcript

1. Programming with Kinect for Windows @latish

2. Latish Sehgal @latish latish.sehgal@ improvingenterprises.com www.dotnetsurfers.com www.dallas-csharp-sig.com

3. ~ 10 years ago

4. None

5. June 2009 Project Natal announced

6. June 2009 Project Natal announced Nov 2010 Kinect Launched

7. None

8. Feb 2012 Windows Hardware, SDK 1.0 Color, Depth & Audio

   streams Skeletal Tracking Speech Recognition June 2009 Project Natal announced Nov 2010 Kinect Launched

9. Feb 2012 Windows Hardware, SDK 1.0 May 2012 SDK 1.5

   Kinect Studio Face Tracking Seated Skeletal Tracking June 2009 Project Natal announced Nov 2010 Kinect Launched

10. Feb 2012 Windows Hardware, SDK 1.0 May 2012 SDK 1.5

    Infrared Stream Accelerometer Data Win8, VS2012 support Oct 2012 SDK 1.6 June 2009 Project Natal announced Nov 2010 Kinect Launched

11. Kinect for Windows

12. Kinect Sensors

13. Viewing Angles

14. Installation •Kinect For Windows SDK •Kinect For Windows Developer Toolkit

    Optional •Coding4Fun Toolkit •Kinect Toolbox http://kinectforwindows.org

15. Demo

16. Video Stream

17. Resolutions 640 x 480 @ 30fps (RGB) 1280 x 960

    @ 12fps (RGB) 640 x 480 @ 15fps (YUV)

18. Demo

19. Depth Stream

20. 640 x 480 320 x 240 80 x 60 @

    30 fps Resolutions
21. None

22. depth depthPoint >> DepthImageFrame.PlayerIndexBitmaskWidth player depthPoint & DepthImageFrame.PlayerIndexBitmask 0001100100000 010

    depth =800mm player 2

23. Demo

24. Skeletal Tracking

25. 20 Tracked Points

26. 2 Actively Tracked, 6 Total

27. Standing vs Seated Tracking

28. None

29. •Each joint has associated tracking state

30. •Each joint has associated tracking state •Tracked, Not tracked or

    Inferred

31. •Each joint has associated tracking state •Tracked, Not tracked or

    Inferred •Use TransformSmoothParameters to smooth joint data to reduce jitter

32. Demo

33. Audio Stream

34. Sound Source Angle vs Beam Angle

35. Grammar <!-­‐-­‐  Confirmation_Yes._value:  string   ["Yes"]  -­‐-­‐> <rule  id="Confirmation_Yes"  

    scope="public">    <example>  yes  </example>    <example>  yes  please  </example>    <one-­‐of>        <item>  yes  </item>        <item>  yeah  </item>        <item>  yep  </item>        <item>  ok  </item>    </one-­‐of>    <item  repeat="0-­‐1">  please  </item>    <tag>  out._value  =  "Yes";</tag> var  grammar  =  new  Choices(); grammar.Add(“yes  please"); grammar.Add(“yes"); grammar.Add(“yeah"); grammar.Add(“ok");  

36. Demo

37. Face Tracking

38. 100 tracked points

39. Head Pose Angles

40. None

41. •Real-time tracking of multiple faces

42. •Real-time tracking of multiple faces •2D/3D Mesh and Points

43. •Real-time tracking of multiple faces •2D/3D Mesh and Points •Animation

    Units

44. Demo

45. Gestures & Poses Install-Package KinectToolbox

46. Algorithmic vs Templated

47. Demo

48. Kinect as a Service kinectservice.codeplex.com

49. None

50. •Available via Nuget

51. •Available via Nuget •Support for Windows 8, WPF and Windows

    Phone

52. Demo

53. Kinect Studio

54. Demo

55. Usability

56. None

57. •Review Microsoft’s Human Interface Guidelines

58. •Review Microsoft’s Human Interface Guidelines •Don’t torture your users

59. •Review Microsoft’s Human Interface Guidelines •Don’t torture your users •Best

    user experiences are context aware

60. •Review Microsoft’s Human Interface Guidelines •Don’t torture your users •Best

    user experiences are context aware •Complement gestures with speech actions

61. •Review Microsoft’s Human Interface Guidelines •Don’t torture your users •Best

    user experiences are context aware •Complement gestures with speech actions •Keep a logical relationship between gestures and corresponding actions

62. Watch another Demo, you will

63. Real world Uses

64. Barbie Dream Closet mattel.com

65. Home Automation http://castleos.com/

66. Alex’s Place (Children’s clinic)

67. and many more..

68. Personal Experience

69. Pros

70. •Your non-geek family/friends will think that your work is really

    cool! Pros

71. •Your non-geek family/friends will think that your work is really

    cool! •Testing will make you move Pros

72. •Your non-geek family/friends will think that your work is really

    cool! •Testing will make you move •This changes everything This changes a few things, but in big ways Pros

73. Cons

74. Cons •Upfront hardware investment

75. Cons •Upfront hardware investment •Testing can make you look awkward

76. Cons •Upfront hardware investment •Testing can make you look awkward

    •Testing will make you move

77. Resources

78. Resources •http://channel9.msdn.com/Series/KinectQuickstart

79. Resources •http://channel9.msdn.com/Series/KinectQuickstart •SDK Samples

80. Resources •http://channel9.msdn.com/Series/KinectQuickstart •SDK Samples •Book “Programming with the Kinect for

    Windows Software Development Kit” by David Catuhe

81. @latish latish.sehgal@improvingenterprises.com www.dotnetsurfers.com github.com/latish