Robocalypse - Full Stack Toronto 2015

Ec1016a9a63da65998b75f35b6ceb3bb?s=47 Joel Lord
November 14, 2015

Robocalypse - Full Stack Toronto 2015

Robocalypse: Controlling Nodebots with Kinect Gestures. A #fstoconf15 talk by Joel Lord

Ec1016a9a63da65998b75f35b6ceb3bb?s=128

Joel Lord

November 14, 2015
Tweet

Transcript

  1. confidential not for distribution A Full Stack 2015 Toronto presentation

    by Joel Lord Robocalypse: Controlling Nodebots with Kinect gestures
  2. Overview About Me Why The Goals What Exists What I

    wanted Gestures 101 Demo
  3. About Me

  4. About Me Joel Lord Development Manager Javascript junkie, technology enthusiast,

    IoT hobbyist. Macadamian http://www.macadamian.com Twitter: @joel__lord Github: github.com/joellord
  5. We’re hiring ! http://www.macadamian.com/careers

  6. Houston… We’ve got a problem.

  7. Why ?

  8. Why •  Worked with Nodebots for a while now

  9. Why •  Worked with Nodebots for a while now • 

    Wanted to explore other possibilities of NodeJs
  10. Why •  Worked with Nodebots for a while now • 

    Wanted to explore other possibilities of NodeJs •  Because controlling a robot with a Kinect is pretty cool
  11. The Goals

  12. The Goals •  Needed a way to use events to

    tell the robot to perform an action •  Gestures seemed the best way to achieve this
  13. The Goals •  Needed a way to use events to

    tell the robot to perform an action •  Gestures seemed the best way to achieve this
  14. What currently exists

  15. What current exists Basic Nodebot Stack •  NodeJs (https://nodejs.org/)

  16. What current exists Basic Nodebot Stack •  NodeJs (https://nodejs.org/) • 

    Socket.io (http://socket.io/)
  17. What current exists Basic Nodebot Stack •  NodeJs (https://nodejs.org/) • 

    Socket.io (http://socket.io/) •  Johnny-Five (http://johnny-five.io/)
  18. What current exists node-openni •  Connects the Kinect to NodeJs

    using OpenNI
  19. What current exists node-openni •  Connects the Kinect to NodeJs

    using OpenNI •  Very basic event for almost everything
  20. What current exists node-openni •  Connects the Kinect to NodeJs

    using OpenNI •  Very basic event for almost everything •  No gestures !
  21. What I needed

  22. What I Needed •  Support for gestures

  23. What I Needed •  Support for gestures •  Support for

    a full “skeleton”
  24. Gestures 101

  25. Gestures 101 The Basics •  Is the initial condition met?

  26. Gestures 101 The Basics •  Is the initial condition met?

    •  Is the condition still met?
  27. Gestures 101 The Basics •  Is the initial condition met?

    •  Is the condition still met? •  Is the final condition met?
  28. Gestures 101 What do I need for gestures? •  A

    skeleton with an event when it changes
  29. Gestures 101 What do I need for gestures? •  A

    skeleton with an event when it changes •  Know the position of the COM
  30. Gestures 101 What do I need for gestures? •  A

    skeleton with an event when it changes •  Know the position of the COM •  Have a “base unit”
  31. Gestures 101 Introducing kinect-gestures •  Gesture detection using a base

    class
  32. Gestures 101 Introducing kinect-gestures •  Gesture detection using a base

    class •  Tracks the skeleton for an initial condition
  33. Gestures 101 Introducing kinect-gestures •  Gesture detection using a base

    class •  Tracks the skeleton for an initial condition •  Checks every 100ms to see if the condition is still met
  34. Gestures 101 Introducing kinect-gestures •  Gesture detection using a base

    class •  Tracks the skeleton for an initial condition •  Checks every 100ms to see if the condition is still met •  If the final condition is met, trigger an event
  35. Gestures 101 Introducing kinect-gestures •  Gesture detection using a base

    class •  Tracks the skeleton for an initial condition •  Checks every 100ms to see if the condition is still met •  If the final condition is met, trigger an event •  Other classes are used to define the actual gestures
  36. Robocalypse Demo Let’s look at some code !

  37. Next few steps •  Return a more friendly coordinate system

    for skeleton •  Add more information to the returned •  Add more gestures (jump, wave, kick)
  38. Questions?

  39. Thank you. @joel__lord github.com/joellord

  40. macadamian.com