Robocalypse: Controlling Nodebots with Kinect gestures

A Full Stack 2015 London presentation by Joel Lord. Links to full video to come...

Transcript

1. confidential not for distribution
   A Full Stack 2015 London presentation by Joel Lord
   Robocalypse: Controlling
   Nodebots with Kinect gestures
   

   View Slide

2. Overview
   About Me
   Why
   The Goals
   What Exists
   What I wanted
   Gestures 101
   Demo
   

   View Slide

3. About Me
   

   View Slide

4. About Me
   Joel Lord
   Development Manager
   Javascript junkie, technology
   enthusiast, IoT hobbyist.
   Macadamian
   http://www.macadamian.com
   Twitter: @joel__lord
   Github: github.com/joellord
   

   View Slide

5. Stereotypes
   I have never fought a polar bear with my bare hands
   

   View Slide

6. Stereotypes
   My yard last April
   

   View Slide

7. Why ?
   

   View Slide

8. Why
   •  Worked with Nodebots for a while
   now
   •  Wanted to explore other possibilities
   of NodeJs
   •  Because controlling a robot with a
   Kinect is pretty cool
   

   View Slide

9. The Goals
   

   View Slide

10. The Goals
    •  Needed a way to use events to tell
    the robot to perform an action
    •  Gestures seemed the best way to
    achieve this
    

    View Slide

11. What currently exists
    

    View Slide

12. What current exists
    Basic Nodebot Stack
    •  NodeJs (https://nodejs.org/)
    •  Socket.io (http://socket.io/)
    •  Johnny-Five (http://johnny-five.io/)
    

    View Slide

13. What current exists
    node-openni
    •  Connects the Kinect to NodeJs using OpenNI
    •  Very basic event for almost everything
    •  No gestures !
    

    View Slide

14. What I needed
    

    View Slide

15. What I Needed
    •  Support for a full “skeleton”
    •  Support for gestures
    

    View Slide

16. What I Needed
    

    View Slide

17. Gestures 101
    

    View Slide

18. Gestures 101
    The Basics
    •  Is the initial condition met?
    •  Is the condition still met?
    •  Is the final condition met?
    

    View Slide

19. Gestures 101
    What do I need for gestures?
    •  A skeleton with an event when it changes
    •  Know the position of the COM
    •  Have a “base unit”
    

    View Slide

20. Gestures 101
    Introducing kinect-gestures
    •  Gesture detection using a base class
    •  Tracks the skeleton for an initial condition
    •  Checks every 100ms to see if the condition is still met
    •  If the final condition is met, trigger an event
    •  Other classes are used to define the actual gestures
    

    View Slide

21. Robocalypse
    Demo
    Let’s look at some code !
    

    View Slide

22. Next few steps
    •  Return a more friendly coordinate system for skeleton
    •  Add more information to the returned
    •  Add more gestures (jump, wave, kick)
    

    View Slide

23. Questions?
    

    View Slide

24. Thank you.
    @joel__lord
    joellord
    

    View Slide

25. macadamian.com
    

    View Slide