Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Robocalypse: Controlling Nodebots with Kinect gestures

Joel Lord
October 28, 2015

Robocalypse: Controlling Nodebots with Kinect gestures

A Full Stack 2015 London presentation by Joel Lord. Links to full video to come...

Joel Lord

October 28, 2015
Tweet

More Decks by Joel Lord

Other Decks in Programming

Transcript

  1. confidential not for distribution
    A Full Stack 2015 London presentation by Joel Lord
    Robocalypse: Controlling
    Nodebots with Kinect gestures

    View Slide

  2. Overview
    About Me
    Why
    The Goals
    What Exists
    What I wanted
    Gestures 101
    Demo

    View Slide

  3. About Me

    View Slide

  4. About Me
    Joel Lord
    Development Manager
    Javascript junkie, technology
    enthusiast, IoT hobbyist.
    Macadamian
    http://www.macadamian.com
    Twitter: @joel__lord
    Github: github.com/joellord

    View Slide

  5. Stereotypes
    I have never fought a polar bear with my bare hands

    View Slide

  6. Stereotypes
    My yard last April

    View Slide

  7. Why ?

    View Slide

  8. Why
    •  Worked with Nodebots for a while
    now
    •  Wanted to explore other possibilities
    of NodeJs
    •  Because controlling a robot with a
    Kinect is pretty cool

    View Slide

  9. The Goals

    View Slide

  10. The Goals
    •  Needed a way to use events to tell
    the robot to perform an action
    •  Gestures seemed the best way to
    achieve this

    View Slide

  11. What currently exists

    View Slide

  12. What current exists
    Basic Nodebot Stack
    •  NodeJs (https://nodejs.org/)
    •  Socket.io (http://socket.io/)
    •  Johnny-Five (http://johnny-five.io/)

    View Slide

  13. What current exists
    node-openni
    •  Connects the Kinect to NodeJs using OpenNI
    •  Very basic event for almost everything
    •  No gestures !

    View Slide

  14. What I needed

    View Slide

  15. What I Needed
    •  Support for a full “skeleton”
    •  Support for gestures

    View Slide

  16. What I Needed

    View Slide

  17. Gestures 101

    View Slide

  18. Gestures 101
    The Basics
    •  Is the initial condition met?
    •  Is the condition still met?
    •  Is the final condition met?

    View Slide

  19. Gestures 101
    What do I need for gestures?
    •  A skeleton with an event when it changes
    •  Know the position of the COM
    •  Have a “base unit”

    View Slide

  20. Gestures 101
    Introducing kinect-gestures
    •  Gesture detection using a base class
    •  Tracks the skeleton for an initial condition
    •  Checks every 100ms to see if the condition is still met
    •  If the final condition is met, trigger an event
    •  Other classes are used to define the actual gestures

    View Slide

  21. Robocalypse
    Demo
    Let’s look at some code !

    View Slide

  22. Next few steps
    •  Return a more friendly coordinate system for skeleton
    •  Add more information to the returned
    •  Add more gestures (jump, wave, kick)

    View Slide

  23. Questions?

    View Slide

  24. Thank you.
    @joel__lord
    joellord

    View Slide

  25. macadamian.com

    View Slide