Mark Gilbert's Blog

Science and technology, served light and fluffy.

Target-Tracking with the Kinect, Part 1 – Intro and Prerequisites

Doug started it.

On some Friday back in late June, he mentioned that Microsoft had released a Beta SDK for the Kinect just the week before.  He asserted “We need to do something with it.  I have a Kinect I can bring in.”  By “do something with it” he meant in the Friday lunch sessions we’d been holding for a year called “Sandbox”.  Sandbox was where a small group of us got together to work with something we didn’t normally get to use during our day jobs.  We tried to keep it light and fluffy, and the Kinect fit both to a T.

Over that next weekend, I mulled over what we could do with the Kinect.  What would be a good enough demonstration of this electronic marvel?  And then it hit me – Joel (another Sandbox participant) had a USB powered foam-missile launcher (like the one pictured here: http://www.amazon.com/Computer-Controlled-Foam-Missile-Launcher/dp/B00100K5RM/ref=pd_sim_t_3).  Now we’re talking!

Microsoft had released a series of tutorials for the SDK, and a .NET interface to the launcher that Chris Smith used for his “evil genius” post.  Chris attributes the original code for this wrapper to Matt Ellis.

We decided that the Kinect would feed commands to the launcher telling it where to aim, and we’d use the speech recognition abilities of the Kinect to let the person in the cross-hairs say the word “fire” to send off a missile.  And so began a series of Fridays where we hacked together an app that turned the Kinect into a target-tracking system for the launcher.  We thought we had the hard stuff already done for us – we simply needed to write something that would connect A to B.  But, as any good project should be, we found the easy parts weren’t so easy, and were pushed and prodded into learning something new.  This is the first of a three-part series describing our solution.

Before we dive into code, I want to call out the software, frameworks, and SDKs that were ultimately needed for this project.  Some of these were called out by the quickstart tutorials, and the rest were discovered along the way.  These were installed in this order:

  • First, our laptop started with Visual Studio 2010 Professional, but in the quickstart tutorials, Dan (Fernandez) mentions that he’s working with the Express version.
  • The sample application that comes with the launcher.  This includes the drivers for the launcher itself, USBHID.dll.  The .NET wrapper provided by Matt Ellis will poke into the OpenHID method to send commands to the launcher.
  • The DirectX End-User Runtime.  This is required by the DirectX SDK, and is available from here.  This installer will need to be run as an Administrator (on Windows 7, anyway), and I had to do a manual restart of the machine after it finished.  The installer will not prompt you to do this, but the DirectX SDK (the next step) wouldn’t install correctly until I did.
  • The latest DirectX SDK.  This is a very large install – 570 MB – available from http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=6812.  I had to make sure nothing else was running when I installed this, and had to install it as an Administrator.
  • The Kinect SDK itself, available from http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/.
  • The Coding4Fun Kinect Library available from http://c4fkinect.codeplex.com.  This is not strictly required, but contains a couple of extension methods that simplify translating the Kinect camera data into images.

The above was enough to get going with tracking, but not for the speech recognition we wanted to incorporate.  In particular, we wanted to be able to say the word “Fire”, and then have the launcher fire the missile.  For this, we needed a few additional pieces.  (I found these out from Patrick Godwin’s excellent post here: http://www.ximplosionx.com/2011/06/22/intro-to-the-kinect-sdkadding-speech-recognition/):

In Part 2, I’ll walk through controlling the launcher manually using Ellis’ class, and then our first pass at controlling the launcher using the data coming off of the Kinect.

In Part 3, I’ll go through the threading that we discovered we needed in order to make the application perform better, and why we ended up converting the entire application to C#.

Update: Full source code for this solution can be found in the “KinectMissileLauncher.zip” archive here: http://tinyurl.com/MarkGilbertSource.

Advertisements

September 7, 2011 - Posted by | Microsoft Kinect, Visual Studio/.NET

4 Comments

  1. […] Part 1 of this series I laid out the prerequisites.  Now we’ll get into how to turn the Kinect into a tracking […]

    Pingback by Target-Tracking with the Kinect, Part 2 – Target Tracking « Mark Gilbert’s Blog | September 10, 2011

  2. […] Part 1 of this series, I went through the prerequisites for getting the Kinect/Foam-Missile Launcher mashup […]

    Pingback by Target-Tracking with the Kinect, Part 3 – Target Tracking Improved, and Speech Recognition « Mark Gilbert’s Blog | September 10, 2011

  3. […] The logic for what we were trying to do was at least an order of magnitude greater than the Target Tracking system my colleagues and I built last year.  It functioned, but it was getting exponentially more difficult to add features to it, let […]

    Pingback by Kinect in the Abstract: Working with the Sealed SkeletonData and JointsCollection classes « Mark Gilbert’s Blog | January 10, 2012

  4. […] and I have been working on another Kinect project, a follow-up to the awesomeness that was the target-tracking system for the missile launcher.  When I have a lull in billable work at Biggs, I pull out the Kinect and plug away at […]

    Pingback by Getting Sucked In « Mark Gilbert’s Blog | April 9, 2012


Sorry, the comment form is closed at this time.

%d bloggers like this: