Kinect v2 Continous Gestures with Visual Gesture Builder

Unity Engine
1

Hello community,

I am currently developing a game with Kinect v2 SKD Unity plugin and the Visual Gesture Database Builder. It is all working great with discrete gestures, but with continuous gestures problems come up. As fast as one continuous gesture gets loaded, into the database, and first skeleton gets detected the debug/release mode would crash instantly. Same code works in a WPF application although… I was already reading about the problem on MSDN and Unity forums. Following suggestions came up there:

  • possible gestures naming problem in the database itself or
  • performance problem

For the naming I tried multiple variations, but mostly I use the naming standard for the .NET framework so it shouldn’t pose a problem. As for performance reasons, I am not sure since also in the WPF application, after loading three or more continuous gestures, the application crashes.

Has anyone had a similar problem or knows what is the reason behind it? The planned database would have around 5 discrete and 5-7 continuous gestures, therefore I would be really grateful for a possible solution or tip what to do.

2

To help future development on this matter, this is what I found out later. There are some methods in the Kinect v2 Unity plugin, which get nullified on second and later call. A cause for crashing. An example is VisualGestureBuilderDatabase method AvailableGestures. Here the code example how to overcome it, for later use in code, with a local variable:

Gesture gestureArray;
using (VisualGestureBuilderDatabase database = VisualGestureBuilderDatabase.Create(databasePath))
{
gestureArray = database.AvailableGestures.ToArray();
}
_gestureFrameSource.AddGestures(gestureArray);

It might well be that this similar error occurs with other methods. Another issue I still have problems with is the Visual Gesture Builder software, part of the Kinect SDK. When having two or more continuous gestures and adding tags to the clips, inside those projects, the app would crash instantly. I understand it’s a preview build by Microsoft, but certainly something they have overseen. Anyone with suggestions or findings on that matter?

3

You’ve gotten farther than me. I cannot even get Unity to load my visual gestures builder database file. I put it in the Assets\StreamingAssets folder, and used Application.streamingAssetsPath to build the folder path, but for some reason Unity just cannot pick it up. Where did you put your Visual Gesture Builder Database file?

4

I solved my problem. I spelled the name of my Visual Gesture Builder Database file wrong in my Unity editor.

5

///


/// Gesture Detector class which listens for VisualGestureBuilderFrame events from the service

public class GestureDetector : IDisposable
{

///


/// Gesture Detector class which listens for VisualGestureBuilderFrame events from the service
/// and calls the OnGestureDetected event handler when a gesture is detected.
///
public class GestureDetector : IDisposable
{
/// Path to the gesture database that was trained with VGB
private readonly string gestureDatabase = “GestureDB\CHANGE_T0_YOUR-DATABASE_NAME.gbd”;

///

Name of the discrete gesture in the database that we want to track
private readonly string seatedGestureName = “CHANGE-T0-YOUR-GESTURE-NAME”;

///

Gesture frame source which should be tied to a body tracking ID
private VisualGestureBuilderFrameSource vgbFrameSource = null;

///

Gesture frame reader which will handle gesture events coming from the sensor
private VisualGestureBuilderFrameReader vgbFrameReader = null;

public event EventHandler OnGestureDetected;
private readonly string seatedGestureName = “CHANGE-T0-YOUR-GESTURE-NAME”;

private VisualGestureBuilderFrameSource vgbFrameSource = null;

private VisualGestureBuilderFrameReader vgbFrameReader = null;

public event EventHandler OnGestureDetected;