Programming with the Kinect for Windows Software Development Kit: Displaying Kinect Data

The audio display manager

The audio stream provides two important pieces of information that the user of your Kinect applications may want to know. The first is the sound source angle, which is the angle (in radians) to the current position of the audio source in camera coordinates.

The second is the beam angle produced by the microphone array. By using the fact that the sound from a particular audio source arrives at each microphone in the array at a slightly different time, beamforming allows applications to determine the direction of the audio source and use the microphone array as a steerable directional microphone.

The beam angle can be important as a visual feedback to indicate which audio source is being used (for speech recognition, for instance), as shown in Figure 3-6.

Figure 3-6 Visual feedback of beam angle.

This visual feedback is a virtual representation of the sensor, and in Figure 3-6, the orange area to the right of center (which appears as gray in the print book) indicates the direction of the beam. (For readers of the print book, Figure 3-6 is orange near the center and fades to black on either side of the beam.)

To recreate the same control, you can add an XAML page with the following XAML declaration:

<Rectangle x:Name="audioBeamAngle" Height="20" Width="300" Margin="5">
    <Rectangle.Fill>
        <LinearGradientBrush StartPoint="0,0" EndPoint="1, 0">
            <GradientStopCollection>
                <GradientStop Offset="0" Color="Black"/>
                <GradientStop Offset="{Binding BeamAngle}" Color="Orange"/>
                <GradientStop Offset="1" Color="Black"/>
            </GradientStopCollection>
        </LinearGradientBrush>
    </Rectangle.Fill>
</Rectangle>

You can see that the rectangle is filled with a LinearGradientBrush starting from black to orange to black. The position of the orange GradientStop can be bound to a BeamAngle property exposed by a class.

The binding code itself is quite obvious:

var kinectSensor = KinectSensor.KinectSensors[0];
var audioManager = new AudioStreamManager(kinectSensor.AudioSource);
audioBeamAngle.DataContext = audioManager;

So you have to create an AudioStreamManager class that exposes a BeamAngle property. The class inherits from the Notifier class you created earlier in this chapter and implements IDisposable:

using Microsoft.Kinect;
public class AudioStreamManager : Notifier, IDisposable
{
    readonly KinectAudioSource audioSource;

    public AudioStreamManager(KinectAudioSource source)
    {
        audioSource = source;
        audioSource.BeamAngleChanged += audioSource_BeamAngleChanged;
    }

    void audioSource_BeamAngleChanged(object sender, BeamAngleChangedEventArgs e)
    {
        RaisePropertyChanged(()=>BeamAngle);
    }

    public double BeamAngle
    {
        get
        {
            return (audioSource.BeamAngle - KinectAudioSource.MinBeamAngle) /
(KinectAudioSource.MaxBeamAngle - KinectAudioSource.MinBeamAngle);
        }
    }

    public void Dispose()
    {
        audioSource.BeamAngleChanged -= audioSource_BeamAngleChanged;
    }
}

There is nothing special to note about this code, except to mention that the computation of the BeamAngle returns a value in the range [0, 1], which in turn will be used to set the offset of the orange GradientStop.

Now you can display all kinds of streams produced by the Kinect sensor to provide reliable visual feedback to the users of your applications.