Sound, at long last.

Friday, 7th March 2008

I have finally got around to adding sound to Cogwheel using Ianier Munoz's waveOut API wrapper.

The technique used is fairly simple. I start with a sound buffer that is a multiple a number of video frames in length (1/60th of a second is one frame) - four seems a good number. This buffer needs to be periodically topped up with sound samples (every four frames in the above example).

I run the emulator for one frame, then generate a frame's worth of audio. I add these samples to a queue. The sound callback then periodically dequeues these samples and appends them to its buffer.

// This is called once every video frame.
// 735 samples at 44100Hz = 1/60th second.
// (Multiplied by two for stereo).
this.Emulator.RunFrame();
short[] Buffer = new short[735 * 2];
this.Emulator.Sound.CreateSamples(Buffer);
this.GeneratedSoundSamples.Enqueue(Buffer);

The important thing is that the sound is always generated after the video frame (and thus after any hardware writes). I log writes to the sound hardware over the period of a frame (along with the number of CPU cycles that have elapsed), then space them out when generating the sound samples so that they play in synch. My previous problems were caused by the sound emulation trying to "look ahead" past what had already been generated.

However, there is a potential problem with this - as the video and sound emulation are not locked in synch with eachother, there are two cases that could crop up:

1. The emulator runs faster than 60Hz, generating too many sound samples.
2. The emulator runs slower than 60Hz, not generating enough sound samples.

The first is the easiest to deal with. In most instances you'd want a couple of extra frames of sound data left in the queue after topping up the sound buffer, in case in the next period not enough are generated. However, if I notice that the queue is longer than entire sound buffer after topping it up, I clear it completely. This would make the sound a little choppy, but so far this hasn't happened in my tests.

The latter is a little more complex. If I just left it the sound buffer would have gaps in it, causing noticable pops (this I have noticed in some of the more processor-intensive games). To cover up the gaps, I generate enough extra frames of sound data to fill the gap. As no sound hardware writes are made, this has the effect of extending any tones that were currently playing, so the sound will play back slightly out of time. However, slightly out of time by a few 60ths of a second is a better solution than a pop.

// This is called when the sound buffer needs topping up.
// That's about once every four frames.
private void SoundBufferFiller(IntPtr data, int size) {

	// Temporary buffer to store the generated samples.
	short[] Generated = new short[size / 2];

	for (int i = 0; i < Generated.Length; i += 735 * 2) {
		if (this.GeneratedSoundSamples.Count > 0) {
			// We've already queued up some sound samples.
			Array.Copy(this.GeneratedSoundSamples.Dequeue(), 0, Generated, i, 735 * 2);
		} else {
			// Erk, we're out of samples... force generate some more and use those instead.
			// (This avoids popping).
			short[] Temp = new short[735 * 2];
			this.Emulator.Sound.CreateSamples(Temp);
			Array.Copy(Temp, 0, Generated, i, 735 * 2);
		}
	}

	// Copy to the sound buffer.
	Marshal.Copy(Generated, 0, data, size / 2);
	
	// If too many samples are being generated (FPS > 60Hz) then make sure it doesn't go out of control.
	while (this.GeneratedSoundSamples.Count > this.SoundBufferSizeInFrames) this.GeneratedSoundSamples.Dequeue();

}