SpotAlign: Probabilistic Sound Alignment using Audio Mosaicing

Date: 2010.06.14 | Category: Projects | Tags:

Today’s video game industry has seen more and more interest in bringing interactive music
applications, such as the Guitar Hero® games, to the general public. While karaoke has been
around for a long time, it has not evolved to include modern sound processing technologies.
This report presents initial work on a voicesubstitution karaoke system, SpotAlign. SpotAlign
takes an user’s voice as input to intelligently reconstruct a preexisting voice track, most likely
the original singer’s, so that it sounds like the preexisting track, but maintaining much of the
interpretation of the user (tempo and volume). Using Michael Casey’s mosaicing system
Soundspotter as a starting point, an audiotoaudio alignment algorithm was developed and
implemented. To my knowledge, this system is unique in its inclusion of features from both
audio mosaicing and audiotoaudio alignment, allowing the user to interpolate the effect
between these two techniques. Download PDF

Example

	Say we have a voice-only pre-recorded track of a famous singer (easy for a music label):
	Carlos Varela – Una Palabra (excerpt)
	Now, we sing into our system the following input(don’t mind my purposelly quirky singing):
	Pablo’s version of Una Palabra Note the different tempo and out of tuneness.
	Our system would spit back the following, in real time:
	Combined input and output(right channel is input, left channel is output)

Get SpotAlign

SpotAlign is a patch and an external binary written for the
graphical DSP language, PureData (Pd). The external uses flext, “a C++ layer for cross platform development of Pd and Max/MSP objects“. Before you open the Pd patch, first you must use flext to compile the source code for your platform (Windows, MacOS, or Linux).

Download program and source here

The binary for Linux will also be included with the source.
The code is linked against the fftw3 and libsndfile
libraries, so you will need these installed.