Voice transformation

Since May 2006, I've been working on digital signal processing on the voice, and in particular the transformation of the voice. The idea is to make the voice of someone sound like another person, either of another sex and age, for instance, or a particular person. I've mostly worked on the first aspect during my time at Ircam. At Vocally I'm also working on the latter.

Voice transformation at Ircam 2006-09:

It started my research on voice transformations in the project VIVOS (ANR 2006-08), which was concerned with the creation of expressive voices for multimedia applications. Objectives of the project were improving French speech synthesis, establishment of methods for automatic generation of speech databases from arbitrary manuscripted recordings, and natural transformation of the characteristics of the voice. The project aimed at applications requiring a high degree of naturalness, such as automatic film dubbing, voices for animation films and artistic purposes, and video games whether for entertainment or education.

While doing research on the variation of the voice depending on sex and age and the variation of the voice quality between different persons, I developed a number of voice-transformation techniques in Matlab. The library, called VoiceTrans, can change the speaker's sex (male/female) and age (child/teenager/adult/old person) and modify the voice quality (such as whisper, breathiness, creakiness, roughness, tenseness, and trembling), and some expressivities (such as bored or excited). You may listen to some sound examples.

I also worked in the project Affective Avatars (ANR 2008-09) whose goal was to create animated affective avatars in real time. The idea was to extract expressive and emotional parameters from the voice to control the lips, facial expressions and gestures of the avatar. In this framework I adapted the transformations of VoiceTrans to real-time and developed a C++ library VoiceForger to perform these transformations in real time. To test the real-time performance and thread handling of the library, I also developed a simple graphical test interface with buttons and slider for Mac OS X. The library is now a part of the plug-in IrcamTools Trax for Pro Tools.