Abstract: A voice conversion system employs a codebook mapping approach to transforming a source voice to sound like a target voice. Each speech frame is represented by a weighted average of codebook entries. The weights represent a perceptual distance of the speech frame and may be refined by a gradient descent analysis. The vocal tract characteristics, represented by a line spectral frequency vector, the excitation characteristics, represented by a linear predictive coding residual, the duration, and the amplitude of the speech frame are transformed in the same weighted-average framework.

Abstract: A system and method for synthesizing a facial image, compares a speech frame from an incoming speech signal with acoustic features stored within visually similar entries in an audio-visual codebook to produce a set of weights. The audio-visual codebook also stores visual features corresponding to the acoustic features. A composite visual feature is generated as a weighted sum of the corresponding visual features, from which the facial image is synthesized. The audio-visual codebook may include multiple samples of the acoustic and visual features for each entry, which corresponds to a sequence of one or more phonemes.

Abstract: A prosody modification system and methodology calculates synchronization marks in an original, quasi-periodic signal to a finer precision than the sampling rate of the original signal. Synthetic synchronization marks are generated according to the desired prosody modification also to a finer precision than the sampling rate of the original signal. Waveforms are extracted from the original signal and are fine-shifted to the exact location on the synthetic time axis by a resampling technique. The fine-shifted waveforms are windowed by an asymmetric filtering window, overlapped, and summed together to produce a synthetic signal.