Multi-channel periodic signal enhancement system
A multi-channel signal enhancement system reinforces signal content and improves the signal-to-noise ratio of a multi-channel signal. The system detects, tracks, and reinforces non-stationary periodic signal components of a multi-channel signal. The periodic signal components of the signal may represent vowel sounds or other voiced sounds. The system may detect, track, or attenuate quasi-stationary signal components in the multi-channel signal.
Latest QNX Software Systems Limited Patents:
This application is a continuation-in-part of and claims the benefit of priority from U.S. application Ser. No. 10/973,575, filed Oct. 26, 2004, which is incorporated by reference.
BACKGROUND OF THE INVENTION1. Technical Field
This disclosure relates to signal processing systems, and in particular to a system that enhances periodic signal components.
2. Related Art
Audio signal processing systems may capture sound, reproduce sound, and convey sound. Audio systems may be susceptible to noise that can corrupt, mask, or otherwise affect signal content. Wind, rain, background noise, such as engine noise, electromagnetic interference, and other noise sources may contribute noise to a signal captured, reproduced, or conveyed to other systems. When the noise level of sound increases, intelligibility of the signal may decrease.
Some systems may attempt to minimize noise by detecting sound through multiple microphones. The signals from each microphone may be combined to limit the noise. Other systems may use noise filters to selectively attenuate sound. The noise filters may indiscriminately eliminate or minimize desired signal content. Therefore, there is a need for a system that enhances signals.
SUMMARYA multi-channel periodic signal enhancement system reinforces signal content and improves the signal-to-noise ratio of a multi-channel signal. The system detects, tracks, and reinforces non-stationary periodic signal components of a multi-channel signal. The periodic signal components of the signal may represent vowel sounds or other voiced sounds. The system may detect, track, or attenuate quasi-stationary signal components in the multi-channel signal.
Other systems, methods, features, and advantages of the invention will be, or will become, apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the invention, and be protected by the following claims.
The invention can be better understood with reference to the following drawings and description. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. Moreover, in the figures, like-referenced numerals designate corresponding parts throughout the different views.
The signal enhancement system detects and tracks one or more fundamental frequency components in a signal. The system may reinforce tracked frequency components. The system may improve the intelligibility of a speech signal or other audio signals. The reinforced signal may have an improved signal-to-noise ratio (SNR).
In
The enhancement system 100 may accept input from the input sources 106. The input sources 106 may include digital signal sources or analog signal sources, such as a microphone 108. The microphone 108 may be connected to the enhancement system 100 through a sampling system 110. The sampling system 110 may convert analog signals from the microphone 108 into digital form at a selected sampling rate.
The sampling rate may be selected to capture any desired frequency content. For speech, the sampling rate may be approximately 8 kHz to about 22 kHz. For music, the sampling rate may be approximately 22 Hz to about 44 kHz. Other sampling rates may be used for speech and/or music.
The digital signal sources may include a communication interface 112, other circuitry or logic in the system in which the enhancement system 100 is implemented, or other signal sources. When the input source is a digital signal source, the signal enhancement system 100 may accept the digital signal samples with or without additional preprocessing.
The signal enhancement system 100 may also connect to post-processing logic 104. The post-processing logic 104 may include an audio reproduction system 114, a digital and/or analog data transmission system 116, or video processing logic 118. Other post-processing logic also may be used.
The audio reproduction system 114 may include digital to analog converters, filters, amplifiers, and other circuitry or logic. The audio reproduction system 114 may include a speech and/or music reproduction system. The audio reproduction system 114 may be implemented in a cellular telephone, wireless telephone, digital media player/recorder, radio, stereo, portable gaming device, or other devices employing sound reproduction.
The video processing system 118 may include circuitry and/or logic that provides a visual output. The signal used to prepare the visual output may be enhanced by the processing performed by the enhancement system 100. The video processing system 118 may control a television or other entertainment device. Alternatively, the video processing system 118 may control a computer monitor or liquid crystal display (LCD).
The transmission system 116 may provide a network connection, digital or analog transmitter, or other transmission circuitry and/or logic. The transmission system 116 may communicate enhanced signals generated by the enhancement system 100 to other devices. In a car phone, for example, the transmission system 116 may communicate enhanced signals from the car phone to a base station or other receiver through a wireless connection such as a ZigBee, Mobile-Fi, Ultrawideband, Wi-fi, or a WiMax protocol.
The enhancement system 100 may process quasi-stationary or non-stationary signals. Non-stationary signals may vary in frequency and/or amplitude relatively quickly over time. Voice is one example of a non-stationary signal.
With few exceptions, even the fundamental frequency component in a speaker's voice changes during speech. The change in fundamental frequency may vary by as much as approximately 50 percent per about 100 ms or more. To the human ear, however, the speaker's voice may have a relatively constant pitch.
Quasi-stationary signals may change in frequency and/or amplitude less frequently than non-stationary signals. Quasi-stationary signals may arise from machine noise, a controlled human voice, or from other sources. Slowly changing engine noise or alternator whine are examples of quasi-stationary signals.
In
Speech signals may include a fundamental frequency component from approximately 70 Hz to about 400 Hz. Male speech may include a fundamental frequency component between approximately 70 Hz to about 200 Hz. Female speech may include a fundamental frequency component between approximately 200 Hz to about 400 Hz. A child's speech may include a fundamental frequency component between approximately 250 Hz to about 400 Hz.
The enhancement system 100 may process input signals that include speech from both male and female voices, either separately or simultaneously and overlapping. In such systems, the maximum pitch period may correspond approximately to the period of the fundamental frequency of the female voice. The maximum pitch period may be approximately about 1/300 Hz (approximately 3.3 ms), or may be another pitch period associated with female voice.
Alternatively, the enhancement system 100 may process male speech. In such systems, the maximum pitch period may correspond to the period of the fundamental frequency of a male voice. The maximum pitch period may be approximately 1/150 Hz (approximately 6.6 ms), or may be another pitch period.
The delay logic 204 may delay the input signal by the number of signal samples corresponding to the maximum pitch period. The number of signal samples may be given by:
NSS=MPP*fs
where NSS is the number of signal samples, MPP is the maximum pitch period and fs is the sampling rate. Assuming an MPP of about 3.3 ms and a sampling rate of about 8 kHz, NSS=approximately 27 samples. In
The delayed input signal may be received by the filter 206. The filter 206 may include a filter output 208 that carries a filtered output signal, labeled y(n) or y. The filter 206 may track one or more frequency components in the input signal based on the delayed input signal. The filter 206 may track the fundamental frequencies in the input signal as the pitch changes during voiced speech.
The filter 206 may reproduce, replicate, approximate, or otherwise include the tracked frequency content in the filtered output signal. The filter 206 may be a finite impulse response filter (FIR) or other type of digital filter. The filter coefficients may be adaptive. The filter 206 may be adapted by a normalized least mean squares (NLMS) technique or other type of adaptive filtering technique, such as recursive least squares (RLS) or proportional LMS. Other tracking logic, including other filters, may also be used.
The filter 206 may converge to the fundamental frequency in the input signal. The range of fundamental frequencies fo over which the filter 206 converges may be given by the following equations:
where ΔF0MAX is the period for the maximum pitch (expressed in terms of samples), fs is the sampling frequency (in units of Hz), and L is the length of the filter 206 (in units of samples). The filter length L may increase or decrease to increase or decrease the frequency extent over which the filter 206 tracks frequency components.
In the example above, the maximum pitch was approximately 300 Hz, and the delay logic 204 may use a 27 sample delay. A filter length L of 64 samples may yield a filter 206 that tracks fundamental frequency content over a frequency range of approximately 88 Hz to about 296 Hz as follows:
The filter 206 may adapt over time. The filter 206 may adapt by evaluating an error signal “e” on a sample-by-sample basis. Alternatively, the filter 206 may adapt based on blocks of samples, or some other basis. In adapting, the filter 206 may change one or more of its filter coefficients. The filter coefficients may change the response of the filter 206. The filter coefficients may adapt the filter 206 so that the filter 206 may minimize the error signal “e”.
The error estimator 210 may generate the error signal e. The error estimator 210 may be an adder, comparator, or other circuitry or logic. The error estimator 210 may compare the input signal “x” with the filtered output signal “y.”
As the filter 206 converges to the fundamental frequency in the input signal, the error signal may decrease. As the error signal decreases, the filtered output signal y may more closely resemble the input signal x delayed by an integer multiple of the signal's fundamental frequencies. The gain control logic 212 may respond to the error signal.
The optional gain control logic 212 may include a multiplier 214 and a gain parameter 216. The gain control logic 212 may attenuate, amplify, or otherwise modify the filtered output signal.
The reinforcement logic 218 may reinforce frequency content in the input signal x with the gain controlled signal Ay. The reinforcement logic 218 may be an adder or other circuitry and/or logic. The reinforcement logic 218 may produce the enhanced output signal according to the equation below:
s=x+Ay
When the error signal e increases, the gain control logic 212 may reduce the gain A. When the gain is reduced, the filtered output signal may contribute less to the enhanced output signal. The relationship between the error signal e and the gain A may be continuous, or may resemble a stepped function, a linear function, or may be discontinuous or non-linear.
In one implementation, the enhancement system 100 may establish one or more error thresholds. As the error signal exceeds an upper threshold, the gain control logic 212 may reduce the gain A to about zero. The upper threshold may be set to the input signal so that if e>x, then the gain A may be set to about zero. As the error signal falls below a lower threshold, the gain control logic 212 may increase the gain A to a value of about one.
When the error signal e exceeds the upper threshold, the filter control logic 220 may reset the filter 206. When the filter 206 is reset, the control logic 220 may reset the filter coefficients to about a value of zero, may re-initialize the filter coefficients, or may take other actions. The control logic 220 may dynamically modify the filter length, may modify the delay implemented by the delay logic 204, or may modify other characteristics of the enhancement system 100. The control logic 220 may modify the enhancement system 100 to adapt to changing environments in which the enhancement system is used, to adapt the enhancement system 100 to a new speaker, or to other applications.
The filter control logic 220 may control how quickly the filter 206 adapts, or may monitor or control other filter characteristics. The control logic 220 may expect quickly changing frequency and amplitude components in the input signal. The control logic 220 may also expect or determine over time that particular frequency components in the input signal are prevalent.
The control logic 220 may determine that the input signal x has changed in frequency content, amplitude, or other characteristics from what is expected or from what has been determined. In response, the control logic 220 may stop the filter 206 from adapting to the new signal content, may slow the rate of adaptation, or may take other actions. The control logic 220 may exercise control over the filter 206 until the input signal characteristics return to an expected signal, until a predetermined time has elapsed, until instructed to release control, or until another time or condition is met.
The delay logic 204 may prevent the filtered output signal from precisely duplicating the current input signal x. Thus, the filtered output signal may closely track the selected periodicities in the input signal x. When the current input signal x is reinforced by the filtered output signal y to produce the output signal s, periodic signal components may combine constructively, and random noise components may combine destructively. Therefore, the periodic signal components may be enhanced more than the noise.
The delay introduced by the delay logic 204 and the filter 206 may be approximately one cycle of a fundamental frequency component tracked by the filter 206. The delay may correspond to the glottal pulse delay for voice sounds, such as vowels. When the filtered output signal is added to the input signal, the delay may allow the fundamental frequency components to add in-phase or approximately in-phase.
When added in-phase, the resulting gain in the fundamental frequency content in the enhanced output signal may be approximately 6 dB or more. The noise in the input signal and the filtered output signal may be out of phase. When the input signal and the filtered output signal are added, the noise may increase less than the enhanced frequency content, for example by 3 dB or less. The enhanced output signal may have an increased signal-to-noise ratio.
The input signal x that the enhancement system 100 processes may include multiple fundamental frequencies. For example, when two speakers are speaking at the same time, the input signal may include two non-stationary fundamental frequencies. When multiple fundamental frequencies are present, the filter 206 may adapt and converge to provide a filtered out signal y, which may be a delayed version of the input signal. The reinforcement logic 218 may reinforce one or more of the fundamental frequencies present in the input signal.
At any instant in time, the coefficients 300 may be analyzed to determine a fast estimate of the fundamental frequencies in the input signal, with good temporal resolution. The coefficients 300 may begin to peak at about coefficient 304 (the fifth filter coefficient), coefficient 306 (the sixth filter coefficient), and coefficient 308 (the seventh filter coefficient). By searching for a coefficient peak or an approximate coefficient peak and determining a corresponding coefficient index c, a fast approximation of the fundamental frequency fa may be made where:
In
The control logic 220 may store historical data on many characteristics of the input signal, including the fundamental frequency of the input signal as it changes over time. The control logic 220 may examine the historical data as an aid in determining whether the characteristics of the input signal have unexpectedly changed. The control logic 220 may respond by exercising adaptation control over the filter 206 or by taking other actions.
The input signal may be delayed and filtered (Act 510). The enhancement system 100 may generate an error signal and may adapt the filter 206 (Act 512). The enhancement system 100 may control the gain of the filtered output signal (Act 514). The enhancement system 100 may add the input signal and the gain controlled signal (Act 516) to produce an enhanced output signal. The enhancement system 100 also may determine fundamental frequency estimates (Act 518). The enhancement system 100 may employ the frequency estimates to exercise adaptation control over the filter 206 (Act 520).
A signal input 606 may connect to the first stage 602. The signal input 606 may connect to the delay logic 608. The delay logic may implement a delay that corresponds to the period of a maximum quasi-stationary frequency that may be suppressed by the first stage 602.
The maximum quasi-stationary frequency may be selected according to detected or expected characteristics of the environment in which the enhancement system 600 is used. The filter control logic 610 may dynamically modify the delay to adapt the first stage 602 to the environment. The filter control logic 610 also may control the quasi-stationary filter 612.
The filter 612 in the first stage may include signal component tracking logic such as an NLMS adapted FIR filter or RLS adapted FIR filter. The filter 612 in the first stage may adapt slowly, for example with a sampling rate of about 8 kHz, a filter length of about 64, and an NLMS step size larger than about zero and less than approximately 0.01. This may allow attenuation of quasi-stationary periodic signals while minimally degrading typical speech signals. The first stage filtered output 614 may provide a filtered output signal that approximately reproduces the quasi-stationary signal component in the input signal.
The suppression logic 616 and slow filter adaptation may allow non-stationary signal components to pass through the first stage 602 to the second stage 604. On the other hand, the suppression logic 616 may suppress quasi-stationary signal components in the input signal. The suppression logic 616 may be implemented as arithmetic logic that subtracts the filtered output signal from the input signal.
The replicated quasi-stationary signal content in the filtered output signal may be removed from the input signal. The output signal produced by the first stage 602 may be shown by the equation below:
x2=e1=x−y1,
where e1 is the first stage output signal, x is the input signal, and y1 is the first stage filtered output. The first stage output 618 may be connected to the second stage 604. The second stage 604 may process the signal x2 with the adaptive filter 206. The filter 206 may adapt quickly, for example with a sampling rate of about 8 kHz, a filter length of about 64, and an NLMS step size larger than approximately 0.6 and less than about 1.0. This may allow the adaptive filter 206 to track the fundamental frequencies in typical speech signals.
The second stage 604 may enhance non-stationary signal components in the first stage output signal. The non-stationary signal components may be present in the input signal as a result of speech, music, or other signal sources. The second stage 604 may process the first stage output signal as described above.
The enhancement system 600 may employ a first suppression stage 602 followed by a second enhancement stage 604. The enhancement system 600 may to reinforce non-stationary signal content, such as voice content. In environments that introduce slowly changing signal components, the enhancement system 600 may remove or suppress the slowly changing signal components. In a wireless phone, for example, the first stage 602 may remove or suppress engine noise, road noise, or other noises, while the second stage 604 enhances non-stationary signal components, such as male or female voice components.
The signal enhancement system 100 may enhance periodic signal content, increase SNR, and/or decrease noise in an input signal. When applied to a voice signal, the enhancement system 100 may reinforce fundamental speech frequencies and may strengthen vowel or other sounds. The enhancement system 100 may enhance other signals, whether they are audible or inaudible.
The overall delay introduced by the delay logic 204 or 608 and the filter 206 or 612 also may be approximately an integer number (one or greater) of cycles of the tracked pitch period. Delaying by additional cycles may allow the input signal to change to a greater degree than waiting for one cycle. Adding a longer delayed filtered signal to the current input signal may produce special effects in the output signal, such as reverberation, while enhancing fundamental frequency components.
The periodic enhancement system of
The multi-channel periodic signal enhancement system 700 may receive the plurality of digitized audio signals x1(n) . . . xN(n) and may enhance signals of any frequency range, whether audible or not. The multi-channel periodic enhancement system 700 may receive the digitized audio signals x1(n) . . . xN(n) from different sources, such as from a telecommunications network 760, the Internet 762, or other sources.
The alignment circuits 914 may align the enhanced channel output signals y1(n) . . . yN(n) in time. When a microphone array 706 is used, the enhanced channel output signals y1(n) . . . yN(n) may require time-alignment because the distance from the speaker 714 to each microphone 722 of the microphone array 706 may be different. Thus, each microphone 722 may receive the speech signal 710 from the speaker 714 at a different time, and the digitized audio signals x1(n) . . . xN(n) and the corresponding enhanced channel output signals y1(n) . . . yN(n) may appear uncorrelated. When the enhanced channel output signals y1(n) . . . yN(n) are time-aligned and added by the output summing circuit 1020, the SNR may increase because the signals may constructively add. The enhanced channel output signals y1(n) . . . yN(n) signals may constructively add when time-aligned because the source signals (the speech signals 710) were correlated when generated. The speech signals 710 may be correlated because they originated from a common source, namely the speaker 714.
In contrast, the noise component 720 received by the microphone array 706, which may be included in the digitized audio signals x1(n) . . . xN(n), may not constructively add because the noise component 720 may not be correlated with the speech signals 710. The noise component 720 may not originate from the speaker 714, thus may not be correlated with the speech signals 710. The noise component 720 may be generated by wind, blowers, machines, mechanical devices, and other sources.
The alignment circuits 914 may align the phase of each enhanced channel output signal y1(n) . . . yN(n). A selection circuit 1028 may compare each of the enhanced channel output signals y1(n) . . . yN(n) and select a common or desired signal yi(n). The selection circuit 1028 may be included within the combiner circuit 910 or may be external to the combiner circuit. The selection circuit 1028 may select a signal with the highest SNR as a desired signal yi(n). In systems using directional microphones, the desired signal yi(n) may be selected based on a microphone oriented at a particular direction having the highest SNR. For example, in a vehicle, the microphone closest to the driver or directed toward the driver may be selected. Other criteria may be used to designate the desired signal yi(n) depending on the system.
Each of the alignment circuits 914 of
The adaptive filter 1024 may process the respective enhanced channel output signal y1(n) . . . yN(n) to time-align each channel with the common or desired signal yi(n). The adaptive filters 1024 may generate respective time-aligned output signals z1(n) . . . zN(n). Corresponding error summing circuits 1036 may sum the respective time-aligned output signals z1(n) . . . zN(n) and the delayed desired signal yi(n) to generate corresponding error signals e1(n) . . . eN(n). The adaptive filter 1024 may minimize the respective error signals e1(n) . . . eN(n) based on a least-squares type of process. The adaptive filters 1024 may use other circuitry configured to execute recursive least-squares, normalized least mean squares, proportional least mean squares, and/or least mean square routines. Other circuits may be used to minimize the respective error signals e1(n) . . . eN(n). Further variations of the minimization process may be used to ensure that the output of the adaptive filters 1024 does not diverge.
Each alignment circuit 914 may include a corresponding multiplier circuit m1 . . . mN that may multiply the time-aligned output signals z1(n) . . . zN(n) by a corresponding gain factor g1(n) . . . gN(n). The gain factor g1(n) . . . gN(n) may provide a gain factor of between about zero and 1, and may scale the magnitude of the respective time-aligned output signal z1(n) . . . zN(n) prior to summing by the output summing circuit 1020. Each time-aligned output signal z1(n) . . . zN(n) may have a different associated SNR. A low gain factor g1(n) . . . gN(n) may be applied to a time-aligned output signal z1(n) . . . zN(n) having a high SNR to minimize its contribution. A high gain factor g1(n) . . . gN(n) may be applied to a time-aligned output signal z1(n) . . . zN(n) having a low SNR to maximize its contribution. In some systems, the gain factors g1(n) . . . gN(n) may extended into sub-bands by dividing the output of each adaptive filter into sub-bands with an independent gain factor for each sub-band.
The delayed desired signal yi(n) may be scaled by a multiplier circuit mi having a gain factor gi(n). The final adder circuit 1022 may add the output of the output summing circuit 1020 to the output of the multiplier circuit mi to generate the combined output signal {circumflex over (z)}(n). The sum of all of the gain factors g1(n) . . . gN(n) and gi(n) may have a maximum value of about 1. The gain factors g1(n) . . . gN(n) and gi(n) may provide flexibility when the time-aligned output signals z1(n) . . . zN(n) are combined to meet criteria or cost functions. In some systems, the background noise in each of the channels may be adjusted to the same level before combining. In other systems, speech signals originating from certain directions may be suppressed.
The selection circuit 1028 may compare each of the digitized audio signals x1(n) . . . xN(n) and select a desired audio signal xi(n). The selection circuit 1028 may be included in the combiner circuit 910 or may be external to the combiner circuit 910. The selection circuit 1028 may select the signal with the highest SNR as a desired signal xi(n). The periodic signal enhancement circuit 904 may process the desired signal xi(n) to generate an enhanced desired signal yi(n).
A delay circuit 1206 may delay the enhanced desired signal yi(n) by a predetermined amount. A multiplier circuit mi may scale the output of the delay circuit 1206 by a gain factor gi to generate a delayed enhanced desired signal y′i(n). The delay may be equal to about one-half of the length of the adaptive filter included in each of the alignment circuits 914. Each alignment circuit 914 may time-align the respective digitized audio signals x1(n) . . . xN(n) with the delayed enhanced desired signal y′i(n).
The output summing circuit 1020 may sum the time-aligned signals generated by the alignment circuits 914. The final adding circuit 1022 may add the output of the output summing circuit 1020 to the delayed enhanced desired signal y′i(n) to generate the combined output signal {circumflex over (z)}(n).
The periodic signal enhancement circuits 904 and the alignment circuits 914 may be implemented in hardware, software, or a combination of hardware and software. The periodic signal enhancement circuits and the alignment circuits may take the form of instructions stored on a machine-readable medium such as a disk, flash card, or other memory. The periodic signal enhancement circuits and alignment circuits may be incorporated into communication devices, sound systems, gaming devices, signal processing software, or other devices and programs.
While various embodiments of the invention have been described, it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the invention. Accordingly, the invention is not to be restricted except in light of the attached claims and their equivalents.
Claims
1. A signal enhancement system having an audio source device configured to generate a plurality of input signals from a common source, the system comprising:
- a plurality of periodic signal enhancement circuits implemented in hardware or executed by a processor, each periodic signal enhancement circuit comprising an adaptive filter and each periodic signal enhancement circuit configured to: receive the respective input signal; add a first periodic signal component in the input signal with a second periodic signal component in an output of the adaptive filter to increase the first periodic signal component in the input signal, the second periodic signal component being at least partially in-phase with the first periodic signal component; and generate an enhanced output signal, the enhanced output signal comprising the increased first periodic signal component;
- a plurality of alignment circuits, each configured to receive the respective enhanced output signal and a common reference signal;
- the respective alignment circuit time-aligning each enhanced output signal with the common reference signal to generate a time-aligned output signal; and
- a summing circuit configured to: add the respective time-aligned output signals and the common reference signal to generate an enhanced time-aligned output signal; and output the enhanced time-aligned output signal.
2. The system according to claim 1, where the common reference signal is selected from the respective enhanced output signals having a highest signal-to-noise ratio.
3. The system according to claim 1, where the plurality of input signals are generated by a common audio source and contain correlated signal components.
4. The system according to claim 1, where the audio source device comprises a microphone array.
5. The system according to claim 1, where the audio source device is selected from the group consisting of a microphone array, a directional microphone array, a plurality of directional microphones, a telecommunications network, or the Internet.
6. The system according to claim 1, where each alignment circuit includes an adaptive filter circuit having a plurality of coefficients.
7. The system according to claim 6, further comprising a multiplier circuit in each alignment circuit configured to scale an output of the adaptive filter circuit by a weighting factor.
8. The system according to claim 7, where a sum of the weighing factors is between about zero and about one.
9. The system according to claim 6, further comprising a delay circuit configured to delay the common reference signal so that the coefficients of the adaptive filter circuit peak at about a mid-point in the respective adaptive filter circuit.
10. The system according to claim 1, further comprising:
- a selection circuit configured to calculate a signal-to-noise ratio for each enhanced output signal;
- a comparison circuit configured to compare the calculated signal-to-noise ratio; and a path selection circuit configured to select the enhanced output signal having a highest signal-to-noise ratio, and output the selected signal as the common reference signal.
11. The system according to claim 1, where each periodic signal enhancement circuit includes a reinforcement logic circuit.
12. The system according to claim 1, where respective alignment circuits apply a least-squares logic to time-align the enhanced output signal with the common reference signal.
13. The system according to claim 12, where the least-squares logic is selected from the group consisting of recursive least-squares, normalized least mean squares, proportional least mean squares, or least mean squares.
14. A signal enhancement system comprising:
- an audio source device configured to generate a plurality of input signals from a common source, the input signals having correlated signal components;
- a plurality of periodic signal enhancement circuits implemented in hardware or executed by a processor, each periodic signal enhancement circuit comprising an adaptive filter and each periodic signal enhancement circuit configured to: receive the respective input signal; add a first periodic signal component in the input signal with a second periodic signal component in an output of the adaptive filter to increase the first periodic signal component in the input signal, the second periodic signal component being at least partially in-phase with the first periodic signal component; and generate an enhanced output signal, the enhanced output signal comprising the increased first periodic signal component;
- a selection circuit configured to calculate a signal-to-noise ratio for each enhanced output signal and select an enhanced output signal having a highest signal-to-noise ratio as a common reference signal;
- a plurality of alignment circuits, each configured to receive the respective enhanced output signal and the common reference signal;
- the respective alignment circuit time-aligning each enhanced output signal with the common reference signal to generate a time-aligned output signal; and
- a summing circuit configured to: add the respective time-aligned output signals and the common reference signal to generate an enhanced time-aligned output signal; and output the enhanced time-aligned output signal.
15. A signal enhancement system having an audio source device configured to generate a plurality of input signals from a common source, the system comprising:
- a selection circuit configured to calculate a signal-to-noise ratio for each input signal and select an input signal having a highest signal-to-noise ratio as a common reference signal;
- a periodic signal enhancement circuit implemented in hardware or executed by a processor, the periodic enhancement circuit comprising an adaptive filter and the periodic signal enhancement circuit configured to: receive the common reference signal; add a first periodic signal component in the common reference signal with a second periodic signal component in an output of the adaptive filter to increase the first periodic signal component in the common reference signal, the second periodic signal component being at least partially in-phase with the first periodic signal component; and generate an enhanced common reference signal, the enhanced common reference signal comprising the increased first periodic signal component;
- a plurality of alignment circuits, each configured to receive the respective input signal and the enhanced common reference signal, the respective alignment circuit time-aligning each input signal with the enhanced common reference signal to generate a time-aligned output signal; and
- a summing circuit configured to: add the respective time-aligned output signals and the enhanced common reference signal to generate an enhanced time-aligned output signal; and output the enhanced time-aligned output signal.
16. A method for enhancing a signal, comprising:
- receiving a plurality of input signals from an audio source device;
- enhancing each of the input signals to generate respective enhanced output signals, where for each of the input signals, enhancing the input signal comprises: adding a first periodic signal component in the input signal with a second periodic signal component in an output of the adaptive filter to increase the first periodic signal component in the input signal, the second periodic signal component being at least partially in-phase with the first periodic signal component, the enhanced output signal comprising the increased first periodic signal component;
- aligning in time each of the enhanced output signals with a common reference signal to generate respective time-aligned output signals, the common reference signal based on an enhanced output signal having a highest signal-to-noise ratio; and
- summing the respective time-aligned output signals and the common reference signal to generate an enhanced time-aligned output signal.
17. The method according to claim 16 further comprising generating the plurality of input signals by a common audio source, the input signals containing correlated signal components.
18. The method according to claim 16, further comprising providing the plurality of input signals using an array of transducers.
19. The method according to claim 16, further comprising adaptively filtering the respective enhanced output signals to time-align the respective enhanced output signal with the common reference signal.
20. The method according to claim 19, further comprising multiplying respective results of the adaptive filtering by a respective weighting factor.
21. The method according to claim 20, where a sum of the respective weighting factors is between about zero and about one.
22. The method according to claim 16, further comprising:
- calculating an signal-to-noise ratio for each enhanced output signal;
- comparing the calculated signal-to-noise ratio;
- selecting an enhanced output signal having the highest signal-to-noise ratio; and
- outputting the selected signal as the common reference signal.
23. The method according to claim 16, where enhancing each of the input signals comprises reinforcing signal components of the respective input signals.
24. The method according to claim 16, further comprising processing the respective enhanced output signals using a least-squares process to time-align the enhanced output signal with the common reference signal.
25. A non-transitory computer-readable storage medium comprising processor-executable instructions to enhance a multi-channel signal configured to:
- receive a plurality of input signals from an audio source device;
- enhance each of the input signals to generate respective enhanced output signals by adding a first periodic signal component in the input signal with a second periodic signal component in an output of the adaptive filter to increase the first periodic signal component in the input signal, the second periodic signal being at least partially in-phase with the second periodic signal component, each of the enhanced output signals comprising the increased first periodic signal component;
- align in time each of the enhanced output signals with a common reference signal to generate respective time-aligned output signals, the common reference signal based on an enhanced output signal having a highest signal-to-noise ratio; and
- sum the respective time-aligned output signals and the common reference signal to generate an enhanced time-aligned output signal.
4238746 | December 9, 1980 | McCool et al. |
4282405 | August 4, 1981 | Taguchi |
4468804 | August 28, 1984 | Kates et al. |
4486900 | December 1984 | Cox et al. |
4531228 | July 23, 1985 | Noso et al. |
4628156 | December 9, 1986 | Irvin |
4630305 | December 16, 1986 | Borth et al. |
4731846 | March 15, 1988 | Secrest et al. |
4791390 | December 13, 1988 | Harris et al. |
4811404 | March 7, 1989 | Vilmur et al. |
4843562 | June 27, 1989 | Kenyon et al. |
4939685 | July 3, 1990 | Feintuch |
4969192 | November 6, 1990 | Chen |
5027410 | June 25, 1991 | Williamson et al. |
5056150 | October 8, 1991 | Yu et al. |
5146539 | September 8, 1992 | Doddington et al. |
5278780 | January 11, 1994 | Eguchi et al. |
5313555 | May 17, 1994 | Kamiya |
5377276 | December 27, 1994 | Terai et al. |
5400409 | March 21, 1995 | Linhard |
5406622 | April 11, 1995 | Silverberg et al. |
5408581 | April 18, 1995 | Suzuki et al. |
5412735 | May 2, 1995 | Engebretson et al. |
5432859 | July 11, 1995 | Yang et al. |
5459813 | October 17, 1995 | Klayman |
5473702 | December 5, 1995 | Yoshida et al. |
5479517 | December 26, 1995 | Linhard |
5494886 | February 27, 1996 | Kehne et al. |
5495415 | February 27, 1996 | Ribbens et al. |
5502688 | March 26, 1996 | Recchione et al. |
5526466 | June 11, 1996 | Takizawa |
5530768 | June 25, 1996 | Yoshizumi |
5568559 | October 22, 1996 | Makino |
5572262 | November 5, 1996 | Ghosh |
5584295 | December 17, 1996 | Muller et al. |
5590241 | December 31, 1996 | Park et al. |
5602962 | February 11, 1997 | Kellermann |
5615298 | March 25, 1997 | Chen |
5617508 | April 1, 1997 | Reaves |
5641931 | June 24, 1997 | Ogai et al. |
5677987 | October 14, 1997 | Seki et al. |
5680508 | October 21, 1997 | Liu |
5692104 | November 25, 1997 | Chow et al. |
5697084 | December 9, 1997 | Tingley |
5701344 | December 23, 1997 | Wakui |
5714997 | February 3, 1998 | Anderson |
5737719 | April 7, 1998 | Terry |
5742694 | April 21, 1998 | Eatwell |
5819215 | October 6, 1998 | Dobson |
5845243 | December 1, 1998 | Smart |
5864798 | January 26, 1999 | Miseki et al. |
5920840 | July 6, 1999 | Satyamurti |
5920848 | July 6, 1999 | Schutzer et al. |
5933801 | August 3, 1999 | Fink et al. |
5949886 | September 7, 1999 | Nevins et al. |
5949888 | September 7, 1999 | Gupta et al. |
5953694 | September 14, 1999 | Pillekamp |
6011853 | January 4, 2000 | Koski et al. |
6084907 | July 4, 2000 | Nagano |
6104992 | August 15, 2000 | Gao et al. |
6111957 | August 29, 2000 | Thomasson |
6144336 | November 7, 2000 | Preston et al. |
6163608 | December 19, 2000 | Romesburg et al. |
6167375 | December 26, 2000 | Miseki et al. |
6173074 | January 9, 2001 | Russo |
6175602 | January 16, 2001 | Gustafson et al. |
6188979 | February 13, 2001 | Ashley |
6192134 | February 20, 2001 | White et al. |
6199035 | March 6, 2001 | Lakaniemi et al. |
6219418 | April 17, 2001 | Eriksson et al. |
6249275 | June 19, 2001 | Kodama |
6282430 | August 28, 2001 | Young |
6285979 | September 4, 2001 | Ginzburg et al. |
6397262 | May 28, 2002 | Hayden et al. |
6405168 | June 11, 2002 | Bayya et al. |
6408273 | June 18, 2002 | Quagliaro et al. |
6434246 | August 13, 2002 | Kates et al. |
6473409 | October 29, 2002 | Malvar |
6493338 | December 10, 2002 | Preston et al. |
6498811 | December 24, 2002 | Van Der Vleuten |
6507814 | January 14, 2003 | Gao |
6587816 | July 1, 2003 | Chazan |
6594367 | July 15, 2003 | Marash et al. |
6628781 | September 30, 2003 | Grundström et al. |
6633894 | October 14, 2003 | Cole |
6643619 | November 4, 2003 | Linhard et al. |
6687669 | February 3, 2004 | Schrögmeier et al. |
6690681 | February 10, 2004 | Preston et al. |
6725190 | April 20, 2004 | Chazan et al. |
6732073 | May 4, 2004 | Kluender et al. |
6771629 | August 3, 2004 | Preston et al. |
6782363 | August 24, 2004 | Lee et al. |
6804640 | October 12, 2004 | Weintraub et al. |
6822507 | November 23, 2004 | Buchele |
6836761 | December 28, 2004 | Kawashima et al. |
6859420 | February 22, 2005 | Coney et al. |
6871176 | March 22, 2005 | Choi et al. |
6885752 | April 26, 2005 | Chabries et al. |
6891809 | May 10, 2005 | Ciccone et al. |
6898293 | May 24, 2005 | Kaulberg |
6910011 | June 21, 2005 | Zakarauskas |
6937978 | August 30, 2005 | Liu |
7003099 | February 21, 2006 | Zhang et al. |
7020291 | March 28, 2006 | Buck et al. |
7117149 | October 3, 2006 | Zakarauskas |
7146012 | December 5, 2006 | Belt et al. |
7146316 | December 5, 2006 | Alves |
7167516 | January 23, 2007 | He |
7167568 | January 23, 2007 | Malvar et al. |
7174022 | February 6, 2007 | Zhang et al. |
7206418 | April 17, 2007 | Yang et al. |
7206421 | April 17, 2007 | Taenzer |
7231347 | June 12, 2007 | Zakarauskas |
7269188 | September 11, 2007 | Smith |
7272566 | September 18, 2007 | Vinton |
7657038 | February 2, 2010 | Doclo et al. |
20010005822 | June 28, 2001 | Fujii et al. |
20010028713 | October 11, 2001 | Walker |
20020052736 | May 2, 2002 | Kim et al. |
20020071573 | June 13, 2002 | Finn |
20020176589 | November 28, 2002 | Buck et al. |
20030040908 | February 27, 2003 | Yang et al. |
20030093265 | May 15, 2003 | Xu et al. |
20030093270 | May 15, 2003 | Domer |
20030097257 | May 22, 2003 | Amada et al. |
20030101048 | May 29, 2003 | Liu |
20030206640 | November 6, 2003 | Malvar et al. |
20030216907 | November 20, 2003 | Thomas |
20030228023 | December 11, 2003 | Burnett et al. |
20040002856 | January 1, 2004 | Bhaskar et al. |
20040024600 | February 5, 2004 | Hamza et al. |
20040071284 | April 15, 2004 | Abutalebi et al. |
20040078200 | April 22, 2004 | Alves |
20040138882 | July 15, 2004 | Miyazawa |
20040165736 | August 26, 2004 | Hetherington et al. |
20040167777 | August 26, 2004 | Hetherington et al. |
20040179610 | September 16, 2004 | Lu et al. |
20050047611 | March 3, 2005 | Mao |
20050075866 | April 7, 2005 | Widrow |
20050111683 | May 26, 2005 | Chabries et al. |
20050114128 | May 26, 2005 | Hetherington et al. |
20050240401 | October 27, 2005 | Ebenezer |
20060015331 | January 19, 2006 | Hui et al. |
20060034447 | February 16, 2006 | Alves et al. |
20060056502 | March 16, 2006 | Callicotte |
20060074646 | April 6, 2006 | Alves et al. |
20060089958 | April 27, 2006 | Geisbrecht et al. |
20060089959 | April 27, 2006 | Nongpiur et al. |
20060100868 | May 11, 2006 | Hetherington et al. |
20060115095 | June 1, 2006 | Giesbrecht et al. |
20060116873 | June 1, 2006 | Hetherington et al. |
20060154623 | July 13, 2006 | Chen et al. |
20060222184 | October 5, 2006 | Buck et al. |
20060251268 | November 9, 2006 | Hetherington et al. |
20060287859 | December 21, 2006 | Hetherington et al. |
20070033031 | February 8, 2007 | Zakarauskas |
20070136055 | June 14, 2007 | Hetherington |
20070223497 | September 27, 2007 | Elson et al. |
20080001951 | January 3, 2008 | Marks et al. |
20080069366 | March 20, 2008 | Soulodre |
20080159507 | July 3, 2008 | Virolainen et al. |
20080165993 | July 10, 2008 | Cho |
20080288219 | November 20, 2008 | Tashev et al. |
20090164212 | June 25, 2009 | Chan et al. |
20090202091 | August 13, 2009 | Pedersen et al. |
2158847 | September 1994 | CA |
2157496 | October 1994 | CA |
2158064 | October 1994 | CA |
0 076 687 | April 1983 | EP |
0 275 416 | July 1988 | EP |
0 558 312 | September 1993 | EP |
0 629 996 | December 1994 | EP |
0 629 996 | December 1994 | EP |
0 750 291 | December 1996 | EP |
0 948 237 | October 1999 | EP |
1 450 353 | August 2004 | EP |
1 450 354 | August 2004 | EP |
1 669 983 | June 2006 | EP |
06269084 | September 1994 | JP |
06319193 | November 1994 | JP |
WO 0041169 | July 2000 | WO |
WO 0156255 | August 2001 | WO |
WO 0173761 | October 2001 | WO |
WO 2006/130668 | December 2006 | WO |
- Tam, et al., “Highly Oversampled Subband Adaptive Filters for Noise Cancellation on a Low-resource DSP System,” Proc. of Int. Conf. on Spoken Language Processing (ICSLP), Sep. 2002, pp. 1-4.
- Avendano, C., Hermansky, H., 'Study on the Dereverberation of Speech Based on Temporal envelope Filtering, Proc. ICSLP '96, pp. 889-892, Oct. 1996.
- Nakatani, T., Miyoshi, M., and Kinoshita, K., “Implementation and Effects of Single Channel Dereverberation Based on the Harmonic Structure of Speech,” Proc. of IWAENC-2003, pp. 91-94, Sep. 2003.
- Fiori, S., Uncini,A., and Piazza, F., “Blind Deconvolution by Modified Bussgang Algorithm”, Dept. of Electronics and Automatics—University of Ancona (Italy), ISCAS 1999.
- Vieira, J., “Automatic Estimation of Reverberation Time,” Audio EngineeringSociety, Convention Paper 6107, 116th Convention, May 8-11, 2004, Berlin, Germany, pp. 1-7.
- Pornimitkul, Pradya Et Al., 2102797 Statistic Digital Signal Processing, Comparison of NLMS and RLS for Acoustic Echo Cancellation (AEC) and White Gaussian Noise (WGN), Department of Electrical Engineering Faculty of Engineering, 2002, pp. 1-19.
- Bilcu, et al., “A New Variable Length LMS Algorithm: Theoretical Analysis and Implementations”, 2002 IEEE, pp. 1031-1034.
- Nascimento, Vitor H., “Improving the Initial Convergence of Adaptive Filters Variable-Length LMS Algorithms”, 2002 IEEE, pp. 667-670.
- Ismo Kauppinen, “Methods for Detecting Impulsive Noise in Speech and Audio Signals”, pp. 967-970, IEEE 2002.
- Saeed V. Vaseghi and Peter J.W. Rayner, “The Effects of Non-Stationary Signal Characteristics on the Performance of Adaptive Audio Restoration System”, pp. 377-380, IEEE 1989.
- Anderson C.M., et al: “Adaptive Enhancement of Finite Bandwidth Signals in White Gaussian Noise”, IEEE Trans. on Acoustics, Speech and Signal Processing, vol. ASSP-31, No. 1, Feb. 1983, pp. 17-28.
- Chang J.H., et al: “Pitch Estimation of Speech Signal Based on Adaptive Lattice Notch Filter”, Signal Processing, Elsevier Science Publishers B.V. Amsterdam, NL, vol. 85, No. 3, Mar. 2005, pp. 637-641.
- Rabiner L.R., et al: “A Comparative Performance Study of Several Pitch Detection Algorithms”, IEEE Trans. on Acoustics, Speech and Signal Processing, vol. ASSP-24, No. 5, Oct. 1976, pp. 399-418.
- Byun K.J., et al: “Noise Whitening-Based Pitch Detection for Speech Highly Corrupted by Colored Noise”, ETRI Journal, vol. 25, No. 1, Feb. 2003, pp. 49-51.
- Campbell D.A., et al: “Dynamic Weight Leakage for LMS Adaptive Linear Predictors”, Tencon '96 Proceedings, 1996 IEEE Tencon Digital Signal Processing Applications Perth, WW, Australia Nov. 26-29, 1996, NY, NY, USA, IEEE, US, vol. 2, Nov. 26, 1996, pp. 574-579.
- Sasaoka N, et al: “A New Noise Reduction System Based on ALE and Noise Reconstruction Filter”, Circuits and Systems, 2005. ISCAS 2005. IEEE International Symposium on KOBE, Japan May 23-26, 2005, Piscataway, NJ USA, IEEE May 23, 2005, pp. 272-275.
- Learned, R.E. et al., A Wavelet Packet Approach to Transient Signal Classification, Applied and Computational Harmonic Analysis, Jul. 1995, pp. 265-278, vol. 2, No. 3, USA, XP 000972660. ISSN: 1063-5203. abstract.
- Quatieri, T.F. et al., Noise Reduction Using a Soft-Decision Sine-Wave Vector Quantizer, International Conference on Acoustics, Speech & Signal Processing, Apr. 3, 1990, pp. 821-824, vol. Conf. 15, IEEE ICASSP, New York, US XP000146895, Abstract, Paragraph 3.1.
- Quelavoine, R. et al., Transients Recognition in Underwater Acoustic with Multilayer Neural Networks, Engineering Benefits from Neural Networks, Proceedings of the International Conference EANN 1998, Gibraltar, Jun. 10-12, 1998 pp. 330-333, XP 000974500. 1998, Turku, Finland, Syst. Eng. Assoc., Finland. ISBN: 951-97868-0-5. abstract, p. 30 paragraph 1.
- Simon, G., Detection of Harmonic Burst Signals, International Journal Circuit Theory and Applications, Jul. 1985, vol. 13, No. 3, pp. 195-201, UK, XP 000974305. ISSN: 0098-9886. abstract.
- Zakarauskas, P., Detection and Localization of Nondeterministic Transients in Time series and Application to Ice-Cracking Sound, Digital Signal Processing, 1993, vol. 3, No. 1, pp. 36-45, Academic Press, Orlando, FL, USA, XP 000361270, ISSN: 1051-2004. entire document.
- Widrow, B. et al., “Adaptive Noise Cancelling: Principles and Applications” 1975, IEEE, vol. 63, No. 13, New York, pp. 1692-1716.
- Kang, Hae-Dong; “Voice Enhancement Using a Single Input Adaptive Noise Elimination Technique Having a Recursive Time-Delay Estimator”, Kyungbook National University (Korea), Doctoral Thesis, Dec. 31, 1993, pp. 11-26.
- Berk et al., “Data Analysis with Microsoft Excel”, Duxbury Press, 1998, pp. 236-239 and 256-259.
- Puder, H. et al., “Improved Noise Reduction for Hands-Free Car Phones Utilizing Information on a Vehicle and Engine Speeds”, Sep. 4-8, 2000, pp. 1851-1854, vol. 3, XP009030255, 2000. Tampere, Finland, Tampere Univ. Technology, Finland Abstract.
- Seely, S., “An Introduction to Engineering Systems”, Pergamon Press Inc., 1972, pp. 7-10.
- Shust, Michael R. and Rogers, James C., Abstract of “Active Removal of Wind Noise From Outdoor Microphones Using Local Velocity Measurements”, J. Acoust. Soc. Am., vol. 104, No. 3, Pt 2, 1998, 1 page.
- Shust, Michael R. and Rogers, James C., “Electronic Removal of Outdoor Microphone Wind Noise”, obtained from the Internet on Oct. 5, 2006 at: <http://www.acoustics.org/press/136th/mshust.htm>, 6 pages.
- Wahab, A. et al., “Intelligent Dashboard With Speech Enhancement”, Information, Communications, and Signal Processing, 1997. ICICS, Proceedings of 1997 International Conference on Singapore, Sep. 9-12, 1997, New York, NY, USA, IEEE, pp. 993-997.
- The prosecution history of U.S. Appl. No. 10/973,575, printed Dec. 30, 2008, including any substantive Office Actions and Applicant Responses.
- The prosecution history of U.S. Appl. No. 11/101,796, printed Dec. 12, 2008, including any substantive Office Actions and Applicant Responses.
- The prosecution history of U.S. Appl. No. 11/102,251, printed Dec. 12, 2008, including any substantive Office Actions and Applicant Responses.
- The prosecution history of U.S. Appl. No. 11/298,052, printed Dec. 30, 2008, including any substantive Office Actions and Applicant Responses.
- The prosecution history of U.S. Appl. No. 11/317,762, printed Jan. 6, 2009, including any substantive Office Actions and Applicant Responses.
- The prosecution history of U.S. Appl. No. 11/757,768, printed Jan. 6, 2009, including any substantive Office Actions and Applicant Responses.
- Koike, Shiin'ichi, “Adaptive Threshold Nonlinear Algorithm for Adaptive Filters with Robustness Against Impulse Noise,” 1996, IEEE, NEC Corporation, Tokyo 108-01, pp. 1644-1647.
- Gui, Y. et al., “Adaptive Subband Wiener Filtering for Speech Enhancement Using Critical-Band Gammatone Filterbank,” IEEE, 2005, pp. 732-735.
- Office Action dated Nov. 26, 2012 from corresponding Canadian Application No. 2 516 881, 7 pages.
- Office Action dated Jul. 13, 2009 from corresponding Canadian Application No. 2 571 417, 4 pages.
- Office Action dated Jun. 19, 2012 from corresponding Canadian Application No. 2 571 417, 2 pages.
- Office Action dated Jul. 28, 2011 from corresponding Canadian Application No. 2 569 223, 3 pages.
Type: Grant
Filed: Aug 31, 2007
Date of Patent: Sep 24, 2013
Patent Publication Number: 20080019537
Assignee: QNX Software Systems Limited (Kanata, Ontario)
Inventors: Rajeev Nongpiur (Burnaby), Phillip Hetherington (Moody)
Primary Examiner: Douglas Godbold
Application Number: 11/849,009
International Classification: G10L 21/02 (20060101);