Patents by Inventor Jimeng Zheng

Jimeng Zheng has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 12207075
    Abstract: A real-time communication software application for generating spatial audio with uniform reverberation in a real-time communication session is performed by an electronic communication device. The application is adapted to remove the reverberation of recorded speech signals from far-end participants by the dereverberation approach, render the direct sound parts by filtering the output signals by head-related transfer functions of desired directions, generate reverberant sound parts by convolving the output signals from with uniform room impulse responses or an artificial reverberator, and combine direct and reverberant sound components to generate spatialized speech signals. When speakers and listeners are located in two virtual conference rooms, the reverberation of the two rooms are coupled. The reverberant sound parts are then generated by convolving the output signals and coupled RIRs from the two rooms.
    Type: Grant
    Filed: March 1, 2024
    Date of Patent: January 21, 2025
    Assignee: Agora Lab, Inc.
    Inventors: Song Li, Jianyuan Feng, Bo Wu, Jimeng Zheng
  • Patent number: 12154586
    Abstract: A computer-implemented method for suppressing noise from audio signal uses both statistical noise estimation and neural network noise estimation to achieve more desirable noise reduction. The method is performed by a noise suppression computer software application running on an electronic device. The noise suppression computer software application first transforms the speech signal in time domain into frequency domain before determining a statistical noise estimate and a neural network noise estimate. The noise suppression computer software application merges the two noise estimates to derive a final noise estimate, and determines and refines a noise suppression filter. The filter is applied to the speech signal in frequency domain to obtain an enhanced signal. The enhanced signal is transformed back into time domain.
    Type: Grant
    Filed: May 24, 2022
    Date of Patent: November 26, 2024
    Assignee: Agora Lab, Inc.
    Inventors: Jimeng Zheng, Bo Wu, Xiaohan Zhao, Liangliang Wang, Ruofei Chen
  • Patent number: 12051441
    Abstract: This application discloses a multi-sound area-based speech detection method and related apparatus, and a storage medium, which is applied to the field of artificial intelligence. The method includes: obtaining sound area information corresponding to N sound areas including multiple users speaking simultaneously; generating a control signal corresponding to each target detection sound area according to user information corresponding to the target detection sound area; processing multi-user speech input signals by using the control signals, to obtain a speech output signal corresponding to each target detection sound area; generating a speech detection result of the target detection sound area according to the speech output signal corresponding to the target detection sound area; and selecting, among the multiple users, a main speaker based on the user information, the speech output signals and speech detection results of multiple users in the N sound areas.
    Type: Grant
    Filed: September 13, 2022
    Date of Patent: July 30, 2024
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Jimeng Zheng, Lianwu Chen, Weiwei Li, Zhiyi Duan, Meng Yu, Dan Su, Kaiyu Jiang
  • Publication number: 20240233719
    Abstract: This application discloses a method for positioning a target audio signal by a computer device. The method includes: performing echo cancellation on the audio signals collected in a plurality of directions in a space, the audio signals comprising a target-audio direct signal; obtaining weights of a plurality of time-frequency points in the echo-canceled audio signals, a weight of each time-frequency point indicating a relative proportion of the target-audio direct signal in the echo-canceled audio signals at the time-frequency point; obtaining a weighted audio signal energy distribution of the audio signals in the plurality of directions by using the weights of the plurality of time-frequency points in the echo-canceled audio signals; and obtaining a sound source azimuth corresponding to the target-audio direct signal in the audio signals by using the weighted audio signal energy distribution of the audio signals in the plurality of directions.
    Type: Application
    Filed: March 20, 2024
    Publication date: July 11, 2024
    Inventors: Jimeng ZHENG, Ian Ernan Liu, Yi Gao, Weiwei Li
  • Publication number: 20240205635
    Abstract: A real-time communication software application for generating spatial audio with uniform reverberation in a real-time communication session is performed by an electronic communication device. The application is adapted to remove the reverberation of recorded speech signals from far-end participants by the dereverberation approach, render the direct sound parts by filtering the output signals by head-related transfer functions of desired directions, generate reverberant sound parts by convolving the output signals from with uniform room impulse responses or an artificial reverberator, and combine direct and reverberant sound components to generate spatialized speech signals. When speakers and listeners are located in two virtual conference rooms, the reverberation of the two rooms are coupled. The reverberant sound parts are then generated by convolving the output signals and coupled RIRs from the two rooms.
    Type: Application
    Filed: March 1, 2024
    Publication date: June 20, 2024
    Inventors: Song Li, Jianyuan Feng, Bo Wu, Jimeng Zheng
  • Patent number: 11967316
    Abstract: Embodiments of this application disclose method and apparatus for positioning a target audio signal by an audio interaction device, and an audio interaction device The method includes: obtaining audio signals in a plurality of directions in a space, and performing echo cancellation on the audio signal, the audio signal including a target-audio direct signal; obtaining weights of a plurality of time-frequency points in the audio signals, a weight of each time-frequency point indicating, at the time-frequency point, a relative proportion of the target-audio direct signal in the audio signals; weighting time-frequency components of the audio signal at the plurality of time-frequency points separately for each of the plurality of directions by using the weights of the plurality of time-frequency points, to obtain a weighted audio signal energy distribution; and obtaining a sound source azimuth corresponding to the target-audio direct signal in the audio signals accordingly.
    Type: Grant
    Filed: February 23, 2021
    Date of Patent: April 23, 2024
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Jimeng Zheng, Ian Ernan Liu, Yi Gao, Weiwei Li
  • Patent number: 11950088
    Abstract: A computer-implemented method for generating spatial audio with uniform reverberation in a real-time communication session is performed by a real-time communication software application running on an electronic communication device. The method includes removing the reverberation of recorded speech signals from far-end participants by the dereverberation approach, rendering the direct sound parts by filtering the output signals by head-related transfer functions of desired directions, generating reverberant sound parts by convolving the output signals from with uniform room impulse responses or an artificial reverberator, combining direct and reverberant sound components to generate spatialized speech signals. When speakers and listeners are located in two virtual conference rooms, the reverberation of the two rooms are coupled. The reverberant sound parts are then generated by convolving the output signals and coupled RIRs from the two rooms.
    Type: Grant
    Filed: July 7, 2022
    Date of Patent: April 2, 2024
    Assignee: Agora Lab, Inc.
    Inventors: Song Li, Jianyuan Feng, Bo Wu, Jimeng Zheng
  • Patent number: 11908456
    Abstract: Embodiments of this application discloses an azimuth estimation method performed at a computing device, the method including: obtaining, in real time, multi-channel sampling signals and buffering the multi-channel sampling signals; performing wakeup word detection on one or more sampling signals of the multi-channel sampling signals, and determining a wakeup word detection score for each channel of the one or more sampling signals; performing a spatial spectrum estimation on the buffered multi-channel sampling signals to obtain a spatial spectrum estimation result, when the wakeup word detection scores of the one or more sampling signals indicates that a wakeup word exists in the one or more sampling signals; and determining an azimuth of a target voice associated with the multi-channel sampling signals according to the spatial spectrum estimation result and a highest wakeup word detection score, thereby improving the accuracy of the azimuth estimation in a voice interaction process.
    Type: Grant
    Filed: August 28, 2020
    Date of Patent: February 20, 2024
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Jimeng Zheng, Yi Gao, Meng Yu, Ian Ernan Liu
  • Publication number: 20240015466
    Abstract: A computer-implemented method for generating spatial audio with uniform reverberation in a real-time communication session is performed by a real-time communication software application running on an electronic communication device. The method includes removing the reverberation of recorded speech signals from far-end participants by the dereverberation approach, rendering the direct sound parts by filtering the output signals by head-related transfer functions of desired directions, generating reverberant sound parts by convolving the output signals from with uniform room impulse responses or an artificial reverberator, combining direct and reverberant sound components to generate spatialized speech signals. When speakers and listeners are located in two virtual conference rooms, the reverberation of the two rooms are coupled. The reverberant sound parts are then generated by convolving the output signals and coupled RIRs from the two rooms.
    Type: Application
    Filed: July 7, 2022
    Publication date: January 11, 2024
    Inventors: Song Li, Jianyuan Feng, Bo Wu, Jimeng Zheng
  • Patent number: 11856376
    Abstract: This application discloses a sound acquisition component array, including: two first sound acquisition components, two second sound acquisition components, and two third sound acquisition components. The two second sound acquisition components are located at a first side of a line connecting the two first sound acquisition components, and the two third sound acquisition components are located at a second side of the connecting line that is opposite to the first side of the connecting line; the two second sound acquisition components are symmetrical about a perpendicular bisector of the connecting line, and the two third sound acquisition components are symmetrical about the perpendicular bisector; and a distance between the two first sound acquisition components, a distance between the two second sound acquisition components, and a distance between the two third sound acquisition components are respectively different from one another along a direction defined by the connecting line.
    Type: Grant
    Filed: May 12, 2021
    Date of Patent: December 26, 2023
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Jimeng Zheng, Yi Gao, Xuan Ji, Weiwei Li, Meng Yu, Kai Xia, Jun Feng, Zhu Chen, Hongyang Chen, Wenbin Yang, Yu Wang, Yong Liu
  • Publication number: 20230386492
    Abstract: A computer-implemented method for suppressing noise from audio signal uses both statistical noise estimation and neural network noise estimation to achieve more desirable noise reduction. The method is performed by a noise suppression computer software application running on an electronic device. The noise suppression computer software application first transforms the speech signal in time domain into frequency domain before determining a statistical noise estimate and a neural network noise estimate. The noise suppression computer software application merges the two noise estimates to derive a final noise estimate, and determines and refines a noise suppression filter. The filter is applied to the speech signal in frequency domain to obtain an enhanced signal. The enhanced signal is transformed back into time domain.
    Type: Application
    Filed: May 24, 2022
    Publication date: November 30, 2023
    Inventors: Jimeng Zheng, Bo Wu, Xiaohan Zhao, Liangliang Wang, Ruofei Chen
  • Publication number: 20230013740
    Abstract: This application discloses a multi-sound area-based speech detection method and related apparatus, and a storage medium, which is applied to the field of artificial intelligence. The method includes: obtaining sound area information corresponding to each sound area in N sound areas; using the sound area as a target detection sound area, and generating a control signal corresponding to the target detection sound area according to sound area information corresponding to the target detection sound area; processing a speech input signal corresponding to the target detection sound area by using the control signal corresponding to the target detection sound area, to obtain a speech output signal corresponding to the target detection sound area; and generating a speech detection result of the target detection sound area according to the speech output signal corresponding to the target detection sound area.
    Type: Application
    Filed: September 13, 2022
    Publication date: January 19, 2023
    Inventors: Jimeng ZHENG, Lianwu CHEN, Weiwei Li, Zhiyi Duan, Meng YU, Dan Su, Kaiyu Jiang
  • Publication number: 20210266664
    Abstract: This application discloses a sound acquisition component array, including: two first sound acquisition components, two second sound acquisition components, and two third sound acquisition components. The two second sound acquisition components are located at a first side of a line connecting the two first sound acquisition components, and the two third sound acquisition components are located at a second side of the connecting line that is opposite to the first side of the connecting line; the two second sound acquisition components are symmetrical about a perpendicular bisector of the connecting line, and the two third sound acquisition components are symmetrical about the perpendicular bisector; and a distance between the two first sound acquisition components, a distance between the two second sound acquisition components, and a distance between the two third sound acquisition components are respectively different from one another along a direction defined by the connecting line.
    Type: Application
    Filed: May 12, 2021
    Publication date: August 26, 2021
    Inventors: Jimeng Zheng, Yi Gao, Xuan Ji, Weiwei Li, Meng Yu, Kai Xia, Jun Feng, Zhu Chen, Hongyang Chen, Wenbin Yang, Yu Wang, Yong Liu
  • Publication number: 20210174792
    Abstract: Embodiments of this application disclose method and apparatus for positioning a target audio signal by an audio interaction device, and an audio interaction device The method includes: obtaining audio signals in a plurality of directions in a space, and performing echo cancellation on the audio signal, the audio signal including a target-audio direct signal; obtaining weights of a plurality of time-frequency points in the audio signals, a weight of each time-frequency point indicating, at the time-frequency point, a relative proportion of the target-audio direct signal in the audio signals; weighting time-frequency components of the audio signal at the plurality of time-frequency points separately for each of the plurality of directions by using the weights of the plurality of time-frequency points, to obtain a weighted audio signal energy distribution; and obtaining a sound source azimuth corresponding to the target-audio direct signal in the audio signals accordingly.
    Type: Application
    Filed: February 23, 2021
    Publication date: June 10, 2021
    Inventors: Jimeng ZHENG, Ian Ernan LIU, Yi GAO, Weiwei LI
  • Publication number: 20200395005
    Abstract: Embodiments of this application discloses an azimuth estimation method performed at a computing device, the method including: obtaining, in real time, multi-channel sampling signals and buffering the multi-channel sampling signals; performing wakeup word detection on one or more sampling signals of the multi-channel sampling signals, and determining a wakeup word detection score for each channel of the one or more sampling signals; performing a spatial spectrum estimation on the buffered multi-channel sampling signals to obtain a spatial spectrum estimation result, when the wakeup word detection scores of the one or more sampling signals indicates that a wakeup word exists in the one or more sampling signals; and determining an azimuth of a target voice associated with the multi-channel sampling signals according to the spatial spectrum estimation result and a highest wakeup word detection score, thereby improving the accuracy of the azimuth estimation in a voice interaction process.
    Type: Application
    Filed: August 28, 2020
    Publication date: December 17, 2020
    Inventors: Jimeng Zheng, Yi Gao, Meng Yu, Ian Eman Liu
  • Patent number: 10856080
    Abstract: Provided are, among other things, systems, methods and techniques for reducing echo in an audio signal. One representative embodiment involves obtaining an input signal, an estimate of a system-characterizing function, and a reference signal, each at a corresponding sample rate and each divided into a plurality of sub-bands; separately processing such sub-bands, where for a given sub-band the estimate of the system-characterizing function and the reference signal are processed to generate an echo-estimation signal and then the echo-estimation signal is subtracted from the input signal to provide an echo-corrected signal for such given sub-band; and combining the echo-corrected signal from each of different ones of the plurality of the sub-bands to provide a final output signal, with the echo-estimation signal generated using a processing sample rate that is lower than the sample rate for the input signal.
    Type: Grant
    Filed: October 16, 2018
    Date of Patent: December 1, 2020
    Assignee: Guoguang Electric Company Limited
    Inventors: Jimeng Zheng, Yuli You
  • Patent number: 10325583
    Abstract: Provided are, among other things, systems, methods and techniques for audio-signal processing. One representative embodiment includes HT sub-band analysis/decomposition modules, e.g., one for each audio channel and one for an echo reference signal. Each HT sub-band analysis/decomposition module includes a Hilbert Transformation module and an analysis/decomposition filter bank and provides sub-band outputs. Echo-cancellation modules, e.g., one for each audio channel, perform echo-cancellation processing on such sub-bands. Beamforming modules, e.g., one for each sub-band, then perform beamforming, e.g., across all audio channels. Finally, a resynthesis stage combines the different sub-band outputs in order to provide a system output signal.
    Type: Grant
    Filed: October 4, 2017
    Date of Patent: June 18, 2019
    Assignee: Guoguang Electric Company Limited
    Inventors: Jimeng Zheng, Yuli You
  • Publication number: 20190103088
    Abstract: Provided are, among other things, systems, methods and techniques for audio-signal processing. One representative embodiment includes HT sub-band analysis/decomposition modules, e.g., one for each audio channel and one for an echo reference signal. Each HT sub-band analysis/decomposition module includes a Hilbert Transformation module and an analysis/decomposition filter bank and provides sub-band outputs. Echo-cancellation modules, e.g., one for each audio channel, perform echo-cancellation processing on such sub-bands. Beamforming modules, e.g., one for each sub-band, then perform beamforming, e.g., across all audio channels. Finally, a resynthesis stage combines the different sub-band outputs in order to provide a system output signal.
    Type: Application
    Filed: October 4, 2017
    Publication date: April 4, 2019
    Inventors: Jimeng Zheng, Yuli You
  • Publication number: 20190082259
    Abstract: Provided are, among other things, systems, methods and techniques for reducing echo in an audio signal. One representative embodiment involves obtaining an input signal, an estimate of a system-characterizing function, and a reference signal, each at a corresponding sample rate and each divided into a plurality of sub-bands; separately processing such sub-bands, where for a given sub-band the estimate of the system-characterizing function and the reference signal are processed to generate an echo-estimation signal and then the echo-estimation signal is subtracted from the input signal to provide an echo-corrected signal for such given sub-band; and combining the echo-corrected signal from each of different ones of the plurality of the sub-bands to provide a final output signal, with the echo-estimation signal generated using a processing sample rate that is lower than the sample rate for the input signal.
    Type: Application
    Filed: October 16, 2018
    Publication date: March 14, 2019
    Inventors: Jimeng Zheng, Yuli You
  • Patent number: 10154343
    Abstract: Provided are, among other things, systems, methods and techniques for reducing echo in an audio signal. One representative embodiment involves obtaining an input signal, an estimate of a system-characterizing function, and a reference signal, each at a corresponding sample rate and each divided into a plurality of sub-bands; separately processing such sub-bands, where for a given sub-band the estimate of the system-characterizing function and the reference signal are processed to generate an echo-estimation signal and then the echo-estimation signal is subtracted from the input signal to provide an echo-corrected signal for such given sub-band; and combining the echo-corrected signal from each of different ones of the plurality of the sub-bands to provide a final output signal, with the echo-estimation signal generated using a processing sample rate that is lower than the sample rate for the input signal.
    Type: Grant
    Filed: September 14, 2017
    Date of Patent: December 11, 2018
    Assignee: Guoguang Electric Company Limited
    Inventors: Jimeng Zheng, Yuli You