Patents by Inventor Jongmo Sung

Jongmo Sung has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11651778
    Abstract: Disclosed are methods of encoding and decoding an audio signal, and an encoder and a decoder for performing the methods. The method of encoding an audio signal includes identifying an input signal corresponding to a low frequency band of the audio signal, windowing the input signal, generating a first latent vector by inputting the windowed input signal to a first encoding model, transforming the windowed input signal into a frequency domain, generating a second latent vector by inputting the transformed input signal to a second encoding model, generating a final latent vector by combining the first latent vector and the second latent vector, and generating a bitstream corresponding to the final latent vector.
    Type: Grant
    Filed: November 8, 2021
    Date of Patent: May 16, 2023
    Assignee: ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTITUTE
    Inventors: Woo-taek Lim, Seung Kwon Beack, Jongmo Sung, Tae Jin Lee, Inseon Jang, Jong-Won Seok, Yunsu Kim
  • Publication number: 20230048402
    Abstract: Provided is an encoding method according to various example embodiments and an encoder performing the method. The encoding method includes outputting a linear prediction(LP) coefficients bitstream and a residual signal by performing a linear prediction analysis on an input signal, outputting a first latent signal obtained by encoding a periodic component of the residual signal, using a first neural network module, outputting a first bitstream obtained by quantizing the first latent signal, using a quantization module, outputting a second latent signal obtained by encoding an aperiodic component of the residual signal, using the first neural network module, and outputting a second bitstream obtained by quantizing the second latent signal, using the quantization module, wherein the aperiodic component of the residual signal is calculated based on a periodic component of the residual signal decoded from the quantized first latent signal output by de-quantizing the first bitstream.
    Type: Application
    Filed: August 9, 2022
    Publication date: February 16, 2023
    Inventors: Jongmo SUNG, Seung Kwon BEACK, Tae Jin LEE, Woo-taek KIM, Inseon JANG
  • Patent number: 11580999
    Abstract: An audio signal encoding method performed by an encoder includes identifying an audio signal of a time domain in units of a block, generating a combined block by combining i) a current original block of the audio signal and ii) a previous original block chronologically adjacent to the current original block, extracting a first residual signal of a frequency domain from the combined block using linear predictive coding of a time domain, overlapping chronologically adjacent first residual signals among first residual signals converted into a time domain, and quantizing a second residual signal of a time domain extracted from the overlapped first residual signal by converting the second residual signal of the time domain into a frequency domain using linear predictive coding of a frequency domain.
    Type: Grant
    Filed: May 26, 2021
    Date of Patent: February 14, 2023
    Assignee: Electronics and Telecommunications Research Institute
    Inventors: Seung Kwon Beack, Jongmo Sung, Mi Suk Lee, Tae Jin Lee, Woo-taek Lim, Inseon Jang
  • Patent number: 11581000
    Abstract: Disclosed is an apparatus and method for encoding/decoding an audio signal using information of a previous frame. An audio signal encoding method includes: generating a current latent vector by reducing dimension of a current frame of an audio signal; generating a concatenation vector by concatenating a previous latent vector generated by reducing dimension of a previous frame of the audio signal with the current latent vector; and encoding and quantizing the concatenation vector.
    Type: Grant
    Filed: November 27, 2020
    Date of Patent: February 14, 2023
    Assignee: Electronics and Telecommunications Research Institute
    Inventors: Woo-Taek Lim, Seung Kwon Beack, Jongmo Sung, Mi Suk Lee, Tae Jin Lee
  • Publication number: 20230038394
    Abstract: Disclosed are a method of encoding and decoding an audio signal and an encoder and a decoder performing the method. The method of encoding an audio signal includes identifying an input signal, and generating a bitstring of each encoding layer by applying, to the input signal, an encoding model including a plurality of successive encoding layers that encodes the input signal, in which a current encoding layer among the encoding layers is trained to generate a bitstring of the current encoding layer by encoding an encoded signal which is a signal encoded in a previous encoding layer and quantizing an encoded signal which is a signal encoded in the current encoding layer.
    Type: Application
    Filed: July 30, 2021
    Publication date: February 9, 2023
    Inventors: Woo-taek LIM, Seung Kwon BEACK, Jongmo SUNG, Tae Jin LEE, Inseon JANG, Minje KIM
  • Publication number: 20230039546
    Abstract: An audio encoding/decoding apparatus and method using vector quantized residual error features are disclosed. An audio signal encoding method includes outputting a bitstream of a main codec by encoding an original signal, decoding the bitstream of the main codec, determining a residual error feature vector from a feature vector of a decoded signal and a feature vector of the original signal, and outputting a bitstream of additional information by encoding the residual error feature vector.
    Type: Application
    Filed: April 1, 2022
    Publication date: February 9, 2023
    Inventors: Inseon JANG, Seung Kwon BEACK, Jongmo SUNG, Tae Jin LEE, Woo-taek LIM, Jongwon SHIN, Youngju CHEON, Sangwook HAN, Soojoong HWANG
  • Patent number: 11562757
    Abstract: An audio signal encoding method performed by an encoder includes identifying a time-domain audio signal in a unit of blocks, quantizing a linear prediction coefficient extracted from a combined block in which a current original block of the audio signal and a previous original block chronologically adjacent to the current original block using frequency-domain linear predictive coding (LPC), generating a temporal envelope by dequantizing the quantized linear prediction coefficient, extracting a residual signal from the combined block based on the temporal envelope, quantizing the residual signal by one of time-domain quantization and frequency-domain quantization, and transforming the quantized residual signal and the quantized linear prediction coefficient into a bitstream.
    Type: Grant
    Filed: July 15, 2021
    Date of Patent: January 24, 2023
    Assignee: ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTITUTE
    Inventors: Seung Kwon Beack, Jongmo Sung, Mi Suk Lee, Tae Jin Lee, Woo-taek Lim, Inseon Jang, Jin Soo Choi
  • Publication number: 20220375483
    Abstract: Disclosed are methods of encoding and decoding an audio signal, and an encoder and a decoder for performing the methods. The method of encoding an audio signal includes identifying an input signal corresponding to a low frequency band of the audio signal, windowing the input signal, generating a first latent vector by inputting the windowed input signal to a first encoding model, transforming the windowed input signal into a frequency domain, generating a second latent vector by inputting the transformed input signal to a second encoding model, generating a final latent vector by combining the first latent vector and the second latent vector, and generating a bitstream corresponding to the final latent vector.
    Type: Application
    Filed: November 8, 2021
    Publication date: November 24, 2022
    Inventors: Woo-taek LIM, Seung Kwon BEACK, Jongmo SUNG, Tae Jin LEE, Inseon JANG, Jong-won SEOK, YUNSU KIM
  • Patent number: 11508386
    Abstract: An inventive concept relates to an audio coding method to which CNN-based frequency spectrum recovery is applied. An inventive concept transmits a part of frequency spectral coefficients generated in transform coding to a decoder and the decoder recovers the frequency spectral coefficient not transmitted. Furthermore, the signs of frequency spectral coefficient are transmitted from an encoder to the decoder depending on a sign transmission rule.
    Type: Grant
    Filed: April 8, 2020
    Date of Patent: November 22, 2022
    Assignees: Electronics and Telecommunications Research Institute, Kwangwoon University Industry-Academic Collaboration Foundation
    Inventors: Hochong Park, Seung Kwon Beack, Jongmo Sung, Seong-Hyeon Shin, Mi Suk Lee, Tae Jin Lee, Jin Soo Choi
  • Patent number: 11508385
    Abstract: Disclosed is a method of processing a residual signal for audio coding and an audio coding apparatus. The method learns a feature map of a reference signal through a residual signal learning engine including a convolutional layer and a neural network and performs learning based on a result obtained by mapping a node of an output layer of the neural network and a quantization level of index of the residual signal.
    Type: Grant
    Filed: November 18, 2019
    Date of Patent: November 22, 2022
    Assignee: Electronics and Telecommunications Research Institute
    Inventors: Seung Kwon Beack, Jongmo Sung, Mi Suk Lee, Tae Jin Lee, Hui Yong Kim
  • Publication number: 20220358940
    Abstract: Disclosed are methods of encoding and decoding an audio signal using side information, and an encoder and a decoder for performing the methods. The method of encoding an audio signal using side information includes identifying an input signal, the input signal being an original audio signal, extracting side information from the input signal using a learning model trained to extract side information from a feature vector of the input signal, encoding the input signal, and generating a bitstream by combining the encoded input signal and the side information.
    Type: Application
    Filed: November 16, 2021
    Publication date: November 10, 2022
    Inventors: Woo-taek LIM, Seung Kwon BEACK, Jongmo SUNG, Tae Jin LEE, Inseon JANG, Jong Won SHIN, Soojoong HWANG, Youngju CHEON, Sangwook HAN
  • Patent number: 11488613
    Abstract: Disclosed are a method for coding a residual signal of LPC coefficients based on collaborative quantization and a computing device for performing the method. The residual signal coding method includes: generating encoded LPC coefficients and LPC residual signals by performing LPC analysis and quantization on an input speech; Determining a predicted LPC residual signal by applying the LPC residual signal to cross module residual learning; Performing LPC synthesis using the coded LPC coefficients and the predicted LPC residual signal; It may include the step of determining an output speech that is a synthesized output according to a result of performing the LPC synthesis.
    Type: Grant
    Filed: November 13, 2020
    Date of Patent: November 1, 2022
    Assignees: Electronics and Telecommunications Research Institute, The Trustees of Indiana University
    Inventors: Minje Kim, Kai Zhen, Mi Suk Lee, Seung Kwon Beack, Jongmo Sung, Tae Jin Lee, Jin Soo Choi
  • Publication number: 20220335963
    Abstract: An audio signal encoding and decoding method using a neural network model, and an encoder and decoder for performing the same are disclosed. A method of encoding an audio signal using a neural network model, the method may include identifying an input signal, generating a quantized latent vector by inputting the input signal into a neural network model encoding the input signal, and generating a bitstream corresponding to the quantized latent vector, wherein the neural network model may include i) a feature extraction layer generating a latent vector by extracting a feature of the input signal, ii) a plurality of downsampling blocks downsampling the latent vector, and iii) a plurality of quantization blocks performing quantization of a downsampled latent vector.
    Type: Application
    Filed: February 11, 2022
    Publication date: October 20, 2022
    Inventors: Inseon JANG, Seung Kwon BEACK, Jongmo SUNG, Tae Jin LEE, Woo-taek LIM, Hong-Goo KANG, Jihyun LEE, Chanwoo LEE, Hyungseob LIM
  • Patent number: 11456001
    Abstract: Disclosed are a method of encoding a high band of an audio, a method of decoding a high band of an audio, and an encoder and a decoder for performing the methods. The method of decoding a high band of an audio, the method performed by a decoder, includes identifying a parameter extracted through a first neural network, identifying side information extracted through a second neural network, and restoring a high band of an audio by applying the parameter and the side information to a third neural network.
    Type: Grant
    Filed: March 10, 2020
    Date of Patent: September 27, 2022
    Assignees: Electronics and Telecommunications Research Institute, KWANGWOON UNIVERSITY INDUSTRY-ACADEMIC COLLABORATION FOUNDATION
    Inventors: Seung Kwon Beack, Jongmo Sung, Mi Suk Lee, Tae Jin Lee, Hochong Park
  • Publication number: 20220262378
    Abstract: An audio signal encoding and decoding method using a learning model, a training method of the learning model, and an encoder and decoder that perform the method, are disclosed. The audio signal decoding method may include extracting a first residual signal and a first linear prediction coefficient by decoding a bitstream received from an encoder, generating a first audio signal from the first residual signal using the first linear prediction coefficient, generating a second linear prediction coefficients and a second residual signal from the first audio signal, obtaining a third linear prediction coefficient by inputting the second linear prediction coefficient into a trained learning model, and generating a second audio signal from the second residual signal using the third linear prediction coefficient.
    Type: Application
    Filed: February 15, 2022
    Publication date: August 18, 2022
    Inventors: Woo-taek LIM, Seung Kwon BEACK, Jongmo SUNG, Tae Jin LEE, Inseon JANG
  • Patent number: 11416742
    Abstract: Provided is a training method of a neural network that is applied to an audio signal encoding method using an audio signal encoding apparatus, the training method including generating a masking threshold of a first audio signal before training is performed, calculating a weight matrix to be applied to a frequency component of the first audio signal based on the masking threshold, generating a weighted error function obtained by correcting a preset error function using the weight matrix, and generating a second audio signal by applying a parameter learned using the weighted error function to the first audio signal.
    Type: Grant
    Filed: September 5, 2018
    Date of Patent: August 16, 2022
    Assignees: Electronics and Telecommunications Research Institute, THE TRUSTEES OF INDIANA UNIVERSITY
    Inventors: Jongmo Sung, Minje Kim, Aswin Sivaraman, Kai Zhen
  • Publication number: 20220238126
    Abstract: Methods of encoding and decoding an audio signal using a learning model and an encoder and a decoder for performing the methods are disclosed. A method of encoding an audio signal using a learning model may include extracting pitch information of the audio signal, determining a dilation factor of a receptive field of a first expandable neural network block to extract a feature map from the audio signal based on the pitch information, generating a first feature map of the audio signal using the first expandable neural network block in which the dilation factor is determined, determining a second feature map by inputting the first feature map into a second expandable neural network block to process the first feature map, and converting the second feature map and the pitch information into a bitstream.
    Type: Application
    Filed: January 7, 2022
    Publication date: July 28, 2022
    Inventors: Jongmo SUNG, Seung Kwon BEACK, Tae Jin LEE, Woo-taek LIM, Inseon JANG
  • Publication number: 20220157326
    Abstract: A method of generating a residual signal performed by an encoder includes identifying an input signal including an audio sample, generating a first residual signal from the input signal using linear predictive coding (LPC), generating a second residual signal having a less information amount than the first residual signal by transforming the first residual signal, transforming the second residual signal into a frequency domain, and generating a third residual signal having a less information amount than the second residual signal from the transformed second residual signal using frequency-domain prediction (FDP) coding.
    Type: Application
    Filed: October 21, 2021
    Publication date: May 19, 2022
    Inventors: Seung Kwon BEACK, Jongmo SUNG, Tae Jin LEE, Woo-taek LIM, Inseon JANG
  • Patent number: 11276413
    Abstract: Disclosed are an audio signal encoding method and audio signal decoding method, and an encoder and decoder performing the same. The audio signal encoding method includes applying an audio signal to a training model including N autoencoders provided in a cascade structure, encoding an output result derived through the training model, and generating a bitstream with respect to the audio signal based on the encoded output result.
    Type: Grant
    Filed: August 16, 2019
    Date of Patent: March 15, 2022
    Assignees: Electronics and Telecommunications Research Institute, THE TRUSTEES OF INDIANA UNIVERSITY
    Inventors: Mi Suk Lee, Jongmo Sung, Minje Kim, Kai Zhen
  • Publication number: 20220020385
    Abstract: An audio signal encoding method performed by an encoder includes identifying a time-domain audio signal in a unit of blocks, quantizing a linear prediction coefficient extracted from a combined block in which a current original block of the audio signal and a previous original block chronologically adjacent to the current original block using frequency-domain linear predictive coding (LPC), generating a temporal envelope by dequantizing the quantized linear prediction coefficient, extracting a residual signal from the combined block based on the temporal envelope, quantizing the residual signal by one of time-domain quantization and frequency-domain quantization, and transforming the quantized residual signal and the quantized linear prediction coefficient into a bitstream.
    Type: Application
    Filed: July 15, 2021
    Publication date: January 20, 2022
    Inventors: Seung Kwon Beack, Jongmo Sung, Mi Suk Lee, Tae Jin Lee, Woo-taek Lim, Inseon Jang, Jin Soo Choi