Patents by Inventor Jongmo Sung

Jongmo Sung has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20220238126
    Abstract: Methods of encoding and decoding an audio signal using a learning model and an encoder and a decoder for performing the methods are disclosed. A method of encoding an audio signal using a learning model may include extracting pitch information of the audio signal, determining a dilation factor of a receptive field of a first expandable neural network block to extract a feature map from the audio signal based on the pitch information, generating a first feature map of the audio signal using the first expandable neural network block in which the dilation factor is determined, determining a second feature map by inputting the first feature map into a second expandable neural network block to process the first feature map, and converting the second feature map and the pitch information into a bitstream.
    Type: Application
    Filed: January 7, 2022
    Publication date: July 28, 2022
    Inventors: Jongmo SUNG, Seung Kwon BEACK, Tae Jin LEE, Woo-taek LIM, Inseon JANG
  • Publication number: 20220157326
    Abstract: A method of generating a residual signal performed by an encoder includes identifying an input signal including an audio sample, generating a first residual signal from the input signal using linear predictive coding (LPC), generating a second residual signal having a less information amount than the first residual signal by transforming the first residual signal, transforming the second residual signal into a frequency domain, and generating a third residual signal having a less information amount than the second residual signal from the transformed second residual signal using frequency-domain prediction (FDP) coding.
    Type: Application
    Filed: October 21, 2021
    Publication date: May 19, 2022
    Inventors: Seung Kwon BEACK, Jongmo SUNG, Tae Jin LEE, Woo-taek LIM, Inseon JANG
  • Patent number: 11276413
    Abstract: Disclosed are an audio signal encoding method and audio signal decoding method, and an encoder and decoder performing the same. The audio signal encoding method includes applying an audio signal to a training model including N autoencoders provided in a cascade structure, encoding an output result derived through the training model, and generating a bitstream with respect to the audio signal based on the encoded output result.
    Type: Grant
    Filed: August 16, 2019
    Date of Patent: March 15, 2022
    Assignees: Electronics and Telecommunications Research Institute, THE TRUSTEES OF INDIANA UNIVERSITY
    Inventors: Mi Suk Lee, Jongmo Sung, Minje Kim, Kai Zhen
  • Publication number: 20220020385
    Abstract: An audio signal encoding method performed by an encoder includes identifying a time-domain audio signal in a unit of blocks, quantizing a linear prediction coefficient extracted from a combined block in which a current original block of the audio signal and a previous original block chronologically adjacent to the current original block using frequency-domain linear predictive coding (LPC), generating a temporal envelope by dequantizing the quantized linear prediction coefficient, extracting a residual signal from the combined block based on the temporal envelope, quantizing the residual signal by one of time-domain quantization and frequency-domain quantization, and transforming the quantized residual signal and the quantized linear prediction coefficient into a bitstream.
    Type: Application
    Filed: July 15, 2021
    Publication date: January 20, 2022
    Inventors: Seung Kwon Beack, Jongmo Sung, Mi Suk Lee, Tae Jin Lee, Woo-taek Lim, Inseon Jang, Jin Soo Choi
  • Publication number: 20220005487
    Abstract: An audio signal encoding and decoding method using a neural network model, a method of training the neural network model, and an encoder and decoder performing the methods are disclosed. The encoding method includes computing the first feature information of an input signal using a recurrent encoding model, computing an output signal from the first feature information using a recurrent decoding model, calculating a residual signal by subtracting the output signal from the input signal, computing the second feature information of the residual signal using a nonrecurrent encoding model, and converting the first feature information and the second feature information to a bitstream.
    Type: Application
    Filed: July 6, 2021
    Publication date: January 6, 2022
    Applicant: Electronics and Telecommunications Research Institute
    Inventors: Jongmo SUNG, Seung Kwon BEACK, Mi Suk LEE, Tae Jin LEE, Woo-taek LIM, Inseon JANG
  • Publication number: 20220005488
    Abstract: The encoding method includes computing the first feature information of an input signal using a recurrent encoding model, quantizing the first feature information and producing the first feature bitstream, computing the first output signal from the quantized first feature information using a recurrent decoding model, computing the second feature information of the input signal using a nonrecurrent encoding model, quantizing the second feature information and producing the second feature bitstream, computing the second output signal from the quantized second feature information using a nonrecurrent decoding model, determining an encoding mode based on the input signal, the first and second output signals, and the first and second feature bitstreams, and outputting an overall bitstream by multiplexing an encoding mode bit and one of the first feature bitstream and the second feature bitstream depending on the encoding mode.
    Type: Application
    Filed: July 6, 2021
    Publication date: January 6, 2022
    Applicant: Electronics and Telecommunications Research Institute
    Inventors: Jongmo SUNG, Seung Kwon BEACK, Mi Suk LEE, Tae Jin LEE, Woo-taek LIM, Inseon JANG
  • Publication number: 20210398547
    Abstract: An audio signal encoding method performed by an encoder includes identifying an audio signal of a time domain in units of a block, generating a combined block by combining i) a current original block of the audio signal and ii) a previous original block chronologically adjacent to the current original block, extracting a first residual signal of a frequency domain from the combined block using linear predictive coding of a time domain, overlapping chronologically adjacent first residual signals among first residual signals converted into a time domain, and quantizing a second residual signal of a time domain extracted from the overlapped first residual signal by converting the second residual signal of the time domain into a frequency domain using linear predictive coding of a frequency domain.
    Type: Application
    Filed: May 26, 2021
    Publication date: December 23, 2021
    Applicant: Electronics and Telecommunications Research Institute
    Inventors: Seung Kwon BEACK, Jongmo SUNG, Mi Suk LEE, Tae Jin LEE, Woo-taek LIM, Inseon JANG
  • Publication number: 20210390967
    Abstract: Disclosed is a method of encoding and decoding an audio signal using linear predictive coding (LPC) and an encoder and a decoder that perform the method. The method of encoding an audio signal to be performed by the encoder includes identifying a time-domain audio signal block-wise, quantizing a linear prediction coefficient obtained from a block of the audio signal through the LPC, generating an envelope based on the quantized linear prediction coefficient, extracting a residual signal based on the envelope and a result of converting the block into a frequency domain, grouping the residual signal by each sub-band and determining a scale factor for quantizing the grouped residual signal, quantizing the residual signal using the scale factor, and converting the quantized residual signal and the quantized linear prediction coefficient into a bitstream and transmitting the bitstream to a decoder.
    Type: Application
    Filed: April 28, 2021
    Publication date: December 16, 2021
    Applicant: Electronics and Telecommunications Research Institute
    Inventors: Seung Kwon Beack, Jongmo SUNG, Mi Suk LEE, Tae Jin LEE, Woo-taek LIM, Inseon JANG, Jin Soo CHOI
  • Publication number: 20210366497
    Abstract: Methods of encoding and decoding a speech signal using a neural network model that recognizes sound sources, and encoding and decoding apparatuses for performing the methods are provided. A method of encoding a speech signal includes identifying an input signal for a plurality of sound sources; generating a latent signal by encoding the input signal; obtaining a plurality of sound source signals by separating the latent signal for each of the plurality of sound sources; determining a number of bits used for quantization of each of the plurality of sound source signals according to a type of each of the plurality of sound sources; quantizing each of the plurality of sound source signals based on the determined number of bits; and generating a bitstream by combining the plurality of quantized sound source signals.
    Type: Application
    Filed: May 20, 2021
    Publication date: November 25, 2021
    Applicants: Electronics and Telecommunications Research Institute, The Trustees of Indiana University
    Inventors: Woo-taek LIM, Seung Kwon BEACK, Jongmo SUNG, Mi Suk LEE, Tae Jin LEE, Inseon JANG, Minje KIM, Haici YANG
  • Publication number: 20210350796
    Abstract: Disclosed is a speech processing apparatus and method using a densely connected hybrid neural network. The speech processing method includes inputting a time domain sample of N*1 dimension for an input speech into a densely connected hybrid network; passing the time domain sample through a plurality of dense blocks in a densely connected hybrid network; reshaping the time domain samples into M subframes by passing the time domain samples through the plurality of dense blocks, inputting the M subframes into gated recurrent unit (GRU) components of N/M-dimension; outputting clean speech from which noise is removed from the input speech by passing the M subframes through GRU components.
    Type: Application
    Filed: May 5, 2021
    Publication date: November 11, 2021
    Applicants: Electronics and Telecommunications Research Institute, The Trustees of Indiana University
    Inventors: Minje KIM, Mi Suk LEE, Seung Kwon BEACK, Jongmo SUNG, Tae Jin LEE, Jin Soo CHOI, Kai ZHEN
  • Patent number: 11133015
    Abstract: A method of predicting a channel parameter of an original signal from a downmix signal is disclosed. The method may include generating an input feature map to be used to predict a channel parameter of the original signal based on a downmix signal of an original signal, determining an output feature map including a predicted parameter to be used to predict the channel parameter by applying the input feature map to a neural network, generating a label map including information associated with the channel parameter of the original signal, and predicting the channel parameter of the original signal by comparing the output feature map and the label map.
    Type: Grant
    Filed: November 5, 2018
    Date of Patent: September 28, 2021
    Assignee: Electronics and Telecommunications Research Institute
    Inventors: Seung Kwon Beack, Woo-taek Lim, Jongmo Sung, Mi Suk Lee, Tae Jin Lee, Hui Yong Kim
  • Publication number: 20210233547
    Abstract: A method and apparatus for processing an audio signal are disclosed. According to an example embodiment, a method of processing an audio signal may include acquiring a final audio signal for an initial audio signal using a plurality of neural network models generating output audio signals by encoding and decoding input audio signals, calculating a difference between the initial audio signal and the final audio signal in a time domain, converting the initial audio signal and the final audio signal into Mel-spectra, calculating a difference between the Mel-spectra of the initial audio signal and the final audio signal in a frequency domain, training the plurality of neural network models based on results calculated in the time domain and the frequency domain, and generating a new final audio signal distinguished from the final audio signal from the initial audio signal using the trained neural network models.
    Type: Application
    Filed: January 22, 2021
    Publication date: July 29, 2021
    Applicants: Electronics and Telecommunications Research Institute, The Trustees of Indiana University
    Inventors: Mi Suk LEE, Seung Kwon BEACK, Jongmo SUNG, Tae Jin LEE, Jin Soo CHOI, Minje KIM, Kai ZHEN
  • Publication number: 20210174815
    Abstract: Disclosed are a quantizing method for a latent vector and a computing device for performing the quantization method. A quantizing method of a latent vector includes performing information shaping on the latent vector resulting from reduction in a dimension of an input signal using a target neural network; clamping a residual signal of the latent vector derived based on the information shaping; performing resealing on the clamped residual signal; and performing quantization on the resealed residual signal.
    Type: Application
    Filed: December 4, 2020
    Publication date: June 10, 2021
    Applicant: Electronics and Telecommunications Research Institute
    Inventors: Seung Kwon BEACK, Jooyoung LEE, Jongmo SUNG, Mi Suk LEE, Tae Jin LEE, Woo-taek LIM, Seunghyun CHO, Jin Soo CHOI
  • Publication number: 20210166706
    Abstract: Disclosed is an apparatus and method for encoding/decoding an audio signal using information of a previous frame. An audio signal encoding method includes: generating a current latent vector by reducing dimension of a current frame of an audio signal; generating a concatenation vector by concatenating a previous latent vector generated by reducing dimension of a previous frame of the audio signal with the current latent vector; and encoding and quantizing the concatenation vector.
    Type: Application
    Filed: November 27, 2020
    Publication date: June 3, 2021
    Applicant: Electronics and Telecommunications Research Institute
    Inventors: Woo-taek LIM, Seung Kwon BEACK, Jongmo SUNG, Mi Suk LEE, Tae Jin LEE
  • Publication number: 20210166701
    Abstract: An audio signal encoding/decoding device and method using a filter bank is disclosed. The audio signal encoding method includes generating a plurality of first audio signals by performing filtering on an input audio signal using an analysis filter bank, generating a plurality of second audio signals by performing downsampling on the first audio signals, and outputting a bitstream by encoding and quantizing the second audio signals.
    Type: Application
    Filed: November 25, 2020
    Publication date: June 3, 2021
    Inventors: Woo-taek LIM, Seung Kwon BEACK, Jongmo SUNG, Mi Suk LEE, Tae Jin LEE
  • Publication number: 20210142812
    Abstract: Disclosed are a method for coding a residual signal of LPC coefficients based on collaborative quantization and a computing device for performing the method. The residual signal coding method includes: generating encoded LPC coefficients and LPC residual signals by performing LPC analysis and quantization on an input speech; Determining a predicted LPC residual signal by applying the LPC residual signal to cross module residual learning; Performing LPC synthesis using the coded LPC coefficients and the predicted LPC residual signal; It may include the step of determining an output speech that is a synthesized output according to a result of performing the LPC synthesis.
    Type: Application
    Filed: November 13, 2020
    Publication date: May 13, 2021
    Applicants: Electronics and Telecommunications Research Institute, The Trustees of Indiana University
    Inventors: Minje KIM, Kai ZHEN, Mi Suk LEE, Seung Kwon BEACK, Jongmo SUNG, Tae Jin LEE, Jin Soo CHOI
  • Publication number: 20210074306
    Abstract: Provided are an audio encoding method, an audio decoding method, an audio encoding apparatus, and an audio decoding apparatus using dynamic model parameters. The audio encoding method using dynamic model parameters may use dynamic model parameters corresponding to each of the levels of the encoding network when reducing the dimension of an audio signal in the encoding network. In addition, the audio decoding method using the dynamic model parameter may use a dynamic model parameter corresponding to each of the levels of the decoding network when extending the dimension of an audio signal in an encoding network.
    Type: Application
    Filed: September 10, 2020
    Publication date: March 11, 2021
    Applicant: Electronics and Telecommunications Research Institute
    Inventors: Jongmo SUNG, Seung Kwon BEACK, Mi Suk LEE, Tae Jin LEE, Woo-taek LIM, Jin Soo CHOI
  • Publication number: 20210005209
    Abstract: Disclosed are a method of encoding a high band of an audio, a method of decoding a high band of an audio, and an encoder and a decoder for performing the methods. The method of decoding a high band of an audio, the method performed by a decoder, includes identifying a parameter extracted through a first neural network, identifying side information extracted through a second neural network, and restoring a high band of an audio by applying the parameter and the side information to a third neural network.
    Type: Application
    Filed: March 10, 2020
    Publication date: January 7, 2021
    Applicants: Electronics and Telecommunications Research Institute, Kwangwoon University Industry-Academic Collaboration Foundation
    Inventors: Seung Kwon BEACK, Jongmo SUNG, Mi Suk LEE, Tae Jin LEE, Hochong PARK
  • Publication number: 20210005208
    Abstract: Disclosed is a method of processing a residual signal for audio coding and an audio coding apparatus. The method learns a feature map of a reference signal through a residual signal learning engine including a convolutional layer and a neural network and performs learning based on a result obtained by mapping a node of an output layer of the neural network and a quantization level of index of the residual signal.
    Type: Application
    Filed: November 18, 2019
    Publication date: January 7, 2021
    Applicant: Electronics and Telecommunications Research Institute
    Inventors: Seung Kwon Beack, Jongmo Sung, Mi Suk Lee, Tae Jin Lee, Hui Yong Kim
  • Patent number: 10839819
    Abstract: Provided is an apparatus and method for encoding/decoding audio based on a block. A method of encoding an audio signal may include dividing each of frame of input signal that constitute an audio signal into a plurality of subframes; transforming the subframes to a frequency domain; determining a two-dimensional (2D) intra block using the subframes transformed to the frequency domain; and encoding the 2D intra block. The 2D intra block may be a block that two-dimensionally displays frequency coefficients of the subframes transformed to the frequency domain using a time and a frequency.
    Type: Grant
    Filed: March 21, 2017
    Date of Patent: November 17, 2020
    Assignee: Electronics and Telecommunications Research Institute
    Inventors: Seung Kwon Beack, Tae Jin Lee, Jongmo Sung, Mi Suk Lee, Dae Young Jang, Jin Soo Choi