Patents Assigned to DEEPBRAIN AI INC.
  • Patent number: 12243550
    Abstract: A speech image providing method according to an embodiment includes generating a standby state image in which a person is in a standby state, generating a plurality of back-motion images at a preset frame interval from the standby state image for image interpolation between a preset reference frame of the standby state image, generating a speech state image in which a person is in a speech state based on a source of speech content, returning the standby state image being played to the reference frame based on the plurality of back-motion images of the standby state image, based on a point of time when the generating of the speech state image is completed, and generating a synthetic speech image in combination with frames of the speech state image from the reference frame.
    Type: Grant
    Filed: July 9, 2021
    Date of Patent: March 4, 2025
    Assignee: DEEPBRAIN AI INC.
    Inventor: Doo Hyun Kim
  • Patent number: 12236943
    Abstract: An apparatus for generating a lip sync image according to disclosed embodiment has one or more processors and a memory which stores one or more programs executed by the one or more processors. The apparatus includes a first artificial neural network model configured to generate an utterance match synthesis image by using a person background image and an utterance match audio signal corresponding to the person background image as an input, and generate an utterance mismatch synthesis image by using the person background image and an utterance mismatch audio signal not corresponding to the person background image as an input, and a second artificial neural network model configured to output classification values for an input pair in which an image and a voice match and an input pair in which an image and a voice do not match by using the input pairs as an input.
    Type: Grant
    Filed: June 8, 2021
    Date of Patent: February 25, 2025
    Assignee: DEEPBRAIN AI INC.
    Inventors: Guem Buel Hwang, Gyeong Su Chae
  • Patent number: 12236558
    Abstract: An image synthesis device according to a disclosed embodiment has one or more processors and a memory which stores one or more programs executed by the one or more processors. The image synthesis device includes a first artificial neural network provided to learn each of a first task of using a damaged image as an input to output a restored image and a second task of using an original image as an input to output a reconstructed image, and a second artificial neural network connected to an output layer of the first artificial neural network, and trained to use the reconstructed image output from the first artificial neural network as an input and improve the image quality of the reconstructed image.
    Type: Grant
    Filed: June 8, 2021
    Date of Patent: February 25, 2025
    Assignee: DEEPBRAIN AI INC.
    Inventors: Gyeong Su Chae, Guem Buel Hwang
  • Patent number: 12205212
    Abstract: A device which generates a speech moving image includes a first encoder, a second encoder, a combination unit, and an image reconstruction unit. The first encoder receives a person background image in which a portion related to speech of a person that is a video part of the speech moving image of the person is covered with a mask, extracts an image feature vector from the person background image, and compresses the extracted image feature vector. The second encoder receives a speech audio signal that is an audio part of the speech moving image, extracts a voice feature vector from the speech audio signal, and compresses the extracted voice feature vector. The combination unit generates a combination vector of the compressed image feature vector and the compressed voice feature vector. The image reconstruction unit reconstructs the speech moving image of the person with the combination as an input.
    Type: Grant
    Filed: December 8, 2020
    Date of Patent: January 21, 2025
    Assignee: DEEPBRAIN AI INC.
    Inventors: Gyeongsu Chae, Guembuel Hwang
  • Patent number: 12205342
    Abstract: A speech video generation device according to an embodiment includes a first encoder that receives an input of a first person background image of a predetermined person partially hidden by a first mask, and extracts a first image feature vector from the first person background image, a second encoder, which receives an input of a second person background image of the person partially hidden by a second mask, and extracts a second image feature vector from the second person background image, a third encoder, which receives an input of a speech audio signal of the person, and extracts a voice feature vector from the speech audio signal, a combining unit, which generates a combined vector of the first image feature vector, the second image feature vector, and the voice feature vector, and a decoder, which reconstructs a speech video of the person using the combined vector as an input.
    Type: Grant
    Filed: December 15, 2020
    Date of Patent: January 21, 2025
    Assignee: DEEPBRAIN AI INC.
    Inventors: Gyeongsu Chae, Guembuel Hwang
  • Patent number: 12198713
    Abstract: A lip sync image generation device based on machine learning according to a disclosed embodiment includes an image synthesis model, which is an artificial neural network model, and which uses a person background image and an utterance audio signal as an input to generate a lip sync image, and a lip sync discrimination model, which is an artificial neural network model, and which discriminates the degree of match between the lip sync image generated by the image synthesis model and the utterance audio signal input to the image synthesis model.
    Type: Grant
    Filed: June 17, 2021
    Date of Patent: January 14, 2025
    Assignee: DEEPBRAIN AI INC.
    Inventor: Gyeong Su Chae
  • Patent number: 12190480
    Abstract: An image synthesis device according to a disclosed embodiment is an image synthesis device has one or more processors and a memory which stores one or more programs executed by the one or more processors. The image synthesis device includes a first artificial neural network model provided to learn each of a first task of using a damaged image as an input to output a restored image and a second task of using an original image as an input to output a reconstructed image, and a second artificial neural network model trained to use the reconstructed image output from the first artificial neural network model as an input and improve the image quality of the reconstructed image.
    Type: Grant
    Filed: June 8, 2021
    Date of Patent: January 7, 2025
    Assignee: DEEPBRAIN AI INC.
    Inventors: Gyeong Su Chae, Guem Buel Hwang
  • Patent number: 12190903
    Abstract: An apparatus for generating a lip sync image according to a disclosed embodiment has one or more processors and a memory which stores one or more programs executed by the one or more processors. The apparatus includes a first artificial neural network model configured to generate an utterance synthesis image by using a person background image and an utterance audio signal corresponding to the person background image as an input, and generate a silence synthesis image by using only the person background image as an input, and a second artificial neural network model configured to output, from a preset utterance maintenance image and the first artificial neural network model, classification values for the preset utterance maintenance image and the silence synthesis image by using the silence synthesis image as an input.
    Type: Grant
    Filed: June 3, 2021
    Date of Patent: January 7, 2025
    Assignee: DEEPBRAIN AI INC.
    Inventors: Guem Buel Hwang, Gyeong Su Chae
  • Patent number: 12148431
    Abstract: A device according to an embodiment has one or more processors and a memory storing one or more programs executable by the one or more processors. The device includes a first encoder configured to receive a person background image corresponding to a video part of a speech video of a person and extract an image feature vector from the person background image, a second encoder configured to receive a speech audio signal corresponding to an audio part of the speech video and extract a voice feature vector from the speech audio signal, a combiner configured to generate a combined vector by combining the image feature vector output from the first encoder and the voice feature vector output from the second encoder, and a decoder configured to reconstruct the speech video of the person using the combined vector as an input.
    Type: Grant
    Filed: June 19, 2020
    Date of Patent: November 19, 2024
    Assignee: DEEPBRAIN AI INC.
    Inventors: Gyeongsu Chae, Guembuel Hwang, Sungwoo Park, Seyoung Jang
  • Patent number: 12131441
    Abstract: A learning device for generating an image according to an embodiment disclosed is a computing device including one or more processors and a memory storing one or more programs executed by the one or more processors. The learning device includes a first machine learning model that generates a mask for masking a portion related to speech in a person basic image with the person basic image as an input, and generates a person background image by synthesizing the person basic image and the mask.
    Type: Grant
    Filed: December 1, 2020
    Date of Patent: October 29, 2024
    Assignee: DEEPBRAIN AI INC.
    Inventors: Gyeongsu Chae, Guembuel Hwang
  • Patent number: 12112571
    Abstract: A neural network-based key point training apparatus according to an embodiment disclosed includes a key point model trained to extract key points from an input image and an image reconstruction model trained to reconstruct the input image with the key points output by the key point model as the input.
    Type: Grant
    Filed: December 1, 2020
    Date of Patent: October 8, 2024
    Assignee: DEEPBRAIN AI INC.
    Inventors: Gyeongsu Chae, Guembuel Hwang
  • Patent number: 12080270
    Abstract: An apparatus for synthesizing speech according to an embodiment is a computing apparatus that includes one or more processors and a memory storing one or more programs executed by the one or more processors. The apparatus for synthesizing speech includes a pre-processing module that marks a preset classification symbol on each of unit texts input; and a speech synthesis module that receives each unit text marked with the classification symbol and synthesizes speech uttering the unit text based on the input unit text.
    Type: Grant
    Filed: December 22, 2020
    Date of Patent: September 3, 2024
    Assignee: DEEPBRAIN AI INC.
    Inventors: Gyeongsu Chae, Dalhyun Kim
  • Patent number: 11972516
    Abstract: A device for generating a speech video according to an embodiment has one or more processor and a memory storing one or more programs executable by the one or more processors, and the device includes a video part generator configured to receive a person background image of a person and generate a video part of a speech video of the person; and an audio part generator configured to receive text, generate an audio part of the speech video of the person, and provide speech-related information occurring during the generation of the audio part to the video part generator.
    Type: Grant
    Filed: June 19, 2020
    Date of Patent: April 30, 2024
    Assignee: DEEPBRAIN AI INC.
    Inventors: Gyeongsu Chae, Guembuel Hwang, Sungwoo Park, Seyoung Jang
  • Patent number: 11967336
    Abstract: A computing device according to an embodiment is a computing device that is provided with one or more processors and a memory storing one or more programs executed by the one or more processors, the computing device includes a standby state video generating module that generates a standby state video in which a person in a video is in a standby state, a speech state video generating module that generates a speech state video in which a person in a video is in a speech state based on a source of speech content, and a video reproducing module that reproduces the standby state video, and generates a synthesized speech video by synthesizing the standby state video being reproduced and the speech state video.
    Type: Grant
    Filed: December 22, 2020
    Date of Patent: April 23, 2024
    Assignee: DEEPBRAIN AI INC.
    Inventor: Doohyun Kim
  • Patent number: 11830120
    Abstract: A computing device according to an embodiment includes one or more processors, a memory storing one or more programs executed by the one or more processors, a standby state image generating module configured to generate a standby state image in which a person is in a standby state, and generate a back-motion image set including a plurality of back-motion images at a preset frame interval from the standby state image for image interpolation between a preset reference frame of the standby state image, a speech state image generating module configured to generate a speech state image in which a person is in a speech state based on a source of speech content, and an image playback module configured to generate a synthetic speech image by combining the standby state image and the speech state image while playing the standby state image.
    Type: Grant
    Filed: July 9, 2021
    Date of Patent: November 28, 2023
    Assignee: DEEPBRAIN AI INC.
    Inventor: Doo Hyun Kim
  • Patent number: 11481443
    Abstract: A method for providing natural language conversation is implemented by an interactive agent system. The method for providing natural language conversation, according to an embodiment of the present invention includes receiving a natural language input; determining a user intent based on the natural language input by processing the natural language input, and providing a natural language response corresponding to the natural language input, based on at least one of the natural language input and the determined user intent. The natural language response may be provided by determining whether a predetermined first condition is satisfied, providing a natural language response belonging to a category of substantial replies when the first condition is satisfied, determining whether a predetermined second condition is satisfied when the first condition is not satisfied, and providing a natural language response belonging to a category of interjections when the second condition is satisfied.
    Type: Grant
    Filed: May 25, 2018
    Date of Patent: October 25, 2022
    Assignee: DEEPBRAIN AI INC.
    Inventors: Jaeho Seol, Seyoung Jang, Dosang Yoon
  • Patent number: 11302332
    Abstract: A method for providing a natural language conversation, which is implemented by an interactive agent system, may include receiving a natural language input, determining a user intent based on the natural language input, and providing a natural language response corresponding to the natural language input, based on the natural language input and/or the determined user intent, which is associated with execution of a specific task, provision of specific information, and/or a simple statement. The provision of the natural language response includes determining whether a first condition is satisfied based on whether it is possible to obtain all sufficient information from the natural language input, without having to request additional information, and when the first condition is satisfied, determining whether a second condition is satisfied and providing a natural language response belonging to a category of substantial replies when the second condition is satisfied.
    Type: Grant
    Filed: October 30, 2018
    Date of Patent: April 12, 2022
    Assignee: DEEPBRAIN AI INC.
    Inventors: Seyoung Jang, Dosang Yoon, Jaeho Seol