Patents Assigned to DEEPBRAIN AI INC.
-
Patent number: 12243550Abstract: A speech image providing method according to an embodiment includes generating a standby state image in which a person is in a standby state, generating a plurality of back-motion images at a preset frame interval from the standby state image for image interpolation between a preset reference frame of the standby state image, generating a speech state image in which a person is in a speech state based on a source of speech content, returning the standby state image being played to the reference frame based on the plurality of back-motion images of the standby state image, based on a point of time when the generating of the speech state image is completed, and generating a synthetic speech image in combination with frames of the speech state image from the reference frame.Type: GrantFiled: July 9, 2021Date of Patent: March 4, 2025Assignee: DEEPBRAIN AI INC.Inventor: Doo Hyun Kim
-
Patent number: 12236943Abstract: An apparatus for generating a lip sync image according to disclosed embodiment has one or more processors and a memory which stores one or more programs executed by the one or more processors. The apparatus includes a first artificial neural network model configured to generate an utterance match synthesis image by using a person background image and an utterance match audio signal corresponding to the person background image as an input, and generate an utterance mismatch synthesis image by using the person background image and an utterance mismatch audio signal not corresponding to the person background image as an input, and a second artificial neural network model configured to output classification values for an input pair in which an image and a voice match and an input pair in which an image and a voice do not match by using the input pairs as an input.Type: GrantFiled: June 8, 2021Date of Patent: February 25, 2025Assignee: DEEPBRAIN AI INC.Inventors: Guem Buel Hwang, Gyeong Su Chae
-
Patent number: 12236558Abstract: An image synthesis device according to a disclosed embodiment has one or more processors and a memory which stores one or more programs executed by the one or more processors. The image synthesis device includes a first artificial neural network provided to learn each of a first task of using a damaged image as an input to output a restored image and a second task of using an original image as an input to output a reconstructed image, and a second artificial neural network connected to an output layer of the first artificial neural network, and trained to use the reconstructed image output from the first artificial neural network as an input and improve the image quality of the reconstructed image.Type: GrantFiled: June 8, 2021Date of Patent: February 25, 2025Assignee: DEEPBRAIN AI INC.Inventors: Gyeong Su Chae, Guem Buel Hwang
-
Patent number: 12205212Abstract: A device which generates a speech moving image includes a first encoder, a second encoder, a combination unit, and an image reconstruction unit. The first encoder receives a person background image in which a portion related to speech of a person that is a video part of the speech moving image of the person is covered with a mask, extracts an image feature vector from the person background image, and compresses the extracted image feature vector. The second encoder receives a speech audio signal that is an audio part of the speech moving image, extracts a voice feature vector from the speech audio signal, and compresses the extracted voice feature vector. The combination unit generates a combination vector of the compressed image feature vector and the compressed voice feature vector. The image reconstruction unit reconstructs the speech moving image of the person with the combination as an input.Type: GrantFiled: December 8, 2020Date of Patent: January 21, 2025Assignee: DEEPBRAIN AI INC.Inventors: Gyeongsu Chae, Guembuel Hwang
-
Patent number: 12205342Abstract: A speech video generation device according to an embodiment includes a first encoder that receives an input of a first person background image of a predetermined person partially hidden by a first mask, and extracts a first image feature vector from the first person background image, a second encoder, which receives an input of a second person background image of the person partially hidden by a second mask, and extracts a second image feature vector from the second person background image, a third encoder, which receives an input of a speech audio signal of the person, and extracts a voice feature vector from the speech audio signal, a combining unit, which generates a combined vector of the first image feature vector, the second image feature vector, and the voice feature vector, and a decoder, which reconstructs a speech video of the person using the combined vector as an input.Type: GrantFiled: December 15, 2020Date of Patent: January 21, 2025Assignee: DEEPBRAIN AI INC.Inventors: Gyeongsu Chae, Guembuel Hwang
-
Patent number: 12198713Abstract: A lip sync image generation device based on machine learning according to a disclosed embodiment includes an image synthesis model, which is an artificial neural network model, and which uses a person background image and an utterance audio signal as an input to generate a lip sync image, and a lip sync discrimination model, which is an artificial neural network model, and which discriminates the degree of match between the lip sync image generated by the image synthesis model and the utterance audio signal input to the image synthesis model.Type: GrantFiled: June 17, 2021Date of Patent: January 14, 2025Assignee: DEEPBRAIN AI INC.Inventor: Gyeong Su Chae
-
Patent number: 12190480Abstract: An image synthesis device according to a disclosed embodiment is an image synthesis device has one or more processors and a memory which stores one or more programs executed by the one or more processors. The image synthesis device includes a first artificial neural network model provided to learn each of a first task of using a damaged image as an input to output a restored image and a second task of using an original image as an input to output a reconstructed image, and a second artificial neural network model trained to use the reconstructed image output from the first artificial neural network model as an input and improve the image quality of the reconstructed image.Type: GrantFiled: June 8, 2021Date of Patent: January 7, 2025Assignee: DEEPBRAIN AI INC.Inventors: Gyeong Su Chae, Guem Buel Hwang
-
Patent number: 12190903Abstract: An apparatus for generating a lip sync image according to a disclosed embodiment has one or more processors and a memory which stores one or more programs executed by the one or more processors. The apparatus includes a first artificial neural network model configured to generate an utterance synthesis image by using a person background image and an utterance audio signal corresponding to the person background image as an input, and generate a silence synthesis image by using only the person background image as an input, and a second artificial neural network model configured to output, from a preset utterance maintenance image and the first artificial neural network model, classification values for the preset utterance maintenance image and the silence synthesis image by using the silence synthesis image as an input.Type: GrantFiled: June 3, 2021Date of Patent: January 7, 2025Assignee: DEEPBRAIN AI INC.Inventors: Guem Buel Hwang, Gyeong Su Chae
-
Patent number: 12148431Abstract: A device according to an embodiment has one or more processors and a memory storing one or more programs executable by the one or more processors. The device includes a first encoder configured to receive a person background image corresponding to a video part of a speech video of a person and extract an image feature vector from the person background image, a second encoder configured to receive a speech audio signal corresponding to an audio part of the speech video and extract a voice feature vector from the speech audio signal, a combiner configured to generate a combined vector by combining the image feature vector output from the first encoder and the voice feature vector output from the second encoder, and a decoder configured to reconstruct the speech video of the person using the combined vector as an input.Type: GrantFiled: June 19, 2020Date of Patent: November 19, 2024Assignee: DEEPBRAIN AI INC.Inventors: Gyeongsu Chae, Guembuel Hwang, Sungwoo Park, Seyoung Jang
-
Patent number: 12131441Abstract: A learning device for generating an image according to an embodiment disclosed is a computing device including one or more processors and a memory storing one or more programs executed by the one or more processors. The learning device includes a first machine learning model that generates a mask for masking a portion related to speech in a person basic image with the person basic image as an input, and generates a person background image by synthesizing the person basic image and the mask.Type: GrantFiled: December 1, 2020Date of Patent: October 29, 2024Assignee: DEEPBRAIN AI INC.Inventors: Gyeongsu Chae, Guembuel Hwang
-
Patent number: 12112571Abstract: A neural network-based key point training apparatus according to an embodiment disclosed includes a key point model trained to extract key points from an input image and an image reconstruction model trained to reconstruct the input image with the key points output by the key point model as the input.Type: GrantFiled: December 1, 2020Date of Patent: October 8, 2024Assignee: DEEPBRAIN AI INC.Inventors: Gyeongsu Chae, Guembuel Hwang
-
Patent number: 12080270Abstract: An apparatus for synthesizing speech according to an embodiment is a computing apparatus that includes one or more processors and a memory storing one or more programs executed by the one or more processors. The apparatus for synthesizing speech includes a pre-processing module that marks a preset classification symbol on each of unit texts input; and a speech synthesis module that receives each unit text marked with the classification symbol and synthesizes speech uttering the unit text based on the input unit text.Type: GrantFiled: December 22, 2020Date of Patent: September 3, 2024Assignee: DEEPBRAIN AI INC.Inventors: Gyeongsu Chae, Dalhyun Kim
-
Patent number: 11972516Abstract: A device for generating a speech video according to an embodiment has one or more processor and a memory storing one or more programs executable by the one or more processors, and the device includes a video part generator configured to receive a person background image of a person and generate a video part of a speech video of the person; and an audio part generator configured to receive text, generate an audio part of the speech video of the person, and provide speech-related information occurring during the generation of the audio part to the video part generator.Type: GrantFiled: June 19, 2020Date of Patent: April 30, 2024Assignee: DEEPBRAIN AI INC.Inventors: Gyeongsu Chae, Guembuel Hwang, Sungwoo Park, Seyoung Jang
-
Patent number: 11967336Abstract: A computing device according to an embodiment is a computing device that is provided with one or more processors and a memory storing one or more programs executed by the one or more processors, the computing device includes a standby state video generating module that generates a standby state video in which a person in a video is in a standby state, a speech state video generating module that generates a speech state video in which a person in a video is in a speech state based on a source of speech content, and a video reproducing module that reproduces the standby state video, and generates a synthesized speech video by synthesizing the standby state video being reproduced and the speech state video.Type: GrantFiled: December 22, 2020Date of Patent: April 23, 2024Assignee: DEEPBRAIN AI INC.Inventor: Doohyun Kim
-
Patent number: 11830120Abstract: A computing device according to an embodiment includes one or more processors, a memory storing one or more programs executed by the one or more processors, a standby state image generating module configured to generate a standby state image in which a person is in a standby state, and generate a back-motion image set including a plurality of back-motion images at a preset frame interval from the standby state image for image interpolation between a preset reference frame of the standby state image, a speech state image generating module configured to generate a speech state image in which a person is in a speech state based on a source of speech content, and an image playback module configured to generate a synthetic speech image by combining the standby state image and the speech state image while playing the standby state image.Type: GrantFiled: July 9, 2021Date of Patent: November 28, 2023Assignee: DEEPBRAIN AI INC.Inventor: Doo Hyun Kim
-
Patent number: 11481443Abstract: A method for providing natural language conversation is implemented by an interactive agent system. The method for providing natural language conversation, according to an embodiment of the present invention includes receiving a natural language input; determining a user intent based on the natural language input by processing the natural language input, and providing a natural language response corresponding to the natural language input, based on at least one of the natural language input and the determined user intent. The natural language response may be provided by determining whether a predetermined first condition is satisfied, providing a natural language response belonging to a category of substantial replies when the first condition is satisfied, determining whether a predetermined second condition is satisfied when the first condition is not satisfied, and providing a natural language response belonging to a category of interjections when the second condition is satisfied.Type: GrantFiled: May 25, 2018Date of Patent: October 25, 2022Assignee: DEEPBRAIN AI INC.Inventors: Jaeho Seol, Seyoung Jang, Dosang Yoon
-
Patent number: 11302332Abstract: A method for providing a natural language conversation, which is implemented by an interactive agent system, may include receiving a natural language input, determining a user intent based on the natural language input, and providing a natural language response corresponding to the natural language input, based on the natural language input and/or the determined user intent, which is associated with execution of a specific task, provision of specific information, and/or a simple statement. The provision of the natural language response includes determining whether a first condition is satisfied based on whether it is possible to obtain all sufficient information from the natural language input, without having to request additional information, and when the first condition is satisfied, determining whether a second condition is satisfied and providing a natural language response belonging to a category of substantial replies when the second condition is satisfied.Type: GrantFiled: October 30, 2018Date of Patent: April 12, 2022Assignee: DEEPBRAIN AI INC.Inventors: Seyoung Jang, Dosang Yoon, Jaeho Seol