Patents by Inventor Jinmiao Huang

Jinmiao Huang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20250173937
    Abstract: A method for speech-driven three dimensional (3D) facial animation can include receiving an input speech audio signal, generating a speaker style vector from the input speech audio signal based on a speaker style embedding model, inputting the input speech audio signal and the speaker style vector into a mesh generation model and generating vertex position information for a 3D facial animation based on the input speech audio signal and the speaker style vector, and outputting the vertex position information.
    Type: Application
    Filed: November 25, 2024
    Publication date: May 29, 2025
    Applicant: LG ELECTRONICS INC.
    Inventors: Mathieu TULI, Eu Wern TEH, Felix TAUBNER, Jinmiao HUANG, Prashant RAINA
  • Publication number: 20250173967
    Abstract: A method for controlling an artificial intelligence (AI) device can include receiving, by a processor, an input two dimensional (2D) image, encoding the 2D image to generate an image feature map, obtaining UV positional encoding information based on a three-dimensional (3D) face model where the UV positional encoding information includes a UV feature map, generating, by the processor, a correlation volume based on the image feature map and the UV feature map, generating flow map information and uncertainty information based on the correlation volume and the UV positional encoding information, and generating probabilistic 2D alignment information based on the flow map information and uncertainty information and outputting the probabilistic 2D alignment information. Also, the method can further include generating 3D reconstruction information based on the probabilistic 2D alignment information for animating a 3D face based on the input 2D image.
    Type: Application
    Filed: November 25, 2024
    Publication date: May 29, 2025
    Applicant: LG ELECTRONICS INC.
    Inventors: Felix TAUBNER, Prashant RAINA, Eu Wern TEH, Mathieu TULI, Jinmiao HUANG
  • Publication number: 20250173936
    Abstract: A method for neural network driven vertex animation can include receiving, by an encoder component of a trained neural network, an input driving signal including audio data, processing, by the encoder component, the input driving signal to generate blendshape coefficient information based on the input driving signal, transmitting, by the encoder component, the blendshape coefficient information to a decoder component of the trained neural network, receiving, by the decoder component, the blendshape coefficient information from the encoder component, and generating, by the decoder component, vertex position information based on the blendshape coefficient information for animating a 3D model.
    Type: Application
    Filed: November 25, 2024
    Publication date: May 29, 2025
    Applicant: LG ELECTRONICS INC.
    Inventors: Felix TAUBNER, Jinmiao HUANG, Prashant RAINA, Eu Wern TEH, Mathieu TULI
  • Publication number: 20250022200
    Abstract: A method for controlling an artificial intelligence (AI) device for implementing a digital avatar can include receiving an audio signal corresponding to a user query, converting, by a speech-to-text neural network model, the audio signal into a text query, inputting the text query into a large language gesture instruction model to generate high level movement instructions, inputting the text query and the high level movement instructions into an information retrieval model to generate a text response including at least one sentence and digital avatar control information, and inputting the text response into a text-to-speech neural network model to generate an audio response.
    Type: Application
    Filed: July 15, 2024
    Publication date: January 16, 2025
    Applicant: LG ELECTRONICS INC.
    Inventors: Jinmiao HUANG, Prashant RAINA, Felix TAUBNER, Mathieu TULI, Eu Wern TEH, Kevin FERREIRA, Xiaoyu YANG
  • Publication number: 20230335118
    Abstract: A computer-implemented method includes receiving enrollment audio from a user comprising a wake word to be enrolled for the device, preprocessing the enrollment audio to obtain a vector representation along at least a feature dimension and a temporal dimension, inputting the extracted vector representation to a trained encoding model to generate an embedding representation of the enrollment audio, wherein the encoding model includes a plurality of mixing blocks, and wherein the feature dimension and the temporal dimension of an output of a first layer of each mixing block are flipped for inputting to a second layer of the mixing block, and storing the generated embedding representation in a memory for use in detecting input of the enrolled wake word.
    Type: Application
    Filed: April 13, 2023
    Publication date: October 19, 2023
    Applicant: LG ELECTRONICS INC.
    Inventors: Jinmiao HUANG, Waseem GHARBIEH, Qianhui WAN
  • Patent number: 11599103
    Abstract: A system for data driven diagnostics of a machine including a machine learning model instantiated in a computer, the machine learning model being configured to: receive operational data of the machine; and process the operational data to determine machine diagnostics information. The machine learning model is trained using simulated defect information received from a simulation environment.
    Type: Grant
    Filed: February 21, 2019
    Date of Patent: March 7, 2023
    Assignee: Dodge Industrial, Inc.
    Inventors: Stefan Rakuff, Jinmiao Huang
  • Patent number: 11440183
    Abstract: For training an object picking robot with real and simulated grasp performance data, grasp locations on an object are assigned based on object physical properties. A simulation experiment for robot grasping is performed using a first set of assigned locations. Based on simulation data from the simulation, a simulated object grasp quality of the robot is evaluated for each of the assigned locations. A first set of candidate grasp locations on the object is determined based on data representative of simulated grasp quality from the evaluation. Based on sensor data from an actual experiment for the robot grasping using each of the candidate grasp locations, an actual object grasp quality is evaluated for each of the candidate locations.
    Type: Grant
    Filed: March 27, 2019
    Date of Patent: September 13, 2022
    Assignee: ABB Schweiz AG
    Inventors: Jinmiao Huang, Carlos Martinez, Sangeun Choi, Thomas A. Fuhlbrigge
  • Patent number: 11407111
    Abstract: A robot is configured to perform a task on an object using a method for generating a 3D model sufficient to determine a collision free path and identify the object in an industrial scene. The method includes determining a predefined collision free path and scanning an industrial scene around the robot. Stored images of the industrial scene are retrieved from a memory and analyzed to construct a new 3D model. After an object is detected in the new 3D model, the robot can further scan the image in the industrial scene while moving along a collision free path until the object is identified at a predefined certainty level. The robot can then perform a robot task on the object.
    Type: Grant
    Filed: June 27, 2018
    Date of Patent: August 9, 2022
    Assignee: ABB Schweiz AG
    Inventors: Biao Zhang, Remus Boca, Carlos W. Morato, Carlos Martinez, Jianjun Wang, Zhou Teng, Jinmiao Huang, Magnus Wahlstrom, Johnny Holmberg
  • Patent number: 11312581
    Abstract: A grasping system includes a robotic arm having a gripper. A fixed sensor monitors a grasp area and an onboard sensor moves with the gripper also monitors the area. A controller receives information indicative of a position of an object to be grasped and operates the robotic arm to bring the gripper into a grasp position adjacent the object based on information provided by the fixed sensor. The controller is also programmed to operate the gripper to grasp the object in response to information provided by the first onboard sensor.
    Type: Grant
    Filed: April 16, 2019
    Date of Patent: April 26, 2022
    Assignee: ABB Schweiz AG
    Inventors: Jinmiao Huang, Sangeun Choi, Carlos Martinez
  • Patent number: 11227078
    Abstract: A method for automated gearbox design includes: instantiating the gearbox model having an initial parameter state in a modeling environment; analyzing and/or characterizing the gearbox model in the modeling environment to determine gearbox model performance; and determining whether the gearbox model performance satisfies a performance target. Upon a determination that the gearbox model performance does not satisfy the performance target: a reward is calculated based on the gearbox model performance; a reinforcement machine learning agent determines a parameter change action based on the reward and a current parameter state of the gearbox model; and an updated parameter state of the gearbox model is determined based on the parameter change action.
    Type: Grant
    Filed: February 21, 2019
    Date of Patent: January 18, 2022
    Assignee: Dodge Acquisition Co.
    Inventors: Stefan Rakuff, Jinmiao Huang
  • Patent number: 11185980
    Abstract: Methods, systems, and software for controlling object picking and placement by a robot system are disclosed. The method includes assigning machine learning training data of a machine learning model for an object. The machine learning training data includes a plurality of known grasp location labels assigned to the object positioned in a plurality of different object poses. The method includes providing the object in a work space of the robot system. For the object in the work space in a first pose of the plurality of different object poses, the method includes: mapping a first candidate grasp location on the object; executing robotic movements for the first candidate grasp location on the object; and evaluating a result of the executing for the first candidate grasp location according to at least one predetermined performance criteria.
    Type: Grant
    Filed: April 16, 2019
    Date of Patent: November 30, 2021
    Assignee: ABB Schweiz AG
    Inventors: Jinmiao Huang, Carlos Martinez, Sangeun Choi
  • Publication number: 20200331709
    Abstract: A grasping system includes a robotic arm having a gripper. A fixed sensor monitors a grasp area and an onboard sensor moves with the gripper also monitors the area. A controller receives information indicative of a position of an object to be grasped and operates the robotic arm to bring the gripper into a grasp position adjacent the object based on information provided by the fixed sensor. The controller is also programmed to operate the gripper to grasp the object in response to information provided by the first onboard sensor.
    Type: Application
    Filed: April 16, 2019
    Publication date: October 22, 2020
    Applicant: ABB Schweiz AG
    Inventors: Jinmiao Huang, Sangeun Choi, Carlos Martinez
  • Publication number: 20200331144
    Abstract: Methods, systems, and software for controlling object picking and placement by a robot system are disclosed. The method includes assigning machine learning training data of a machine learning model for an object. The machine learning training data includes a plurality of known grasp location labels assigned to the object positioned in a plurality of different object poses. The method includes providing the object in a work space of the robot system. For the object in the work space in a first pose of the plurality of different object poses, the method includes: mapping a first candidate grasp location on the object; executing robotic movements for the first candidate grasp location on the object; and evaluating a result of the executing for the first candidate grasp location according to at least one predetermined performance criteria.
    Type: Application
    Filed: April 16, 2019
    Publication date: October 22, 2020
    Applicant: ABB Schweiz AG
    Inventors: Jinmiao Huang, Carlos Martinez, Sangeun Choi
  • Publication number: 20200306959
    Abstract: For training an object picking robot with real and simulated grasp performance data, grasp locations on an object are assigned based on object physical properties. A simulation experiment for robot grasping is performed using a first set of assigned locations. Based on simulation data from the simulation, a simulated object grasp quality of the robot is evaluated for each of the assigned locations. A first set of candidate grasp locations on the object is determined based on data representative of simulated grasp quality from the evaluation. Based on sensor data from an actual experiment for the robot grasping using each of the candidate grasp locations, an actual object grasp quality is evaluated for each of the candidate locations.
    Type: Application
    Filed: March 27, 2019
    Publication date: October 1, 2020
    Applicant: ABB Schweiz AG
    Inventors: Jinmiao Huang, Carlos Martinez, Sangeun Choi, Thomas A. Fuhlbrigge
  • Publication number: 20200272705
    Abstract: A method for automated gearbox design includes: instantiating the gearbox model having an initial parameter state in a modeling environment; analyzing and/or characterizing the gearbox model in the modeling environment to determine gearbox model performance; and determining whether the gearbox model performance satisfies a performance target. Upon a determination that the gearbox model performance does not satisfy the performance target: a reward is calculated based on the gearbox model performance; a reinforcement machine learning agent determines a parameter change action based on the reward and a current parameter state of the gearbox model; and an updated parameter state of the gearbox model is determined based on the parameter change action.
    Type: Application
    Filed: February 21, 2019
    Publication date: August 27, 2020
    Applicant: ABB Schweiz AG
    Inventors: Stefan Rakuff, Jinmiao Huang
  • Publication number: 20200272139
    Abstract: A system for data driven diagnostics of a machine including a machine learning model instantiated in a computer, the machine learning model being configured to: receive operational data of the machine; and process the operational data to determine machine diagnostics information. The machine learning model is trained using simulated defect information received from a simulation environment.
    Type: Application
    Filed: February 21, 2019
    Publication date: August 27, 2020
    Applicant: ABB Schweiz AG
    Inventors: Stefan Rakuff, Jinmiao Huang
  • Publication number: 20200001458
    Abstract: A robot is configured to perform a task on an object using a method for generating a 3D model sufficient to determine a collision free path and identify the object in an industrial scene. The method includes determining a predefined collision free path and scanning an industrial scene around the robot. Stored images of the industrial scene are retrieved from a memory and analyzed to construct a new 3D model. After an object is detected in the new 3D model, the robot can further scan the image in the industrial scene while moving along a collision free path until the object is identified at a predefined certainty level. The robot can then perform a robot task on the object.
    Type: Application
    Filed: June 27, 2018
    Publication date: January 2, 2020
    Inventors: Biao Zhang, Remus Boca, Carlos W. Morato, Carlos Martinez, Jianjun Wang, Zhou Teng, Jinmiao Huang, Magnus Wahlstrom, Johnny Holmberg