Patents by Inventor Jinmiao Huang
Jinmiao Huang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20250173937Abstract: A method for speech-driven three dimensional (3D) facial animation can include receiving an input speech audio signal, generating a speaker style vector from the input speech audio signal based on a speaker style embedding model, inputting the input speech audio signal and the speaker style vector into a mesh generation model and generating vertex position information for a 3D facial animation based on the input speech audio signal and the speaker style vector, and outputting the vertex position information.Type: ApplicationFiled: November 25, 2024Publication date: May 29, 2025Applicant: LG ELECTRONICS INC.Inventors: Mathieu TULI, Eu Wern TEH, Felix TAUBNER, Jinmiao HUANG, Prashant RAINA
-
Publication number: 20250173967Abstract: A method for controlling an artificial intelligence (AI) device can include receiving, by a processor, an input two dimensional (2D) image, encoding the 2D image to generate an image feature map, obtaining UV positional encoding information based on a three-dimensional (3D) face model where the UV positional encoding information includes a UV feature map, generating, by the processor, a correlation volume based on the image feature map and the UV feature map, generating flow map information and uncertainty information based on the correlation volume and the UV positional encoding information, and generating probabilistic 2D alignment information based on the flow map information and uncertainty information and outputting the probabilistic 2D alignment information. Also, the method can further include generating 3D reconstruction information based on the probabilistic 2D alignment information for animating a 3D face based on the input 2D image.Type: ApplicationFiled: November 25, 2024Publication date: May 29, 2025Applicant: LG ELECTRONICS INC.Inventors: Felix TAUBNER, Prashant RAINA, Eu Wern TEH, Mathieu TULI, Jinmiao HUANG
-
Publication number: 20250173936Abstract: A method for neural network driven vertex animation can include receiving, by an encoder component of a trained neural network, an input driving signal including audio data, processing, by the encoder component, the input driving signal to generate blendshape coefficient information based on the input driving signal, transmitting, by the encoder component, the blendshape coefficient information to a decoder component of the trained neural network, receiving, by the decoder component, the blendshape coefficient information from the encoder component, and generating, by the decoder component, vertex position information based on the blendshape coefficient information for animating a 3D model.Type: ApplicationFiled: November 25, 2024Publication date: May 29, 2025Applicant: LG ELECTRONICS INC.Inventors: Felix TAUBNER, Jinmiao HUANG, Prashant RAINA, Eu Wern TEH, Mathieu TULI
-
Publication number: 20250022200Abstract: A method for controlling an artificial intelligence (AI) device for implementing a digital avatar can include receiving an audio signal corresponding to a user query, converting, by a speech-to-text neural network model, the audio signal into a text query, inputting the text query into a large language gesture instruction model to generate high level movement instructions, inputting the text query and the high level movement instructions into an information retrieval model to generate a text response including at least one sentence and digital avatar control information, and inputting the text response into a text-to-speech neural network model to generate an audio response.Type: ApplicationFiled: July 15, 2024Publication date: January 16, 2025Applicant: LG ELECTRONICS INC.Inventors: Jinmiao HUANG, Prashant RAINA, Felix TAUBNER, Mathieu TULI, Eu Wern TEH, Kevin FERREIRA, Xiaoyu YANG
-
Publication number: 20230335118Abstract: A computer-implemented method includes receiving enrollment audio from a user comprising a wake word to be enrolled for the device, preprocessing the enrollment audio to obtain a vector representation along at least a feature dimension and a temporal dimension, inputting the extracted vector representation to a trained encoding model to generate an embedding representation of the enrollment audio, wherein the encoding model includes a plurality of mixing blocks, and wherein the feature dimension and the temporal dimension of an output of a first layer of each mixing block are flipped for inputting to a second layer of the mixing block, and storing the generated embedding representation in a memory for use in detecting input of the enrolled wake word.Type: ApplicationFiled: April 13, 2023Publication date: October 19, 2023Applicant: LG ELECTRONICS INC.Inventors: Jinmiao HUANG, Waseem GHARBIEH, Qianhui WAN
-
Patent number: 11599103Abstract: A system for data driven diagnostics of a machine including a machine learning model instantiated in a computer, the machine learning model being configured to: receive operational data of the machine; and process the operational data to determine machine diagnostics information. The machine learning model is trained using simulated defect information received from a simulation environment.Type: GrantFiled: February 21, 2019Date of Patent: March 7, 2023Assignee: Dodge Industrial, Inc.Inventors: Stefan Rakuff, Jinmiao Huang
-
Patent number: 11440183Abstract: For training an object picking robot with real and simulated grasp performance data, grasp locations on an object are assigned based on object physical properties. A simulation experiment for robot grasping is performed using a first set of assigned locations. Based on simulation data from the simulation, a simulated object grasp quality of the robot is evaluated for each of the assigned locations. A first set of candidate grasp locations on the object is determined based on data representative of simulated grasp quality from the evaluation. Based on sensor data from an actual experiment for the robot grasping using each of the candidate grasp locations, an actual object grasp quality is evaluated for each of the candidate locations.Type: GrantFiled: March 27, 2019Date of Patent: September 13, 2022Assignee: ABB Schweiz AGInventors: Jinmiao Huang, Carlos Martinez, Sangeun Choi, Thomas A. Fuhlbrigge
-
Patent number: 11407111Abstract: A robot is configured to perform a task on an object using a method for generating a 3D model sufficient to determine a collision free path and identify the object in an industrial scene. The method includes determining a predefined collision free path and scanning an industrial scene around the robot. Stored images of the industrial scene are retrieved from a memory and analyzed to construct a new 3D model. After an object is detected in the new 3D model, the robot can further scan the image in the industrial scene while moving along a collision free path until the object is identified at a predefined certainty level. The robot can then perform a robot task on the object.Type: GrantFiled: June 27, 2018Date of Patent: August 9, 2022Assignee: ABB Schweiz AGInventors: Biao Zhang, Remus Boca, Carlos W. Morato, Carlos Martinez, Jianjun Wang, Zhou Teng, Jinmiao Huang, Magnus Wahlstrom, Johnny Holmberg
-
Patent number: 11312581Abstract: A grasping system includes a robotic arm having a gripper. A fixed sensor monitors a grasp area and an onboard sensor moves with the gripper also monitors the area. A controller receives information indicative of a position of an object to be grasped and operates the robotic arm to bring the gripper into a grasp position adjacent the object based on information provided by the fixed sensor. The controller is also programmed to operate the gripper to grasp the object in response to information provided by the first onboard sensor.Type: GrantFiled: April 16, 2019Date of Patent: April 26, 2022Assignee: ABB Schweiz AGInventors: Jinmiao Huang, Sangeun Choi, Carlos Martinez
-
Patent number: 11227078Abstract: A method for automated gearbox design includes: instantiating the gearbox model having an initial parameter state in a modeling environment; analyzing and/or characterizing the gearbox model in the modeling environment to determine gearbox model performance; and determining whether the gearbox model performance satisfies a performance target. Upon a determination that the gearbox model performance does not satisfy the performance target: a reward is calculated based on the gearbox model performance; a reinforcement machine learning agent determines a parameter change action based on the reward and a current parameter state of the gearbox model; and an updated parameter state of the gearbox model is determined based on the parameter change action.Type: GrantFiled: February 21, 2019Date of Patent: January 18, 2022Assignee: Dodge Acquisition Co.Inventors: Stefan Rakuff, Jinmiao Huang
-
Patent number: 11185980Abstract: Methods, systems, and software for controlling object picking and placement by a robot system are disclosed. The method includes assigning machine learning training data of a machine learning model for an object. The machine learning training data includes a plurality of known grasp location labels assigned to the object positioned in a plurality of different object poses. The method includes providing the object in a work space of the robot system. For the object in the work space in a first pose of the plurality of different object poses, the method includes: mapping a first candidate grasp location on the object; executing robotic movements for the first candidate grasp location on the object; and evaluating a result of the executing for the first candidate grasp location according to at least one predetermined performance criteria.Type: GrantFiled: April 16, 2019Date of Patent: November 30, 2021Assignee: ABB Schweiz AGInventors: Jinmiao Huang, Carlos Martinez, Sangeun Choi
-
Publication number: 20200331709Abstract: A grasping system includes a robotic arm having a gripper. A fixed sensor monitors a grasp area and an onboard sensor moves with the gripper also monitors the area. A controller receives information indicative of a position of an object to be grasped and operates the robotic arm to bring the gripper into a grasp position adjacent the object based on information provided by the fixed sensor. The controller is also programmed to operate the gripper to grasp the object in response to information provided by the first onboard sensor.Type: ApplicationFiled: April 16, 2019Publication date: October 22, 2020Applicant: ABB Schweiz AGInventors: Jinmiao Huang, Sangeun Choi, Carlos Martinez
-
Publication number: 20200331144Abstract: Methods, systems, and software for controlling object picking and placement by a robot system are disclosed. The method includes assigning machine learning training data of a machine learning model for an object. The machine learning training data includes a plurality of known grasp location labels assigned to the object positioned in a plurality of different object poses. The method includes providing the object in a work space of the robot system. For the object in the work space in a first pose of the plurality of different object poses, the method includes: mapping a first candidate grasp location on the object; executing robotic movements for the first candidate grasp location on the object; and evaluating a result of the executing for the first candidate grasp location according to at least one predetermined performance criteria.Type: ApplicationFiled: April 16, 2019Publication date: October 22, 2020Applicant: ABB Schweiz AGInventors: Jinmiao Huang, Carlos Martinez, Sangeun Choi
-
Publication number: 20200306959Abstract: For training an object picking robot with real and simulated grasp performance data, grasp locations on an object are assigned based on object physical properties. A simulation experiment for robot grasping is performed using a first set of assigned locations. Based on simulation data from the simulation, a simulated object grasp quality of the robot is evaluated for each of the assigned locations. A first set of candidate grasp locations on the object is determined based on data representative of simulated grasp quality from the evaluation. Based on sensor data from an actual experiment for the robot grasping using each of the candidate grasp locations, an actual object grasp quality is evaluated for each of the candidate locations.Type: ApplicationFiled: March 27, 2019Publication date: October 1, 2020Applicant: ABB Schweiz AGInventors: Jinmiao Huang, Carlos Martinez, Sangeun Choi, Thomas A. Fuhlbrigge
-
Publication number: 20200272705Abstract: A method for automated gearbox design includes: instantiating the gearbox model having an initial parameter state in a modeling environment; analyzing and/or characterizing the gearbox model in the modeling environment to determine gearbox model performance; and determining whether the gearbox model performance satisfies a performance target. Upon a determination that the gearbox model performance does not satisfy the performance target: a reward is calculated based on the gearbox model performance; a reinforcement machine learning agent determines a parameter change action based on the reward and a current parameter state of the gearbox model; and an updated parameter state of the gearbox model is determined based on the parameter change action.Type: ApplicationFiled: February 21, 2019Publication date: August 27, 2020Applicant: ABB Schweiz AGInventors: Stefan Rakuff, Jinmiao Huang
-
Publication number: 20200272139Abstract: A system for data driven diagnostics of a machine including a machine learning model instantiated in a computer, the machine learning model being configured to: receive operational data of the machine; and process the operational data to determine machine diagnostics information. The machine learning model is trained using simulated defect information received from a simulation environment.Type: ApplicationFiled: February 21, 2019Publication date: August 27, 2020Applicant: ABB Schweiz AGInventors: Stefan Rakuff, Jinmiao Huang
-
Publication number: 20200001458Abstract: A robot is configured to perform a task on an object using a method for generating a 3D model sufficient to determine a collision free path and identify the object in an industrial scene. The method includes determining a predefined collision free path and scanning an industrial scene around the robot. Stored images of the industrial scene are retrieved from a memory and analyzed to construct a new 3D model. After an object is detected in the new 3D model, the robot can further scan the image in the industrial scene while moving along a collision free path until the object is identified at a predefined certainty level. The robot can then perform a robot task on the object.Type: ApplicationFiled: June 27, 2018Publication date: January 2, 2020Inventors: Biao Zhang, Remus Boca, Carlos W. Morato, Carlos Martinez, Jianjun Wang, Zhou Teng, Jinmiao Huang, Magnus Wahlstrom, Johnny Holmberg