Abstract: In an embodiment, a method for human motion generation with open vocabulary scene-and-text context is provided. The method involves receiving an input that includes a 3D point cloud of a scene containing a goal object with a natural language instruction related to the goal object. A text tokenizer is applied to the text to obtain tokenized text, and a text encoder from a pre-trained vision-language model generates text features. First scene features are generated by applying a pre-trained U-Net scene encoder to the 3D point cloud, which are down sampled to obtain second scene features. A conditional latent is obtained by fusing the second scene features with the text features. A conditional motion generator predicts motion parameters for a parametric human body model over a specific time duration. Finally, 3D human meshes for multiple motion frames are obtained based on the motion parameters and the parametric human body model.

Abstract: In an example, a method may include obtaining, from a data source, first data including multiple frames each including a human face. The method may include automatically detecting, in each of the multiple frames, one or more facial landmarks and one or more action units (AUs) associated with the human face. The method may also include automatically generating one or more semantic masks based at least on the one or more facial landmarks, the one or more semantic masks individually corresponding to the human face. The method may further include obtaining a facial hyperspace using at least the first data, the one or more AUs, and the semantic masks. The method may also include generating a synthetic image of the human face using a first frame of the multiple frames and one or more AU intensities individually associated with the one or more AUs.

Abstract: In an example, a method may include deforming a first ray associated with a dynamic object at a first time using a first neural network and a latent code to obtain a deformed ray. The method may also include obtaining a hyperspace code associated with the first ray by inputting the first ray, the first time, and the latent code into a second neural network. The method may further include sampling one or more points from the deformed ray. The method may also include combining the sampled points and the hyperspace code into a network input. The method may further include inputting the network input into a third neural network to obtain RGB values for rendering images of a three-dimensional scene representative of the dynamic object at a second time.

Abstract: In an example, a method may include obtaining, from a data source, first data including multiple frames each including a human face. The method may include automatically detecting, in each of the multiple frames, one or more facial landmarks and one or more action units (AUs) associated with the human face. The method may also include automatically generating one or more semantic masks based at least on the one or more facial landmarks, the one or more semantic masks individually corresponding to the human face. The method may further include obtaining a facial hyperspace using at least the first data, the one or more AUs, and the semantic masks. The method may also include generating a synthetic image of the human face using a first frame of the multiple frames and one or more AU intensities individually associated with the one or more AUs.