Abstract: Systems and methods are provided for machine learning-based rendering of a clothed human with a realistic 3D appearance by virtually draping one or more garments or items of clothing on a 3D human body model. The machine learning model may be trained to drape a garment on a 3D body mesh using training data that includes a variety 3D body meshes reflecting a variety of different body types. The machine learning model may include an encoder trained to extract body features from an input 3D mesh, and a decoder network trained to drape the garment on the input 3D mesh based at least in part on spectral decomposition of a mesh associated with the garment. The trained machine learning model may then be used to drape the garment or a variation of the garment on a new input body mesh.

Abstract: Disclosed are systems and techniques for extracting user body shape (e.g., a representation of the three-dimensional body surface) from user behavioral data. The behavioral data may not be explicitly body-shape-related, and can include shopping history, social media likes, or other recorded behaviors of the user within (or outside of) a networked content delivery environment. The determined body shape can be used, for example, to generate a virtual fitting room user interface.

Abstract: Systems and methods are provided for generating an image of a posed human figure or other subject using a neural network that is trained to translate a set of points to realistic images by reconstructing projected surfaces directly in the pixel space or image space. Input to the image generation process may include parameterized control features, such as body shape parameters, pose parameters and/or a virtual camera position. These input parameters may be applied to a three-dimensional model that is used to generate the set of points, such as a sparsely populated image of color and depth information at vertices of the three-dimensional model, before additional image generation occurs directly in the image space. The visual appearance or identity of the synthesized human in successive output images may remain consistent, such that the output is both controllable and predictable.

Abstract: A system configured to perform an accurate and fast estimation of an object shape from a single input image. The system may process image data representing a first surface of an object using image-to-image translation techniques. A first trained model may generate depth information for the object, such as front distance estimates and back distance estimates. The system may use the depth information to generate an output mesh shaped like the object, such as, in the case of a pliable object a reposable avatar. The system may improve depth estimation by including a loss on surface normals in the first trained model. A second trained model may generate color information to be applied to the output mesh to accurately represent the object. The output mesh may include detailed geometry and appearance of the object, useful for a variety of purposes such as gaming, virtual/augmented reality, virtual shopping, and other implementations.

Abstract: A system configured to process an input point cloud, which represents an object using unstructured data points, to generate a feature vector that has an ordered structure and a fixed length. The system may process the input point cloud using a basis point set to generate the feature vector. For example, for each basis point in the basis point set, the system may identify a closest data point in the point cloud data and store a distance value or other information associated with the closest data point in the feature vector. The system may process the feature vector using a trained model to generate output data, such as performing point cloud registration to generate mesh data, point cloud classification to generate classification data, and/or the like.