Abstract: Approaches presented herein provide for automatic generation of representative two-dimensional (2D) images for three-dimensional (3D) objects or assets. In generating these 2D images, a set of options is determined such as may relate to viewpoint or other parameters of a virtual camera. A set of sample points is determined from which to generate 2D images of a 3D model, for example, with 2D images being processed using a classifier to determine which of these images generates a classification with highest confidence or probability, individually or with respect to other classifications. The sample point for this selected image can then be used to select nearby sample points as part of a refinement or optimization process, where 2D images can again be generated and processed using a classifier to identify a 2D image with highest classification probability or confidence, which can be selected as representative of the 3D object or asset.

Abstract: Approaches presented herein can provide for the automatic generation of a digital representation of an environment that may include multiple objects of various object types. An initial representation (e.g., a point cloud) of the environment can be generated from registered image or scan data, for example, and objects in the environment can be segmented and identified based at least on that initial representation. For objects that are recognized based on these segmentations, stored accurate representations can be substituted for those objects in the representation of the environment, and if no such model is available then a mesh or other representation of that object can be generated and positioned in the environment. A result can then include a 3D representation of a scene or environment in which objects are identified and segmented as individual objects, and representations of the scene or environment can be viewed, and interacted with, through various viewports, positions, and perspectives.

Abstract: Animation can be generated with a high perceptive quality by utilizing a trained neural network that takes as input a current state of a virtual character to be animated and predict how this character would appear in one or more subsequent frames. Such a process can be performed recursively to generate the data for these frames. During training, each frame of a generated sequence can be predicted from a result for a previous frame, and this generated sequence can be compared with a ground truth sequence using a generative network. Differences between the ground truth and generated animation sequences can be minimized, whereby a specific objective function does not need to be manually defined. Minimizing differences between the generated animation sequences and ground truth sequences during training improves the quality of network predictions for single frames at inference time.

Abstract: A method for predicting three-dimensional body poses from image sequences of an object, the method performed on a processor of a computer having memory, the method including the steps of accessing the image sequences from the memory, finding bounding boxes around the object in consecutive frames of the image sequence, compensating motion of the object to form spatio-temporal volumes, and learning a mapping from the spatio-temporal volumes to a three-dimensional body pose in a central frame based on a mapping function.