Abstract: In various examples there is an apparatus for computing an image depicting a face of a wearer of a head mounted display (HMD), as if the wearer was not wearing the HMD. An input image depicts a partial view of the wearer's face captured from at least one face facing capture device in the HMD. A machine learning apparatus is available which has been trained to compute expression parameters from the input image. A 3D face model that has expressions parameters is accessible as well as a photorealiser being a machine learning model trained to map images rendered from the 3D face model to photorealistic images. The apparatus computes expression parameter values from the image using the machine learning apparatus. The apparatus drives the 3D face model with the expression parameter values to produce a 3D model of the face of the wearer and then renders the 3D model from a specified viewpoint to compute a rendered image. The rendered image is upgraded to a photorealistic image using the photorealiser.

Abstract: There is a method of computing a stylized, animatable representation of a subject from a family of stylized animatable representations. The method comprises: accessing a realistic representation of the subject; computing a mesh mapping using a model that was formed using a data set of training examples which pair realistic representations of other subjects with instances of the family. The method also comprises applying the mesh mapping to the realistic representation of the subject to produce a target mesh; and selecting the stylized animatable representation from the family, by assessing closeness of the target mesh with instances of the family.

Abstract: Keypoints are predicted in an image. A neural network is executed that is configured to predict each of the keypoints as a 2D random variable, normally distributed with a 2D position and 2×2 covariance matrix. The neural network is trained to maximize a log-likelihood that samples from each of the predicted keypoints equal a ground truth. The trained neural network is used to predict keypoints of an image without generating a heatmap.

Abstract: Keypoints are predicted in an image. Predictions are generated for each of the keypoints of an image as a 2D random variable, normally distributed with location (x, y) and standard deviation sigma. A neural network is trained to maximize a log-likelihood that samples from each of the predicted keypoints equal a ground truth. The trained neural network is used to predict keypoints of an image without generating a heatmap.

Abstract: In various examples there is an apparatus for computing an image depicting a face of a wearer of a head mounted display (HMD), as if the wearer was not wearing the HMD. An input image depicts a partial view of the wearer's face captured from at least one face facing capture device in the HMD. A machine learning apparatus is available which has been trained to compute expression parameters from the input image. A 3D face model that has expressions parameters is accessible as well as a photorealiser being a machine learning model trained to map images rendered from the 3D face model to photorealistic images. The apparatus computes expression parameter values from the image using the machine learning apparatus. The apparatus drives the 3D face model with the expression parameter values to produce a 3D model of the face of the wearer and then renders the 3D model from a specified viewpoint to compute a rendered image. The rendered image is upgraded to a photorealistic image using the photorealiser.

Abstract: A computer system is provided that includes an input system and a processor that may be configured to control a virtual manipulator based on input data received from the input system. The processor is further configured to determine an initial state of a system that includes at least an initial state of the virtual manipulator and an initial state of a virtual object, detect at least one contact point between a portion of the virtual manipulator and the virtual object. The processor is further configured to calculate a subsequent state for the virtual object that minimizes a set of energies or residuals defined in terms of the one or more positional quantities determined for the initial state of the system and the one or more positional quantities determined for the subsequent state of the system using a position-based energy minimizing function.

Abstract: A computer system is provided that includes an input system and a processor that may be configured to control a virtual manipulator based on input data received from the input system. The processor is further configured to determine an initial state of a system that includes at least an initial state of the virtual manipulator and an initial state of a virtual object, detect at least one contact point between a portion of the virtual manipulator and the virtual object. The processor is further configured to calculate a subsequent state for the virtual object that minimizes a set of energies or residuals defined in terms of the one or more positional quantities determined for the initial state of the system and the one or more positional quantities determined for the subsequent state of the system using a position-based energy minimizing function.