Abstract: Methods are provided for generating training data in a form of a plurality of frames of facial animation, each of the plurality of frames represented as a three-dimensional (3D) mesh comprising a plurality of vertices. The training data is usable to train an actor-specific actor-to-mesh conversion model which, when trained, receives a performance of the actor captured by a head-mounted camera (HMC) set-up and infers a corresponding actor-specific 3D mesh of the performance of the actor. The methods may involve performing a blendshape optimization to obtain a blendshape-optimized 3D mesh and performing a mesh-deformation refinement on the blendshape-optimized 3D mesh to obtain a mesh-deformation-optimized 3D mesh. The training data may be generated on the basis of the mesh-deformation-optimized 3D mesh.

Abstract: Method for generating frames of facial animation comprises: obtaining a plurality of frames of training data comprising, for each frame: a training representation comprising geometric information for a plurality of primary face vertices and a plurality of secondary facial component vertices. The facial animation training data comprises indices of a subset of the plurality of primary face vertices. The method comprises: training a secondary facial component model using the facial animation training data; obtaining frames of primary face animation, each frame comprising geometric information for a plurality of primary face vertices; and, for each frame of primary face animation, generating a corresponding frame of secondary facial component animation based on the frame of primary face animation and the secondary facial component model, the corresponding frame of the secondary facial component animation comprising geometric information for secondary facial component vertices based on the primary face geometry.

Abstract: A method of preparing a training data set for an artificial intelligence engine operable to convert markerless digital images of a target face into a mapping. The method comprises training an intermediate artificial intelligence by providing markered digital images of one or more skin parts to the intermediate artificial intelligence engine as a source training data set and providing markerless digital images of the one or more skin parts to the intermediate artificial intelligence engine as a target training data set. The intermediate artificial intelligence engine removes markers from markered digital images of the face of a first person to create corresponding synthetic markerless digital images of the face of the first person. The markered digital images of the face of the first person are converted into a mapping. The output training data set comprises the synthetic markerless digital images of the face of the first person and the mapping.