Abstract: A system may perform motion capture using motion capture targets with concave reflector structures. For example, the motion capture target include a target body and a plurality of tracking markers located on respective portions of the surface of the target body. At least one tracking marker of the plurality of tracking markers may be a concave reflector structure including a tapered hole in the surface of the target body and at least a portion of a surface of the tapered hole may be reflective.

Abstract: A method of generating a three-dimensional (3D) model includes obtaining a set of two-dimensional (2D) images of a scene acquired by one or more cameras from a plurality of camera angles at a plurality of camera positions. Each 2D image corresponds to a respective camera angle and a respective camera position. The method further includes obtaining the respective camera angle and the respective camera position for each 2D image, and generating one or more semantic masks from the set of 2D images. Each semantic mask corresponds to a class of one or more objects in the scene. The method further includes training a neural radiance field (NeRF) model, using the set of 2D images and the one or more semantic masks as a training dataset, to obtain a trained NeRF model. The trained NeRF model is an implicit 3D model of the one or more objects in the scene.

Abstract: The present disclosure provides a system for customizing virtual entities via a multi-track system. The system can generate a user interface that uses multiple tracks to manage and display virtual entities and virtual objects. The virtual entity can be in-line with a track, and virtual display objects can move along tracks that intersect with the virtual entity track. When a particular virtual display object intersects with the virtual entity track, a three-dimensional representation of an item associated with the particular virtual display object can be rendered with the virtual entity.

Abstract: A method, device, and computer-readable storage medium for simplifying a mesh including bevels. The method includes: receiving a polygonal mesh representing a three-dimensional (3D) object; identifying a set of edges in the polygonal mesh as bevel edges; performing a mesh simplification operation on the polygonal mesh to generate a simplified mesh, wherein the mesh simplification operation removes at least one edge that includes a vertex that is of a bevel edge, and wherein two vertices in the polygonal mesh are collapsed to a collapse vertex in the simplified mesh; and updating stored normals of the collapse vertex based on copying stored normals of the two vertices removed from the polygonal mesh to the collapse vertex.

Abstract: Systems and methods for user interface navigation may include a computing system which causes display of a user interface including a set of first user interface elements. The computing system receives a request to expand a switcher menu including a set of second user interface elements including at least one of the first user interface elements. Each second user interface element, when selected within the switcher menu, is displayed with one or more respective third user interface elements. The computing system causes display of a set of third user interface elements corresponding to one of the second user interface elements, and causes, responsive to a second request to navigate from the switcher menu to one of the set of third user interface elements, the switcher menu to collapse and display the second user interface element with at least some of the third user interface elements.

Abstract: System and methods for using a deep learning framework to customize animation of an in-game character of a video game. The system can be preconfigured with animation rule sets corresponding to various animations. Each animation can be comprised of a series of distinct poses that collectively form the particular animation. The system can provide an animation-editing interface that enables a user of the video game to make modifications to at least one pose or frame of the animation. The system can realistically extrapolate these modifications across some or all portions of the animation. In addition or alternatively, the system can realistically extrapolate the modifications across other types of animations.

Abstract: A first set of contact points for a hull object colliding with a first primitive of a mesh object is determined. An edge clip is generated for an edge of the first primitive based on determining that the hull object generates contact points with the edge. A face walk is performed into a second primitive of the mesh object across the edge for the edge clip. A set of initial contact points for the hull object colliding with the second primitive is generated using the algorithm. A second set of contact points is generated based on projecting each initial contact point in a direction of a contact normal for the first set of contact points onto a plane created by extending a feature of the first primitive. Contact points in the first and second set of contact points are output, but not the contacts along the edge clip.

Abstract: A method, device, and computer-readable storage medium for generating levels-of-detail (LODs) for blendshape assets is disclosed. Given a base target and a set of blendshape targets, a LOD for the base target is generated based on performing edge collapse operations on the base target, where the edge collapse operations that are performed are informed by not only the geometry of the base target, but also the geometries of the blendshape targets. In some cases, the same edge collapse operations can also be performed on the blendshape targets to generate LODs for the blendshape targets, thereby maintaining topological consistency between the LODs for the base target and the blendshape targets. In some cases, a morphing operator can take as input a LOD of the base target and generate a LOD for a blendshape target.

Abstract: Unresolved primitives of mesh objects are determined. A primary feature of a primitive of the mesh object is determined to be a vertex or edge. The primary feature is marked as a first unresolved primitive. All unresolved primitives or the mesh object colliding with a convex hull object are collected into a sorted list. The first unresolved primitive is determined to be a first link in a chain of unresolved primitives. An edge of the first unresolved primitive is determined to be marked as being walked through to continue to build the chain of unresolved primitives based on the primary feature and features connected to the primary feature. An index of the first unresolved primitive and an index of the edge marked as being walked through are added to the chain. A determination is made to continue to build the chain based on the extracted features of the neighboring primitive.

Abstract: This specification provides a computer-implemented method for generating facial animation data. The facial animation data animates a face in a video game in accordance with speech sounds and speech emotion of speech audio. Speech audio data representing the speech audio is processed using a machine-learned speech emotion recognition model. One or more emotion representations that represent the emotion of the speech audio are generated. For each of the one or more emotion representations, a respective conditioning input for a facial animation generative model is determined. Facial animation data for the speech audio data is generated, comprising processing, using the facial animation generative model: (i) input data derived from the speech audio data, and (ii) the one or more conditioning inputs.

Abstract: Generating a simplified mesh is disclosed, comprising: receiving an input mesh associated with an outer bound mesh and an inner bound mesh, where each vertex of the input mesh is located outside the outer bound mesh and inside the inner bound mesh; performing one or more mesh simplification operations on the input mesh to generate a level-of-detail (LOD) mesh corresponding to the input mesh; determining that a first vertex of the LOD mesh is located outside the outer bound mesh or inside the inner bound mesh; determining a bound vector for the first vertex that binds the first vertex to the outer bound mesh or inner bound mesh; and generating an updated LOD mesh based on moving the first vertex based on the bound vector, wherein the first vertex in the updated LOD mesh is located inside the outer bound mesh and outside the inner bound mesh.

Abstract: A computer-implemented method for generating a machine-learned inverse rig model. The machine-learned inverse rig model outputs rig parameters for a facial animation rig used to generate animations of facial expressions in video games. The method comprises receiving one or more training examples. Each training example comprises target mesh data for a face portraying a particular facial expression. For each training example, predicted rig parameters are generated using an inverse rig model, comprising processing the target mesh data of the training example. Predicted mesh data is generated, comprising processing the predicted rig parameters using a forward rig model. A loss is determined, comprising performing a comparison between the predicted mesh data and the target mesh data. The inverse rig model is trained, comprising updating parameters of the inverse rig model based on the losses of one or more training examples.

Abstract: The present disclosure discloses the use of machine learning to address the process of motion synthesis and generation of intermediate poses for virtual entities. A transformer-based model can be used to generate intermediate poses for an animation based on a set of key frames.

Abstract: The present disclosure provides a system for generating and rendering virtual objects, such as mesh particles, using dynamic color blending within the virtual environment. Mesh particles may be divided up into portions. For example, the portions of the mesh particle may be a single pixel or a group of pixels. The color of the mesh particles can be dynamically determined for the portions of a mesh particle.

Abstract: Systems and methods for performing a processing operation for a tiled image region are disclosed. The tiled image region may include a plurality of tiles or images. Further, the tiled image region may correspond to a plurality of image resolutions. A system may execute a game development application to perform the processing operation for the tiled image region. The system may identify the tiled image region corresponding to the processing operation. The system can utilize a texture array, a lookup texture, and a scaling factor to determine position data for the tiled image region. The system can then render a continuous image region that represents the tiled image region. The system can seamlessly process the continuous image region according to the processing operation and use the continuous image region to update the tiled image region.

Abstract: A system may perform motion capture using polarized light. For example, the system may determine one or more filtered pixels having an associated degree of linear polarization above a threshold from a plurality of pixels of an image captured by a polarization camera and determine a set of pixels of the filtered pixels are associated with a polarized tag, the polarized tag being on a subject of motion capture. The system may then determine an orientation of the polarized tag based on one or more angles of polarization of the pixels associated with the polarized tag and generate a motion capture pose for a model based on a location of the set of pixels of the filtered pixels in the image and the orientation of the polarized tag.

Abstract: A system may perform motion capture using motion capture targets with concave reflector structures. For example, the motion capture target include a target body and a plurality of tracking markers located on respective portions of the surface of the target body. At least one tracking marker of the plurality of tracking markers may be a concave reflector structure including a tapered hole in the surface of the target body and at least a portion of a surface of the tapered hole may be reflective.

Abstract: A device may access a feature vector generated based on interactions by a user with a video game. The device may access a cluster map comprising a mapping of user clusters, wherein each location within the cluster map is associated with a set of users whose feature vectors are within a threshold degree of similarity of each other. The cluster map may be generated using a plurality of extracted feature vectors obtained from interaction information. A device may determine a map location within the cluster map associated with the user based at least in part on the feature vector. A device may determine a target map location within the cluster map. A device may determine a guidance action based at least in part on the target map location and the map location associated with the user. A device may execute the guidance action.