Abstract: Some implementations relate to methods, systems, and computer-readable media for digital advertising. In some implementations, a computer-implemented method includes presenting virtual items at a virtual user engagement station. The virtual user engagement station may include a threshold radius for activation of a change in configuration. Upon an avatar crossing the threshold radius, the avatar may be allowed to temporarily wear or interact with the presented virtual items, display different virtual items, and/or purchase virtual items.

Abstract: A metaverse application receives a user-provided audio stream associated with a user. The metaverse application obtains portions of one or more audio streams. The metaverse application divides the user-provided audio stream into a plurality of portions, wherein each portion corresponds to a particular time window of the audio stream. The metaverse application providing the plurality of portions of the user-provided audio stream as input to an audio machine-learning model. The audio machine-learning model outputs, based on the portions of the user-provided audio stream, a determination of abuse in a particular portion of the plurality of portions. The metaverse application performs a remedial action responsive to the determination of abuse in the particular portion.

Abstract: Some implementations relate to providing adaptive, distributed simulation of fluids with rigid body coupling. According to one aspect, a method simulates, with a coarse resolution on one or more server devices, a global flow field for a fluid within a virtual environment. The method determines regions within the grid to obtain additional detail for, based on the presence of one or more rigid objects within the virtual environment. For each region, the computer-implemented method: assigns client device(s) to the region; sends fluid simulation data pertaining to the region to the client devices; and receives refined fluid simulation data for the region from the client devices. The method replaces the fluid simulation data in the global fluid simulation pertaining to the regions with the refined fluid simulation data for the regions. The method then updates the global flow field for the fluid in real time based on the simulation data.

Abstract: Real-time extraction of human poses from video data for animation of avatars. In some implementations, a computer-implemented method includes obtaining an input video including a plurality of video frames in a sequence that depict movement of a person based on a plurality of poses of the person in the input video. Keypoints of the person are detected in the video frames of the input video data, and a sequence of 3D body poses are determined that correspond to the plurality of poses of the person in the video frames of the input video. Determining the 3D body poses includes using a spatial-temporal transformer to determine joint angles of the keypoints, where the spatial-temporal transformer separately encodes inputs in spatial dimensions within each video frame and a temporal dimension across the video frames.

Abstract: A metaverse application generates a first abuse score for the audio stream associated with a first user based on rules that analyze user signals, wherein the user signals include one or more selected from the group of one or more second users muting the first user, a history of past moderation actions associated with the first user, one or more third users blocking the first user, and combinations thereof. The metaverse application includes an audio-classifier model that obtains a second abuse score for an audio stream associated with the first user, wherein the audio stream is provided as input to the audio-classifier model. The metaverse application determines that the first user is committing abuse based on one or more selected from the group of the first abuse score, the second abuse score, and combinations thereof exceeding a threshold score value. Responsive to determining that the first user is committing abuse, the metaverse application performs a remedial action.

Abstract: Implementations described herein relate to methods, systems, and computer-readable media to localize dynamic content. In some implementations, a computer-implemented method includes receiving visual content associated with a game from a first client locale, the visual content including text being represented at the first client locale by a first language, converting the text to generate translated text in at least a second language associated with a second client locale, and storing the translated text in a database in association with the visual content.

Abstract: Implementations relate to methods, systems, and computer-readable media to render hair and hair-like features on user devices. In some implementations, the method includes receiving, at a user device that participates in a virtual experience, a hair model for a 3D object, wherein the hair model includes a plurality of curves, hair geometry metadata associated with one or more curves of the plurality of curves, and hair simulation metadata, generating, at the user device, hair geometry for the 3D object based on the hair model, the hair geometry metadata, and a type of the user device, simulating, at the user device, hair for the 3D object based on the hair geometry, the hair simulation metadata, and one or more physics parameters of the virtual experience, and rendering the 3D object with the simulated hair on the user device.

Abstract: Implementations described herein relate to methods, systems, and computer-readable media to display a virtual character within a virtual environment on a display device. In some implementations, a method can include obtaining an input pose of the virtual character, wherein the virtual character is based on a rig that comprises a plurality of joints, receiving an indication of one or more of a position and an orientation of a target end effector located on the rig, determining an output pose for the rig wherein the determining comprises calculating a respective orientation and position of one or more joints of the plurality of joints of the rig based on the position of the target end effector and rotation constraints of a plurality of joints of a reference rig, and displaying the virtual character in the output pose on the display device.

Abstract: Some implementations relate to methods, systems, and computer-readable media for determining updated states of rigid body objects. In some implementations, the method includes receiving a model representation of rigid body objects, receiving an input state of the rigid body objects, determining that a relative velocity meets a threshold velocity, responsive to determining that a first rigid body object and a second rigid body object are in contact and that the relative velocity meets the threshold velocity, constructing a merged representation that represents the first rigid body object and the second rigid body object, applying a solver based on the input state and the merged representation to obtain an updated state for the first rigid body object and the second rigid body object, and causing the first rigid body object and the second rigid body object in the updated state to be displayed within a virtual environment.

Abstract: Generation of avatar animations for virtual environments using motion modifiers. In some implementations, a computer-implemented method includes determining a base animation defined by animation curves that each include position values that indicate positions of an associated portion of an avatar at points of time. One or more motion modifiers and parameters therefor are determined, each motion modifier composed of one or more modification primitives configured to modify respective animation curves of the base animation that are associated with respective portions of the avatar. A modified animation is generated by applying the motion modifiers to the base animation, where each of the modification primitives of each motion modifier modifies a respective animation curve for an associated portion of the avatar based on the parameters of the motion modifiers.

Abstract: Automatic extraction of salient object properties in virtual environments for object modification and transmission. In some implementations, a computer-implemented method includes determining a reference avatar and obtaining properties of an object in the virtual environment, the properties including spatial, visual, and/or audio properties. Saliency factors of the object are determined, each saliency factor normalized to a numeric range and based on a different set of properties of the object, where one or more saliency factors are additionally based on a property of the reference avatar. A saliency measure of the object is determined with respect to the reference avatar based on a combination of the saliency factors. If the saliency measure is greater than a threshold saliency measure, the reference avatar or object are automatically modified in the virtual environment based on the object, and otherwise the modification is omitted.

Abstract: Implementations described herein relate to methods, systems, and computer-readable media for 2D control over a 3D virtual environment. A method may include providing a 3D virtual experience, receiving first indication that the avatar is within a threshold distance of at least one interactive displayed graphical element and second indication that the at least one interactive displayed graphical element is within a viewport, receiving an activation signal to activate a 2D interaction mode, automatically providing a projection of the at least two portions onto a 2D plane within the viewport associated with the avatar responsive to the activation signal and the indications, receiving a selection of one or more of the bounding box representations, and performing a function associated with the selection, the function identical to an associated function of the at least one interactive graphical element.

Abstract: Some implementations relate to methods, systems, and computer-readable media to stylize a three-dimensional (3D) avatar in a virtual experience using controls to set constraints. The stylizing may include performing applying a scaling control to a body part of the avatar body to change a size of at least one component of the body part, applying a positional control to the body part of the avatar body to change at least one from the group of a shape of the at least one component of the body part, a position of the at least one component of the body part, and a combination thereof, and/or applying an orientation control to the body part of the avatar body to change an orientation of the at least one component of the body part; and deforming the body part of the avatar body accordingly based on minimizing an energy function.

Abstract: Layered clothing (LC) may be fitted on a three-dimensional (3D) avatar for a virtual experience. Some implementations relate to methods, systems, and computer-readable media to perform validation checks to check the quality (such as correctness and functionality) of the layered clothing. In some implementations, the method includes performing at least one static validation check on the clothing item to validate the clothing item when layered over an underlying surface, based on at least one property of the clothing item from the group comprising: an inner cage of the clothing item, an outer cage of the clothing item, a reference mesh of the clothing item, and combinations thereof; and in response to detecting at least one failure result from the at least one static validation check, providing an identified issue based on the at least one static validation check having the at least one failure result.

Abstract: Some implementations relate to methods, systems, and computer-readable media to fit a clothing item onto an avatar body in a three-dimensional environment. To obtain a more tailored/precise fitting, dimensions of the clothing item are constrained according to various reference dimensions. A reference region for the avatar body is identified along with a reference dimension. A first region of a clothing item is identified, the first region having a first dimension. At least one second dimension of the clothing item is identified, and at least one first relationship between the first dimension and the second dimension is determined. The first dimension is changed to correspond to the reference dimension and the at least one second dimension is changed to scale the clothing item along the at least one second dimension, wherein the at least one first relationship is maintained after the changing of the at least one second dimension.

Abstract: Implementations relate to methods, systems, and computer-readable media for performing similarity analysis of three-dimensional (3D) objects. In some implementations, the method includes obtaining a 3D model of the object that includes a mesh and a texture, encoding the mesh into a mesh feature vector using a first neural network, encoding the texture into a texture feature vector using a second neural network, computing a mesh distance between the mesh feature vector and a reference mesh feature vector of a reference object, computing a texture distance between the texture feature vector and a reference texture feature vector of the reference object, determining, based on the mesh distance and the texture distance, whether the object matches the reference object, classifying the object as a dissimilar object if the object does not match the reference object, and classifying the object as a similar object if the object matches the reference object.

Abstract: A metaverse application receives user input from a user during gameplay of a virtual experience. The metaverse application renders a first game state of gameplay of the virtual experience on a user device based on the user input, wherein the first game state is described by a set of properties. The metaverse application generates a list of changes in properties between the first game state and a second game state, wherein the second game state occurred before the first game state. The metaverse application receives a request from the user to replay the second game state. The metaverse application renders the second game state of gameplay by reversing the list of changes between the first game state and the second game state.

Abstract: Some implementations relate to methods, systems, and computer-readable media to provide scalable soft body locomotion/animation for a virtual experience, such as a three-dimensional (3D) environment. In some implementations, the method includes building a control space having information representative of forces corresponding to natural movement of the soft body, wherein the soft body is part of a virtual environment, coupling the control space and a physical space to define a controller pipeline that performs animation of the soft body, performing the animation of the soft body using the controller pipeline, and causing the animation of the soft body to be displayed in a user interface of the virtual environment. Building the control space may comprise simulating the forces corresponding to the natural movement of the soft body by solving an elastodynamic optimization problem using auxiliary variables as degrees of freedom.

Abstract: Some implementations relate to methods, systems, and computer-readable media to display a rigid body object within a virtual environment. In some implementations, the method includes obtaining a model representation of the rigid body object, obtaining a state of the rigid body object in the virtual environment, determining a plurality of constraints that act on the rigid body object, determining a plurality of Jacobians and a plurality of Hessians, wherein each Jacobian of the plurality of Jacobians and each Hessian of the plurality of Hessians is associated with a corresponding constraint of the plurality of constraints, applying a two-stage primal solver to a set of equations of motion for the rigid body object based on the plurality of Jacobians and the plurality of Hessians, and displaying the rigid body object in the updated position in the virtual environment.

Abstract: Various input modes and output modes may be used for a three-dimensional (3D) environment. A user may use a particular input mode (e.g., text, audio, video, etc.) for animating a 3D avatar of the user in the 3D environment. The user may use a particular output mode (e.g., text, audio, 3D animation, etc.) in the presentation of the 3D environment. The input/output modes may change based on conditions such as a location of the user.