Abstract: The present invention sets forth techniques for animating a three-dimensional (3D) character model. The disclosed techniques include receiving one or more spatial constraints associated with a first set of one or more reference points included in a first 3D character model, and generating, via a first machine learning model and based on the one or more spatial constraints, a set of trajectories associated with each reference point included in the first set of one or more reference points. The techniques also include generating, via a second machine learning model and based on the set of trajectories, a set of framewise positions associated with a second set of one or more reference points included in a second 3D character model. The techniques further include generating an output animation depicting motion of the second 3D character model based on the set of framewise positions.

Abstract: One embodiment of the present invention sets forth a technique for generating a motion for a virtual character. The technique includes determining a graph representation of a plurality of sets of joints corresponding to a sequence of poses for the virtual character based on(i) a base motion associated with the sequence of poses and(ii) a set of constraints associated with one or more joints included in the plurality of sets of joints. The technique also includes generating, via execution of a first neural network, a set of updated node states for the plurality of sets of joints based on the graph representation. The technique further includes generating, based on the set of updated node states, the motion that includes(i) a first set of joint positions for the plurality of sets of joints and(ii) a first set of joint orientations for the plurality of sets of joints.

Abstract: One embodiment of the present invention sets forth a technique for generating a motion for a virtual character. The technique includes determining a graph representation of a plurality of sets of joints corresponding to a sequence of poses for the virtual character based on (i) one or more input poses for the virtual character and (ii) a set of constraints associated with one or more joints included in the plurality of sets of joints. The technique also includes generating, via execution of a neural network, a set of updated node states for the plurality of sets of joints based on the graph representation. The technique further includes generating, based on the updated node states, the motion that includes (i) a first set of joint positions for the plurality of sets of joints and (ii) a first set of joint orientations for the plurality of sets of joints.

Abstract: One embodiment of the present invention sets forth a technique for generating a pose for a virtual character. The technique includes determining a set of joint representations corresponding to a set of joints in the virtual character based on (i) a base pose for the virtual character and (ii) a set of constraints associated with one or more joints included in the set of joints. The technique also includes generating, via execution of a first neural network, a set of updated joint states for the set of joints based on the set of joint representations. The technique further includes generating, based on the set of updated joint states, an output pose that includes (i) a first set of joint positions for the set of joints and (ii) a first set of joint orientations for the set of joints.

Abstract: The present invention sets forth a technique for performing automated motion adaptation in an animated sequence. The technique includes generating a set of one or more motion sequences based on motion capture data and selecting one of the set of motion sequences based on a score function value. The technique also includes adapting the selected motion sequence such that the adapted motion sequence, when applied to an animation character model, causes the animation character model to move from a specified starting position associated with the animation character to a specified goal position associated with the animation character.

Abstract: One embodiment of the present invention sets forth a technique for training a neural motion controller. The technique includes determining a first set of features associated with a first control signal for a virtual character. The technique also includes matching the first set of features to a first sequence of motions included in a plurality of sequences of motions. The technique further includes training the neural motion controller based on one or more motions included in the first sequence of motions and the first control signal.

Abstract: Techniques are disclosed for learning a machine learning model that maps control data, such as renderings of skeletons, and associated three-dimensional (3D) information to two-dimensional (2D) renderings of a character. The machine learning model may be an adaptation of the U-Net architecture that accounts for 3D information and is trained using a perceptual loss between images generated by the machine learning model and ground truth images. Once trained, the machine learning model may be used to animate a character, such as in the context of previsualization or a video game, based on control of associated control points.

Abstract: In particular embodiments, a 2D representation of an object may be provided. A first method may comprise: receiving sketch input identifying a target position for a specified portion of the object; computing a deformation for the object within the context of a character rig specification for the object; and displaying an updated version of the object. A second method may comprise detecting sketch input; classifying the sketch input, based on the 2D representation, as an instantiation of the object; instantiating the object using a 3D model of the object; and displaying a 3D visual representation of the object.

Abstract: Techniques are disclosed for learning a machine learning model that maps control data, such as renderings of skeletons, and associated three-dimensional (3D) information to two-dimensional (2D) renderings of a character. The machine learning model may be an adaptation of the U-Net architecture that accounts for 3D information and is trained using a perceptual loss between images generated by the machine learning model and ground truth images. Once trained, the machine learning model may be used to animate a character, such as in the context of previsualization or a video game, based on control of associated control points.

Abstract: A simulation engine models interactions between a simulated character and real-world objects to produce a physically realistic augmented reality (AR) simulation. The simulation engine recognizes a given real-world object and then identifies, within a library of object models, an object model corresponding to that object. The simulation engine projects the object model onto the real-world object such that the object model is geometrically aligned with the real-world object. When the simulated character encounters the real-world object, the simulation engine models interactions between the simulated character and the real-world object by adjusting the kinematics of the simulated character relative to the object model associated with the real-world object.

Abstract: A simulation engine models interactions between a simulated character and real-world objects to produce a physically realistic augmented reality (AR) simulation. The simulation engine recognizes a given real-world object and then identifies, within a library of object models, an object model corresponding to that object. The simulation engine projects the object model onto the real-world object such that the object model is geometrically aligned with the real-world object. When the simulated character encounters the real-world object, the simulation engine models interactions between the simulated character and the real-world object by adjusting the kinematics of the simulated character relative to the object model associated with the real-world object.

Abstract: Embodiments capture one or more images of a visual scene that includes a first physical object. A first region of the first physical object to apply one or more augmentations to is determined. Embodiments determine which configuration the first physical object is currently in. The first physical object is configured to be physically manipulatable into each of a plurality of configurations. A sequence of frames is rendered for display in which the first region of the first physical object is animated in a predefined manner depicting a virtual light source within the first physical object, based on the determined configuration of the first physical object, by applying the one or more augmentations to a first virtual object generated based on predefined geometric information corresponding to a determined object type of the first physical object. The rendered sequence of frames is output for display using one or more display devices.

Abstract: Embodiments provide techniques for adjusting coloration of an image. A first selection of first one or more reference points within a first image is received. A second selection of a second one or more reference points within a second image is also received. Embodiments determine a coloration difference between a coloration of the first one or more reference points within the first image and a coloration of the second one or more reference points within the second image. The coloration of at least a portion of the first image is then adjusted, based on the determined coloration difference.

Abstract: Embodiments capture one or more images of a visual scene that includes a first physical object. A first region of the first physical object to apply one or more augmentations to is determined. Embodiments determine which configuration the first physical object is currently in. The first physical object is configured to be physically manipulatable into each of a plurality of configurations. A sequence of frames is rendered for display in which the first region of the first physical object is animated in a predefined manner depicting a virtual light source within the first physical object, based on the determined configuration of the first physical object, by applying the one or more augmentations to a first virtual object generated based on predefined geometric information corresponding to a determined object type of the first physical object. The rendered sequence of frames is output for display using one or more display devices.

Abstract: Techniques for displaying an augmented reality toy. Embodiments capture a visual scene for display. The visual scene includes a first physical object and is captured using one or more camera devices. The first physical object is identified as a first predetermined object type, based on one or more object identifiers associated with the first physical object. Embodiments retrieve predefined geometric information corresponding to the first predetermined object type and render a sequence of frames for display in which the captured visual scene is augmented, based on the predefined geometric information.

Abstract: Embodiments provide techniques for adjusting coloration of an image. A first selection of first one or more reference points within a first image is received. A second selection of a second one or more reference points within a second image is also received. Embodiments determine a coloration difference between a coloration of the first one or more reference points within the first image and a coloration of the second one or more reference points within the second image. The coloration of at least a portion of the first image is then adjusted, based on the determined coloration difference.

Abstract: In particular embodiments, a 2D representation of an object may be provided. A first method may comprise: receiving sketch input identifying a target position for a specified portion of the object; computing a deformation for the object within the context of a character rig specification for the object; and displaying an updated version of the object. A second method may comprise detecting sketch input; classifying the sketch input, based on the 2D representation, as an instantiation of the object; instantiating the object using a 3D model of the object; and displaying a 3D visual representation of the object.

Abstract: Embodiments provide techniques for adjusting coloration of an image. A visual scene is captured using one or more camera devices, where the visual scene includes a reference object. Embodiments retrieve coloration information corresponding to the reference object. The coloration information generally describes coloration of the reference object under predefined conditions, such as neutral lighting conditions, desired lighting conditions, and so on. Additionally, a coloration difference is calculated between a depiction of the reference object within the captured visual scene and the retrieved coloration information. The coloration of the visual scene is adjusted based on the determined coloration difference.

Abstract: Embodiments provide techniques for adjusting coloration of an image. A visual scene is captured using one or more camera devices, where the visual scene includes a reference object. Embodiments retrieve coloration information corresponding to the reference object. The coloration information generally describes coloration of the reference object under predefined conditions, such as neutral lighting conditions, desired lighting conditions, and so on. Additionally, a coloration difference is calculated between a depiction of the reference object within the captured visual scene and the retrieved coloration information. The coloration of the visual scene is adjusted based on the determined coloration difference.

Abstract: Techniques for displaying an augmented reality toy. Embodiments capture a visual scene for display. The visual scene includes a first physical object and is captured using one or more camera devices. The first physical object is identified as a first predetermined object type, based on one or more object identifiers associated with the first physical object. Embodiments retrieve predefined geometric information corresponding to the first predetermined object type and render a sequence of frames for display in which the captured visual scene is augmented, based on the predefined geometric information.