Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising: capturing a video that depicts a person; identifying a set of skeletal joints of the person depicted in the video; storing a movement vector representing previously captured three-dimensional (3D) movement of the set of skeletal joints of the person depicted in the video; receiving input that selects a 3D avatar; and animating, based on the movement vector, the 3D avatar to mimic the previously captured 3D movement of the set of skeletal joints of the person depicted in the video.

Abstract: Embodiments of the present disclosure relate generally to systems for enhancing a first media item through the addition of a supplemental second media item. A user may provide a request to enhance a selected media item, and in response, an enhancement system retrieves and presents a curated collection of supplemental content to be added to the media, to the user. The user may review the curated collection of supplemental content, for example by providing a tactile input to scroll through the curated collection of content.

Abstract: A messaging system performs virtual clothing try-on. A method of virtual clothing try-on may include accessing a target garment image and a person image of a person wearing a source garment and processing the person image to generate a source garment mask and a person mask. The method may further include processing the source garment mask, the person mask, the target garment image, and a target garment mask to generate a warping, the warping indicating a warping to apply to the target garment image. The method may further include processing the target garment to warp the target garment in accordance with the warping to generate a warped target garment image, processing the warped target garment image to blend with the person image to generate a person with a blended target garment image, and processing the person with blended target garment image to fill in holes to generate an output image.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising: capturing a video that depicts a person; identifying a set of skeletal joints of the person depicted in the video; storing a movement vector representing previously captured three-dimensional (3D) movement of the set of skeletal joints of the person depicted in the video; receiving input that selects a 3D avatar; and animating, based on the movement vector, the 3D avatar to mimic the previously captured 3D movement of the set of skeletal joints of the person depicted in the video.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising: receiving, by a client device associated with a first user, a communication from a second user; retrieving, from the communication, a movement vector representing three-dimensional (3D) movement of a set of skeletal joints of the second user; receiving, by the client device associated with the first user, input that selects a 3D avatar; and animating, based on the movement vector, the 3D avatar to mimic the 3D movement of the set of skeletal joints of the second user.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising: receiving a video that depicts a person; identifying a set of skeletal joints corresponding to limbs of the person; tracking 3D movement of the set of skeletal joints corresponding to the limbs of the person in the video; causing display of a 3D virtual object that has a plurality of limbs including one or more extra limbs than the limbs of the person in the video; and moving the one or more extra limbs of the 3D virtual object based on the movement of the set of skeletal joints corresponding to the limbs of the person in the video.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising: receiving a video that depicts a person; tracking three-dimensional (3D) movement of the person within the video using a 3D reference point; computing a 3D position for placement of an augmented reality item relative to the 3D reference point; causing to be displayed the augmented reality item within the video at the 3D position; and updating the 3D position of the augmented reality item in the video as the 3D reference point changes based on the 3D movement of the person.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving a video that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body depicted in the video; tracking movement of the plurality of skeletal joints across a set of frames of the video; and smoothing the movement of a first set of the plurality of skeletal joints between frames in the first set of frames independently of smoothing movement of a second set of the plurality of skeletal joints in the first set of frames.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include operations comprising: receiving a monocular image that includes a depiction of a body of a user in a given pose; detecting a plurality of skeletal joints of the body depicted in the monocular image; generating an outline of the body of the user based on the plurality of skeletal joints; retrieving a mask that represents a target pose; comparing the outline of the body depicted in the monocular image with the mask that represents the target pose; and determining that the given pose of the body of the user matches the target pose in response to comparing the outline of the body with the mask.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include operations comprising receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body based on the monocular image; accessing a video feed comprising a plurality of monocular images received prior to the monocular image; filtering, using the video feed, the plurality of skeletal joints of the body detected based on the monocular image; and determining a pose represented by the body depicted in the monocular image based on the filtered plurality of skeletal joints of the body.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body depicted in the monocular image; and determining a pose represented by the body depicted in the monocular image based on the detected plurality of skeletal joints of the body. A pose of an avatar is modified to match the pose represented by the body depicted in the monocular image by adjusting a set of skeletal joints of a rig of an avatar based on the detected plurality of skeletal joints of the body; and the avatar having the modified pose that matches the pose represented by the body depicted in the monocular image is generated for display.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include operations comprising receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body based on the monocular image; accessing a video feed comprising a plurality of monocular images received prior to the monocular image; filtering, using the video feed, the plurality of skeletal joints of the body detected based on the monocular image; and determining a pose represented by the body depicted in the monocular image based on the filtered plurality of skeletal joints of the body.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include operations comprising receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body based on the monocular image; accessing a video feed comprising a plurality of monocular images received prior to the monocular image; filtering, using the video feed, the plurality of skeletal joints of the body detected based on the monocular image; and determining a pose represented by the body depicted in the monocular image based on the filtered plurality of skeletal joints of the body.

Abstract: Embodiments of the present disclosure relate generally to systems for enhancing a first media item through the addition of a supplemental second media item. A user may provide a request to enhance a selected media item, and in response, an enhancement system retrieves and presents a curated collection of supplemental content to be added to the media, to the user. The user may review the curated collection of supplemental content, for example by providing a tactile input to scroll through the curated collection of content.

Abstract: Embodiments of the present disclosure relate generally to systems for enhancing a first media item through the addition of a supplemental second media item. A user may provide a request to enhance a selected media item, and in response, an enhancement system retrieves and presents a curated collection of supplemental content to be added to the media, to the user. The user may review the curated collection of supplemental content, for example by providing a tactile input to scroll through the curated collection of content.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body depicted in the monocular image; and determining a pose represented by the body depicted in the monocular image based on the detected plurality of skeletal joints of the body. A pose of an avatar is modified to match the pose represented by the body depicted in the monocular image by adjusting a set of skeletal joints of a rig of an avatar based on the detected plurality of skeletal joints of the body; and the avatar having the modified pose that matches the pose represented by the body depicted in the monocular image is generated for display.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body depicted in the monocular image; and determining a pose represented by the body depicted in the monocular image based on the detected plurality of skeletal joints of the body. A pose of an avatar is modified to match the pose represented by the body depicted in the monocular image by adjusting a set of skeletal joints of a rig of an avatar based on the detected plurality of skeletal joints of the body; and the avatar having the modified pose that matches the pose represented by the body depicted in the monocular image is generated for display.

Abstract: Embodiments of the present disclosure relate generally to systems for enhancing a first media item through the addition of a supplemental second media item. A user may provide a request to enhance a selected media item, and in response, an enhancement system retrieves and presents a curated collection of supplemental content to be added to the media, to the user. The user may review the curated collection of supplemental content, for example by providing a tactile input to scroll through the curated collection of content.

Abstract: A method, apparatus, system, and article of manufacture provide the ability to seamlessly transfer a texture. An image that has an object to be concealed is acquired. A conceal tool is activated. A source patch of a first collection of pixels within the image is defined. A target patch of a second collection of pixels within the image is defined and located over the object to be concealed. The source patch is mapped onto the target patch using a transformation (T). A contrast between the source patch and the target patch along a boundary of the source patch and the target patch is measured. A color one of the one or more pixels in the second collection is accepted/rejected based on the contrast.

Abstract: A method, apparatus, system, and article of manufacture provide the ability to seamlessly transfer a texture. An image that has an object to be concealed is acquired. A conceal tool is activated. A source patch of a first collection of pixels within the image is defined. A target patch of a second collection of pixels within the image is defined and located over the object to be concealed. The source patch is mapped onto the target patch using a transformation (T). A contrast between the source patch and the target patch along a boundary of the source patch and the target patch is measured. A color one of the one or more pixels in the second collection is accepted/rejected based on the contrast.