Abstract: A messaging system performs image processing to relight objects with neural networks for images provided by users of the messaging system. A method of relighting objects with neural networks includes receiving an input image with first lighting properties comprising an object with second lighting properties and processing the input image using a convolutional neural network to generate an output image with the first lighting properties and comprising the object with third lighting properties, where the convolutional neural network is trained to modify the second lighting properties to be consistent with lighting conditions indicated by the first lighting properties to generate the third lighting properties.

Abstract: A messaging system processes three-dimensional (3D) models to generate ground truths for training machine learning models for applications of the messaging system. A method of generating ground truths for machine learning includes generating a plurality of first rendered images from a first 3D base model where each first rendered image includes the 3D base model modified by first augmentations of a plurality of augmentations.

Abstract: A messaging system processes three-dimensional (3D) models to generate ground truths for training machine learning models for applications of the messaging system. A method of generating ground truths for machine learning includes generating a plurality of first rendered images from a first 3D base model where each first rendered image includes the 3D base model modified by first augmentations of a plurality of augmentations.

Abstract: A messaging system performs image processing to relight objects with neural networks for images provided by users of the messaging system. A method of relighting objects with neural networks includes receiving an input image with first lighting properties comprising an object with second lighting properties and processing the input image using a convolutional neural network to generate an output image with the first lighting properties and comprising the object with third lighting properties, where the convolutional neural network is trained to modify the second lighting properties to be consistent with lighting conditions indicated by the first lighting properties to generate the third lighting properties.

Abstract: Systems and methods for single image-based body animation are provided. An example method includes receiving an input image including an image of a body of a person, fitting a model to the image of the body of the person, where the model is configured to receive a set of pose parameters corresponding to a pose of the body and generate, based on the set of pose parameters, an output image including an image of the body adopting the pose, receiving a further set of pose parameters corresponding to a further pose of the body, providing the further set of pose parameters to the model to generate the output image of the body adopting the further pose, and generating, based on the output image, a frame of an output video including the output image.

Abstract: Disclosed are systems and methods for single image-based body animation. An example method includes receiving an input image, the input image including a body image of a person, extracting the body image of the person from the input image, fitting a generic model to the body image, where the generic model is configured to receive a set of pose parameters corresponding to a pose of the person and generate a generic body shape adopting the pose, generating a three-dimensional (3D) model, where the 3D model is configured to receive a set of further pose parameters corresponding to the pose of the person and generate an output image of the person adopting the pose, the output image including a feature of the body image being omitted from the generic body shape, and providing a further set of further pose parameters to generate a frame of an output video.

Abstract: A messaging system processes three-dimensional (3D) models to generate ground truths for training machine learning models for applications of the messaging system. A method of generating ground truths for machine learning includes generating a plurality of first rendered images from a first 3D base model where each first rendered image includes the 3D base model modified by first augmentations of a plurality of augmentations.

Abstract: Provided are systems and methods for animating a single image of a human body and applying effects. An example method includes receiving an input image that includes an image of a body, receiving, through an interface, a user input including parameters associated with a motion, generating, based on the image of the body and the parameters associated with the motion, a video featuring the body repeating the motion, and providing an option to share the parameters associated with the motion with a further user of a further computing device. The method further includes, prior to generating the video, receiving a further user input including a hair animation effect selected from a list of hair animation effects, and while generating the video, applying the hair animation effect to a hair image in frames of the video.

Abstract: Provided are systems and methods for animating a single image of a human body and applying effects. An example method includes providing a database of motions; receiving an input image including a body of a person; receiving a user input including a motion selected from the database of motions; segmenting the input image into a body portion and a background portion; fitting the body portion to a hair model; generating, based on the body portion and the selected motion, a video featuring the body of the person repeating the selected motion; where generating the video includes detecting positions of key points associated with a head of the person in a frame of the video, generating an image of hair of the person based on the positions of the key points and the hair model, and inserting the image of the hair in the frame; and displaying the generated video.

Abstract: A messaging system performs image processing to estimate lighting properties with neural networks for images provided by users of the messaging system. A method of estimating light properties includes receiving an input image with first lighting properties and processing the input image using a convolutional neural network to generate an estimate of the first lighting properties. The method may further include modifying the input image with an augmentation to generate a modified input image, where the augmentation has second lighting properties, and changing the second lighting properties of the augmentation in the modified input image to the estimate of the first lighting properties.

Abstract: Disclosed are systems and methods for single image-based body animation. An example method includes receiving an input image, the input image including a body image of a person, extracting the body image of the person from the input image, fitting a generic model to the body image, where the generic model is configured to receive a set of pose parameters corresponding to a pose of the person and generate a generic body shape adopting the pose, generating a three-dimensional (3D) model, where the 3D model is configured to receive a set of further pose parameters corresponding to the pose of the person and generate an output image of the person adopting the pose, the output image including a feature of the body image being omitted from the generic body shape, and providing a further set of further pose parameters to generate a frame of an output video.

Abstract: Provided are systems and methods for single image-based body animation. An example method includes receiving an input image that includes a body of a person and segmenting the input image into a body portion and a background portion. The method further includes fitting a model to the body portion. The model is configured to receive a set of pose parameters representing a pose of the body and generate an output image including an image of the body adopting the pose. The method further includes receiving a series of further sets of pose parameters, each representing at least one of further poses of the body. The further sets of pose parameters are generated using a generic model. The method also includes generating a series of output images of the body adopting the further poses and generating an output video based on the series of output images.

Abstract: A messaging system performs image processing to relight objects with neural networks for images provided by users of the messaging system. A method of relighting objects with neural networks includes receiving an input image with first lighting properties comprising an object with second lighting properties and processing the input image using a convolutional neural network to generate an output image with the first lighting properties and comprising the object with third lighting properties, where the convolutional neural network is trained to modify the second lighting properties to be consistent with lighting conditions indicated by the first lighting properties to generate the third lighting properties.

Abstract: Provided are systems and methods for single image-based body animation. An example method includes receiving an input image that includes a body of a person and segmenting the input image into a body portion and a background portion. The method further includes fitting a model to the body portion. The model is configured to receive a set of pose parameters representing a pose of the body and generate an output image including an image of the body adopting the pose. The method further includes receiving a series of further sets of pose parameters, each representing at least one of further poses of the body. The further sets of pose parameters are generated using a generic model. The method also includes generating a series of output images of the body adopting the further poses and generating an output video based on the series of output images.

Abstract: Provided are systems and methods for animating a single image of a human body and applying effects. An example method includes providing a database of motions; receiving an input image including a body of a person; receiving a user input including a motion selected from the database of motions; segmenting the input image into a body portion and a background portion; fitting the body portion to a hair model; generating, based on the body portion and the selected motion, a video featuring the body of the person repeating the selected motion; where generating the video includes detecting positions of key points associated with a head of the person in a frame of the video, generating an image of hair of the person based on the positions of the key points and the hair model, and inserting the image of the hair in the frame; and displaying the generated video.

Abstract: Provided are systems and methods for single image-based body animation. An example method includes receiving a input image, the input image including a body of a person, segmenting the input image into a body portion and a background portion, wherein the body portion includes pixels corresponding to the body of the person, fitting a model to the body portion, wherein the model is configured to receive pose parameters representing a pose of the body and generate an output image including an image of the body adopting the pose, receiving a series of further pose parameters, each of the series of further pose parameters representing one of further poses of the body, providing each of the series of further pose parameters to the model to generate a series of output images of the body adopting the further poses, and generating, based on the series of output images, an output video.

Abstract: Provided are systems and methods for single image-based body animation. An example method includes receiving a input image, the input image including a body of a person, segmenting the input image into a body portion and a background portion, wherein the body portion includes pixels corresponding to the body of the person, fitting a model to the body portion, wherein the model is configured to receive pose parameters representing a pose of the body and generate an output image including an image of the body adopting the pose, receiving a series of further pose parameters, each of the series of further pose parameters representing one of further poses of the body, providing each of the series of further pose parameters to the model to generate a series of output images of the body adopting the further poses, and generating, based on the series of output images, an output video.

Abstract: Provided are systems and methods for animating a single image of a human body and applying effects. An example method includes providing, by a computer device, a database of motions; receiving, by a computing device, an input image, the input image including a body of a person; receiving, by the computing device, a user input including a motion selected from the database of motions; segmenting, by the computing device, the input image into a body portion and a background portion; generating, by the computing device and based on the body portion and the selected motion, a video featuring the body of the person repeating the selected motion; and displaying, by the computing device, the generated video; receiving, by a computer device, a further user input including clothes, scene, illumination effect, and additional objects; and, while generating the video, modifying frames of the video based on the further user input.