Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for accessing a set of images depicting at least a portion of a face. A set of facial regions of the face is identified, each facial region of the set of facial regions intersecting another facial region with at least one common vertex that is a member of a set of facial vertices. For each facial region of the set of facial regions, a weight formed from a set of region coefficients is generated. Based on the set of facial regions and the weight of each facial region of the set of facial regions, the face is tracked across the set of images.

Abstract: Systems and methods herein describe novel motion representations for animating articulated objects consisting of distinct parts. The described systems and method access source image data, identify driving image data to modify image feature data in the source image sequence data, generate, using an image transformation neural network, modified source image data comprising a plurality of modified source images depicting modified versions of the image feature data, the image transformation neural network being trained to identify, for each image in the source image data, a driving image from the driving image data, the identified driving image being implemented by the image transformation neural network to modify a corresponding source image in the source image data using motion estimation differences between the identified driving image and the corresponding source image, and stores the modified source image data.

Abstract: Systems and methods herein describe novel motion representations for animating articulated objects consisting of distinct parts. The described systems and method access source image data, identify driving image data to modify image feature data in the source image sequence data, generate, using an image transformation neural network, modified source image data comprising a plurality of modified source images depicting modified versions of the image feature data, the image transformation neural network being trained to identify, for each image in the source image data, a driving image from the driving image data, the identified driving image being implemented by the image transformation neural network to modify a corresponding source image in the source image data using motion estimation differences between the identified driving image and the corresponding source image, and stores the modified source image data.

Abstract: Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for accessing a set of images depicting at least a portion of a face. A set of facial regions of the face is identified, each facial region of the set of facial regions intersecting another facial region with at least one common vertex that is a member of a set of facial vertices. For each facial region of the set of facial regions, a weight formed from a set of region coefficients is generated. Based on the set of facial regions and the weight of each facial region of the set of facial regions, the face is tracked across the set of images.

Abstract: A correlation system can be implemented on a user device (e.g., smartphone) to perform image processing tasks e.g., point to point correlations for tasks such as alignment, tracking, and reconstruction of model data. The correlation system can implement normalized cross correlation in a least squares optimization scheme without use of approximations. Normalized cross correlation optimized via least squares can further implement global and local cost functions that are invariant to dynamic lighting conditions.

Abstract: Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.

Abstract: A correlation system can be implemented on a user device (e.g., smartphone) to perform image processing tasks e.g., point to point correlations for tasks such as alignment, tracking, and reconstruction of model data. The correlation system can implement normalized cross correlation in a least squares optimization scheme without use of approximations. Normalized cross correlation optimized via least squares can further implement global and local cost functions that are invariant to dynamic lighting conditions.

Abstract: Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.

Abstract: Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.

Abstract: Systems and methods herein describe an image compression system. The image compression system generates a first generative adversarial network (GAN), identifies a threshold, based on the threshold, generates a second GAN by pruning channels of the first GAN, trains the second GAN using similarity-based knowledge distillation from the first GAN, and stores the trained second GAN.

Abstract: Systems and methods herein describe a motion retargeting system. The motion retargeting system accesses a plurality of two-dimensional images comprising a person performing a plurality of body poses, extracts a plurality of implicit volumetric representations from the plurality of body poses, generates a three-dimensional warping field, the three-dimensional warping field configured to warp the plurality of implicit volumetric representations from a canonical pose to a target pose, and based on the three-dimensional warping field, generates a two-dimensional image of an artificial person performing the target pose.

Abstract: A correlation system can be implemented on a user device (e.g., smartphone) to perform image processing tasks e.g., point to point correlations for tasks such as alignment, tracking, and reconstruction of model data. The correlation system can implement normalized cross correlation in a least squares optimization scheme without use of approximations. Normalized cross correlation optimized via least squares can further implement global and local cost functions that are invariant to dynamic lighting conditions.

Abstract: A correlation system can be implemented on a user device (e.g., smartphone) to perform image processing tasks e.g., point to point correlations for tasks such as alignment, tracking, and reconstruction of model data. The correlation system can implement normalized cross correlation in a least squares optimization scheme without use of approximations. Normalized cross correlation optimized via least squares can further implement global and local cost functions that are invariant to dynamic lighting conditions.

Abstract: Systems and methods herein describe novel motion representations for animating articulated objects consisting of distinct parts. The described systems and method access source image data, identify driving image data to modify image feature data in the source image sequence data, generate, using an image transformation neural network, modified source image data comprising a plurality of modified source images depicting modified versions of the image feature data, the image transformation neural network being trained to identify, for each image in the source image data, a driving image from the driving image data, the identified driving image being implemented by the image transformation neural network to modify a corresponding source image in the source image data using motion estimation differences between the identified driving image and the corresponding source image, and stores the modified source image data.

Abstract: Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for accessing a set of images depicting at least a portion of a face. A set of facial regions of the face is identified, each facial region of the set of facial regions intersecting another facial region with at least one common vertex that is a member of a set of facial vertices. For each facial region of the set of facial regions, a weight formed from a set of region coefficients is generated. Based on the set of facial regions and the weight of each facial region of the set of facial regions, the face is tracked across the set of images.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for accessing a set of images depicting at least a portion of a face. A set of facial regions of the face is identified, each facial region of the set of facial regions intersecting another facial region with at least one common vertex which is a member of a set of facial vertices. For each facial region of the set of facial regions, a weight formed from a set of region coefficients is generated. Based on the set of facial regions and the weight of each facial region of the set of facial regions, the face is tracked across the set of images.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for determining orientation or location of a device. The program and method include receiving data from one or more sensors of a device; generating a first spline that represents location of the device in a world coordinate frame based on the data received from the one or more sensors; generating a second spline that represents orientation of the device in the world coordinate frame based on the data received from the one or more sensors; computing a motion model that represents motion of the device over a continuous time interval based on the first and second splines; and determining an expected location or orientation of the device based on the motion model.

Abstract: A system for capturing 3D image data of a scene, including three light sources, each configured to emit light at a different wavelength to the other two sources and to illuminate the scene to be captured; a first video camera configured to receive light from the light sources which has been reflected from the scene, to isolate light received from each of the light sources, and to output data relating to the image captured for each of the three light sources; a depth sensor configured to capture depth map data of the scene; and an analysis unit configured to receive data from the first video camera and process the data to obtain data relating to a normal field obtained from the images captured for each of the three light sources, and to combine the normal field data with the depth map data to capture 3D image data of the scene.