Abstract: A scaling application estimates a downscaling kernel used to generate a downscaled image. The scaling application upscales the downscaled image based on the estimated downscaling kernel, thereby generating a higher resolution version of the downscaled image with minimal visual artifacts. The scaling application includes various networks that perform the above operations. A kernel mapping network generates a degradation map based on the estimated downscaling kernel. A degradation-aware generator network generates a reconstructed image based on the downscaled image and the degradation map. A kernel discriminator network generates an image delta that reflects visual artifacts present in the reconstructed image. The scaling application includes a parameter optimizer that iteratively modifies the estimated downscaling kernel to reduce visual artifacts indicated in the image delta.

Abstract: In various embodiments, a differentiable rendering application enables an inverse rendering application to infer attributes associated with a 3D scene. In operation, the differentiable rendering application renders an image based on a first set of points associated with the 3D scene. The differentiable rendering application then generates an artificial gradient that approximates a change in a value of a first pixel included in the image with respect to a change in an attribute of a first point included in the first set of points. Subsequently, the inverse rendering application performs optimization operation(s) on the first point based on the artificial gradient to generate a second set of points. Notably, an error associated with the second set of points is less than an error associated with the first set of points.

Abstract: In various embodiments, a differentiable rendering application enables an inverse rendering application to infer attributes associated with a 3D scene. In operation, the differentiable rendering application renders an image based on a first set of points associated with the 3D scene. The differentiable rendering application then generates an artificial gradient that approximates a change in a value of a first pixel included in the image with respect to a change in an attribute of a first point included in the first set of points. Subsequently, the inverse rendering application performs optimization operation(s) on the first point based on the artificial gradient to generate a second set of points. Notably, an error associated with the second set of points is less than an error associated with the first set of points.

Abstract: A scaling application estimates a downscaling kernel used to generate a downscaled image. The scaling application upscales the downscaled image based on the estimated downscaling kernel, thereby generating a higher resolution version of the downscaled image with minimal visual artifacts. The scaling application includes various networks that perform the above operations. A kernel mapping network generates a degradation map based on the estimated downscaling kernel. A degradation-aware generator network generates a reconstructed image based on the downscaled image and the degradation map. A kernel discriminator network generates an image delta that reflects visual artifacts present in the reconstructed image. The scaling application includes a parameter optimizer that iteratively modifies the estimated downscaling kernel to reduce visual artifacts indicated in the image delta.

Abstract: The present disclosure relates to techniques for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. The object is reconstructed in three dimensions by computing depth values for edges of the object in the set of two-dimensional images. The set of two-dimensional images may be samples of a light field surrounding the object. The depth values may be computed by exploiting local gradient information in the set of two-dimensional images. After computing the depth values for the edges, depth values between the edges may be determined by identifying types of the edges (e.g., a texture edge, a silhouette edge, or other type of edge). Then, the depth values from the set of two-dimensional images may be aggregated in a three-dimensional space using a voting scheme, allowing the reconstruction of the object in three dimensions.

Abstract: Particular embodiments perform a light path analysis of an image comprising a scene, wherein the scene comprises at least one refractive or reflective object. The image may be decomposed based on the light path analysis into a plurality of components, each of the components representing a contribution to lighting in the scene by a different type of light interaction. For each of the components, one or more motion vectors are extracted for each of the components in order to capture motion in the scene. Finally, a final contribution of each of the components to the image is computed based on the motion vectors.

Abstract: Particular embodiments decompose an image comprising a scene into a diffuse component and a specular component. Each of the components represent a contribution to lighting in the scene. A set of motion vectors may be extracted in order to capture motion in the scene. Finally, a final contribution of each of the components to the image may be computed based on the motion vectors.

Abstract: Enhanced removing of noise and outliers from one or more point sets generated by image-based 3D reconstruction techniques is provided. In accordance with the disclosure, input images and corresponding depth maps can be used to remove pixels that are geometrically and/or photometrically inconsistent with the colored surface implied by the input images. This allows standard surface reconstruction methods (such as Poisson surface reconstruction) to perform less smoothing and thus achieve higher quality surfaces with more features. In some implementations, the enhanced point-cloud noise removal in accordance with the disclosure can include computing per-view depth maps, and detecting and removing noisy points and outliers from each per-view point cloud by checking if points are consistent with the surface implied by the other input views.

Abstract: The present disclosure relates to techniques for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. The object is reconstructed in three dimensions by computing depth values for edges of the object in the set of two-dimensional images. The set of two-dimensional images may be samples of a light field surrounding the object. The depth values may be computed by exploiting local gradient information in the set of two-dimensional images. After computing the depth values for the edges, depth values between the edges may be determined by identifying types of the edges (e.g., a texture edge, a silhouette edge, or other type of edge). Then, the depth values from the set of two-dimensional images may be aggregated in a three-dimensional space using a voting scheme, allowing the reconstruction of the object in three dimensions.

Abstract: Enhanced removing of noise and outliers from one or more point sets generated by image-based 3D reconstruction techniques is provided. In accordance with the disclosure, input images and corresponding depth maps can be used to remove pixels that are geometrically and/or photometrically inconsistent with the colored surface implied by the input images. This allows standard surface reconstruction methods (such as Poisson surface reconstruction) to perform less smoothing and thus achieve higher quality surfaces with more features. In some implementations, the enhanced point-cloud noise removal in accordance with the disclosure can include computing per-view depth maps, and detecting and removing noisy points and outliers from each per-view point cloud by checking if points are consistent with the surface implied by the other input views.

Abstract: Enhanced removing of noise and outliers from one or more point sets generated by image-based 3D reconstruction techniques is provided. In accordance with the disclosure, input images and corresponding depth maps can be used to remove pixels that are geometrically and/or photometrically inconsistent with the colored surface implied by the input images. This allows standard surface reconstruction methods (such as Poisson surface reconstruction) to perform less smoothing and thus achieve higher quality surfaces with more features. In some implementations, the enhanced point-cloud noise removal in accordance with the disclosure can include computing per-view depth maps, and detecting and removing noisy points and outliers from each per-view point cloud by checking if points are consistent with the surface implied by the other input views.

Abstract: The present disclosure relates to techniques for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. The object is reconstructed in three dimensions by computing depth values for edges of the object in the set of two-dimensional images. The set of two-dimensional images may be samples of a light field surrounding the object. The depth values may be computed by exploiting local gradient information in the set of two-dimensional images. After computing the depth values for the edges, depth values between the edges may be determined by identifying types of the edges (e.g., a texture edge, a silhouette edge, or other type of edge). Then, the depth values from the set of two-dimensional images may be aggregated in a three-dimensional space using a voting scheme, allowing the reconstruction of the object in three dimensions.

Abstract: The present disclosure relates to techniques for reconstructing an object in three dimensions that is captured in a set of two-dimensional images. The object is reconstructed in three dimensions by computing depth values for edges of the object in the set of two-dimensional images. The set of two-dimensional images may be samples of a light field surrounding the object. The depth values may be computed by exploiting local gradient information in the set of two-dimensional images. After computing the depth values for the edges, depth values between the edges may be determined by identifying types of the edges (e.g., a texture edge, a silhouette edge, or other type of edge). Then, the depth values from the set of two-dimensional images may be aggregated in a three-dimensional space using a voting scheme, allowing the reconstruction of the object in three dimensions.

Abstract: Enhanced removing of noise and outliers from one or more point sets generated by image-based 3D reconstruction techniques is provided. In accordance with the disclosure, input images and corresponding depth maps can be used to remove pixels that are geometrically and/or photometrically inconsistent with the colored surface implied by the input images. This allows standard surface reconstruction methods (such as Poisson surface reconstruction) to perform less smoothing and thus achieve higher quality surfaces with more features. In some implementations, the enhanced point-cloud noise removal in accordance with the disclosure can include computing per-view depth maps, and detecting and removing noisy points and outliers from each per-view point cloud by checking if points are consistent with the surface implied by the other input views.

Abstract: Systems and method for the reconstruction of an articulated object are disclosed herein, The articulated object can be reconstructed from image data collected by a moving camera over a period of time. A plurality of 2D feature points can be identified within the image data. These 2D feature points can be converted into three-dimensional space, which converted points are identified as 3D feature points. These 3D feature points can be used to identify one or several rigidity constrains and/or kinematic constraints. These rigidity and/or kinematic constraints can be applied to a model of the reconstructed articulated object.

Abstract: Systems and method for the reconstruction of an articulated object are disclosed herein, The articulated object can be reconstructed from image data collected by a moving camera over a period of time. A plurality of 2D feature points can be identified within the image data. These 2D feature points can be converted into three-dimensional space, which converted points are identified as 3D feature points. These 3D feature points can be used to identify one or several rigidity constrains and/or kinematic constraints. These rigidity and/or kinematic constraints can be applied to a model of the reconstructed articulated object.

Abstract: Particular embodiments decompose an image comprising a scene into a diffuse component and a specular component. Each of the components represent a contribution to lighting in the scene. A set of motion vectors may be extracted in order to capture motion in the scene. Finally, a final contribution of each of the components to the image may be computed based on the motion vectors.

Abstract: Particular embodiments perform a light path analysis of an image comprising a scene, wherein the scene comprises at least one refractive or reflective object. The image may be decomposed based on the light path analysis into a plurality of components, each of the components representing a contribution to lighting in the scene by a different type of light interaction. For each of the components, one or more motion vectors are extracted for each of the components in order to capture motion in the scene. Finally, a final contribution of each of the components to the image is computed based on the motion vectors.

Abstract: System, method, and computer program product to perform an operation, comprising sampling a plurality of points and a plurality of segments of a curve on a surface of a three-dimensional model, storing each sampled point as a respective vertex of a plurality of vertices and each sampled segment as a respective half-edge in a curve network of the model surface, upon determining that a first half-edge and a second half-edge connect two of the plurality of vertices, generating a first halfchain connecting the first half-edge and the second half-edge, wherein each connected vertex comprises either a corner or an open endpoint, and upon determining that three consecutive halfchains form a loop comprising at least three corners, generating a first patch for a space enclosed by the loop, wherein the first patch is represented as a quad mesh with a respective set of vertices, faces, and half-edges.

Abstract: The disclosure provides an approach for transferring image edits from a source image to target images. In one embodiment, a warp application receives a user-selected region of interest in a source image and determines for the region of interest content-aware bounded weight functions and seed locations for the same. For each of the target images, the warping application initializes a linear blend skinning subspace warp to a projection onto a feature space of a piecewise affine map from scale invariant feature transform features of the source image to the target image. After initializing the warps, the warping application iteratively optimizes the warps by applying the inverse compositional Lucas-Kanade procedure and using the content-aware weight functions in said procedure. Edits made to the source image may automatically be transferred to target images by warping those edits via the optimized warp function for the respective target images.