Abstract: Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.

Abstract: Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.

Abstract: Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.

Abstract: First and second objects are detected within an image. The first object includes first pixel columns, and the second object includes second pixel columns. A rightmost one of the first pixel columns is adjacent to a leftmost one of the second pixel columns. A first equation is fitted to respective depths of the first pixel columns, and a first depth is computed of the rightmost one of the first pixel columns in response to the first equation. A second equation is fitted to respective depths of the second pixel columns, and a second depth is computed of the leftmost one of the second pixel columns in response to the second equation. The first and second objects are merged in response to the first and second depths being sufficiently similar to one another, and in response to the first and second equations being sufficiently similar to one another.

Abstract: First information is about respective depths of pixel coordinates within an image. Second information is about respective depths of the pixel coordinates within a ground plane. In response to comparing the first information against the second information, respective markings are generated to identify whether any one or more of the pixel coordinates within the image has significant protrusion from the ground plane. In response to a particular depth of a representative pixel coordinate within the image, a window of pixel coordinates is identified that is formed by different pixel coordinates and the representative pixel coordinate. In response to the respective markings, respective probabilities are computed for the pixel coordinates, so that the respective probability for the representative pixel coordinate is computed in response to the respective markings of all pixel coordinates within the window. In response to the respective probabilities, at least one object is detected within the image.