Abstract: A deep neural network based hair rendering system is presented to model high frequency component of furry objects. Compared with existing approaches, the present method can generate photo-realistic rendering results. An acceleration method is applied in our framework, which can speed up training and rendering processes. In addition, a patch-based training scheme is introduced, which significantly increases the quality of outputs and preserves high frequency details.

Abstract: An image rendering system comprising a preprocessing unit coupled to a feature extract unit and a color rendering unit over a data bus. The preprocessing unit generates vector representations of spatial coordinates of sample points along camera rays corresponding to pixels of an image to be rendered. The feature extract unit generates a feature map of the image based on the vector representations, color and intensity values of the sample point through a first machine learning model. The color rendering unit renders the image based on the feature map through a second machine learning model. The first machine learning model is different from the second machine learning model.

Abstract: An image-based method of modeling and rendering a three-dimensional model of an object is provided. The method comprises: obtaining a three-dimensional point cloud at each frame of a synchronized, multi-view video of an object, wherein the video comprises a plurality of frames; extracting a feature descriptor for each point in the point cloud for the plurality of frames without storing the feature descriptor for each frame; producing a two-dimensional feature map for a target camera; and using an anti-aliased convolutional neural network to decode the feature map into an image and a foreground mask.

Abstract: A method of processing light field images for separating a transmitted layer from a reflection layer. The method comprises capturing a plurality of views at a plurality of viewpoints with different polarization angles; obtaining an initial disparity estimation for a first view using SIFT-flow, and warping the first view to a reference view; optimizing an objective function comprising a transmitted layer and a secondary layer using an Augmented Lagrange Multiplier (ALM) with Alternating Direction Minimizing (ADM) strategy; updating the disparity estimation for the first view; repeating the steps of optimizing the objective function and updating the disparity estimation until the change in the objective function between two consecutive iterations is below a threshold; and separating the transmitted layer and the secondary layer using the disparity estimation for the first view.

Abstract: A computer-implemented method includes encoding a radiance field of an object onto a machine learning model; conducting, based on a set of training images of the object, a training process on the machine learning model to obtain a trained machine learning model, wherein the training process includes a first training process using a plurality of first test sample points followed by a second training process using a plurality of second test sample points located within a threshold distance from a surface region of the object; obtaining target view parameters indicating a view direction of the object; obtaining a plurality of rays associated with a target image of the object; obtaining render sample points on the plurality of rays associated with the target image; and rendering, by inputting the render sample points to the trained machine learning model, colors associated with the pixels of the target image.

Abstract: Systems, methods, and non-transitory computer-readable media are configured to obtain a set of content items to train a neural radiance field-based (NeRF-based) machine learning model for object recognition. Depth maps of objects depicted in the set of content items can be determined. A first set of training data comprising reconstructed content items depicting only the objects can be generated based on the depth maps. A second set of training data comprising one or more optimal training paths associated with the set of content items can be generated based on the depth maps. The one or more optimal training paths are generated based at least in part on a dissimilarity matrix associated with the set of content items. The NeRF-based machine learning model can be trained based on the first set of training data and the second set of training data.

Abstract: An image-based method of modeling and rendering a three-dimensional model of an object is provided. The method comprises: obtaining a three-dimensional point cloud at each frame of a synchronized, multi-view video of an object, wherein the video comprises a plurality of frames; extracting a feature descriptor for each point in the point cloud for the plurality of frames without storing the feature descriptor for each frame; producing a two-dimensional feature map for a target camera; and using an anti-aliased convolutional neural network to decode the feature map into an image and a foreground mask.

Abstract: A method of detecting and recognizing faces using a light field camera array is provided. The method includes capturing multi-view color images using the light field camera array; obtaining a depth map; conducting light field rendering using a weight function comprising a depth component and a sematic component, where the weight function assigns a ray in the light field with a weight; and detecting and recognizing a face.

Abstract: A method of image processing for occlusion removal in images and videos captured by light field camera systems. The method comprises: capturing a plurality of camera views using a plurality of cameras; capturing a plurality of depth maps using a plurality of depth sensors; generating a depth map for each camera view; calculating a target view on a focal plane corresponding to a virtual camera; set a weighting function on the pixels on the camera views based on the depth map and a virtual distance; and blending the pixels in accordance with the weighting function to generate a refocused target view.

Abstract: The present invention discloses a method for real-time rendering of giga-pixel images. Image data are subject to offline pre-processing, and then are subject to data decoding and redirection through a decoding module. A corresponding scheduling strategy is determined according to different image inputs, and rendering is executed by a renderer. The present invention may realize the real-time rendering of a giga-pixel panoramic view in a conventional display device, to greatly reduce the resource allocated for rendering of giga-pixel images. The present invention may render an image originally requiring a 40G or more video memory capacity on a common video card with a 1G-4G video memory capacity.

Abstract: The present invention discloses a method for real-time rendering of giga-pixel images. Image data are subject to offline pre-processing, and then are subject to data decoding and redirection through a decoding module. A corresponding scheduling strategy is determined according to different image inputs, and rendering is executed by a renderer. The present invention may realize the real-time rendering of a giga-pixel panoramic view in a conventional display device, to greatly reduce the resource allocated for rendering of giga-pixel images. The present invention may render an image originally requiring a 40G or more video memory capacity on a common video card with a 1G-4G video memory capacity.

Abstract: A method of processing light field images for separating a transmitted layer from a reflection layer. The method comprises capturing a plurality of views at a plurality of viewpoints with different polarization angles; obtaining an initial disparity estimation for a first view using SIFT-flow, and warping the first view to a reference view; optimizing an objective function comprising a transmitted layer and a secondary layer using an Augmented Lagrange Multiplier (ALM) with Alternating Direction Minimizing (ADM) strategy; updating the disparity estimation for the first view; repeating the steps of optimizing the objective function and updating the disparity estimation until the change in the objective function between two consecutive iterations is below a threshold; and separating the transmitted layer and the secondary layer using the disparity estimation for the first view.

Abstract: A method of image processing for occlusion removal in images and videos captured by light field camera systems. The method comprises: capturing a plurality of camera views using a plurality of cameras; capturing a plurality of depth maps using a plurality of depth sensors; generating a depth map for each camera view; calculating a target view on a focal plane corresponding to a virtual camera; set a weighting function on the pixels on the camera views based on the depth map and a virtual distance; and blending the pixels in accordance with the weighting function to generate a refocused target view.

Abstract: A method of detecting and recognizing faces using a light field camera array is provided. The method includes capturing multi-view color images using the light field camera array; obtaining a depth map; conducting light field rendering using a weight function comprising a depth component and a sematic component, where the weight function assigns a ray in the light field with a weight; and detecting and recognizing a face.