Abstract: Systems and methods relate to viewing distance based resolution and/or quality control for video coding. An illustrative system receives, such as at a server, a distance between a user and a device. The system includes selecting, based at least in part on the distance between the user and the device, a resolution and/or quality level for each layer of a plurality of layers of an image. The system transmits a version of each layer at the selected quality level to the device to cause the device to reconstruct the image based on the transmitted version of each layer and generate for display the reconstructed image.

Abstract: Systems and methods relate to segmenting object(s) of the first image data from a first viewpoint; generating respective segment texture data and segment depth data for each segmented object; arranging the segmented object(s) into a configuration based on the second viewpoint; compiling depth data corresponding to each pixel location in the configuration based on the second viewpoint; forming a synthesized depth map based at least in part on a closest depth value for each overlapping depth value at each pixel location of the compiled depth data; forming a synthesized texture map based on correspondence between segment depth data and respective segment texture data; and generating multiple focal planes (MFPs) based on the synthesized texture map and the synthesized depth map to enable generating for display second image data based on the MFPs.

Abstract: System and methods are provided for using multi-format representation and coding of visual information. The system accesses an image data that comprises a texture data and a depth map; decomposes the depth map into a plurality of component depth maps (CDMs); and generates multiple focal planes (MFPs) comprising a plurality of focal planes. Each respective focal plane is based on the texture data and a respective CDM of the plurality of CDMs. The system selects a data subset including one or more of: (a) the texture data, (b) the depth map, (c) the plurality of CDMs, or (d) the plurality of focal planes; generates encoded data based on the selected data subset; and transmits, over a communication network, the encoded data to a client device to cause the client device to: generate for display or for further processing an image based on the encoded data.

Abstract: Systems and methods are provided for using a Multi Focal Plane (MFP) prediction in predictive coding. The system detects a camera viewpoint change between a current frame from a current camera viewpoint to a previous frame from a previous camera viewpoint, decomposes a reconstructed previous frame to a plurality of focal planes, adjusts the plurality of focal planes from the previous camera viewpoint to correspond with the current camera viewpoint, generates an MFP prediction by summing pixel values of the adjusted plurality of focal planes along a plurality of optical axes from the current camera viewpoint, determines an MFP prediction error between the MFP prediction and the current frame, quantizes and codes the MFP prediction error, and transmits, to a receiver over a communication network, the camera viewpoint change and the coded quantized MFP prediction error for reconstruction of the current frame and display of the 3D scene.

Abstract: Systems and methods relate to viewing distance based resolution and/or quality control for video coding. An illustrative system receives, such as at a server, a distance between a user and a device. The system includes selecting, based at least in part on the distance between the user and the device, a resolution and/or quality level for each layer of a plurality of layers of an image. The system transmits a version of each layer at the selected quality level to the device to cause the device to reconstruct the image based on the transmitted version of each layer and generate for display the reconstructed image.

Abstract: Systems and methods relate to segmenting texture data and depth data of first image data of a first viewpoint into layers; generating respective multiple focal planes (MFPs) for each respective layer; blanking out pixels on the respective MFPs for each respective layer that are occluded by pixels on layers that are closer to a first viewpoint; and generating second image data for a second viewpoint to enable display of the second image data by: shifting and scaling the respective MFPs for each respective layer corresponding to the second viewpoint, wherein layers closer to an origin of the second viewpoint are shifted and scaled more than layers farther from the origin of the second viewpoint; and blanking out pixels on the shifted and scaled respective MFPs for each respective layer that are occluded by pixels on layers that are closer to the second viewpoint.

Abstract: Systems and methods for encoding/decoding a 3D image are provided. The system decomposes depth map into a plurality of component depth maps (CDMs) for a plurality of depth ranges, wherein each component depth map corresponds to a focal plane of a multiple focal plane (MFP) decomposition of the image data. The system generates a plurality of component depth map focal planes (CDMFPs) by combining each respective CDM with the depth map. The system scales data in each CDMFP by a respective scaling factor. The system generates for transmission a plurality of encoded scaled CDMFP data streams for the plurality of depth ranges, wherein each respective scaled CDMFP data stream is based at least in part on a respective scaled CDMFP.

Abstract: Systems and methods for rendering a 3D image are provided. The system receives image data including a texture matrix and a corresponding depth matrix. The system generates a first weight matrix based on a number of least significant bits of each depth value of the depth matrix. The system generating a second weight matrix that is a complement of the first weight matrix. The system generates the first folded focal plane matrix by applying weights of the first weight matrix to the texture matrix. The system generates the second folded focal plane matrix by applying weights of the second weight matrix to the texture matrix. The system provides the first and the second folded focal planes matrices to a display device for generating a 3D representation of the received texture matrix.

Abstract: Systems and methods relate to segmenting texture data and depth data of first image data of a first viewpoint into layers; generating respective multiple focal planes (MFPs) for each respective layer; blanking out pixels on the respective MFPs for each respective layer that are occluded by pixels on layers that are closer to a first viewpoint; and generating second image data for a second viewpoint to enable display of the second image data by: shifting and scaling the respective MFPs for each respective layer corresponding to the second viewpoint, wherein layers closer to an origin of the second viewpoint are shifted and scaled more than layers farther from the origin of the second viewpoint; and blanking out pixels on the shifted and scaled respective MFPs for each respective layer that are occluded by pixels on layers that are closer to the second viewpoint.

Abstract: Systems and methods relate to segmenting object(s) of the first image data from a first viewpoint; generating respective segment texture data and segment depth data for each segmented object; arranging the segmented object(s) into a configuration based on the second viewpoint; compiling depth data corresponding to each pixel location in the configuration based on the second viewpoint; forming a synthesized depth map based at least in part on a closest depth value for each overlapping depth value at each pixel location of the compiled depth data; forming a synthesized texture map based on correspondence between segment depth data and respective segment texture data; and generating multiple focal planes (MFPs) based on the synthesized texture map and the synthesized depth map to enable generating for display second image data based on the MFPs.

Abstract: Systems and methods for rendering a 3D image are provided. The system receives texture and depth data (depth map) for an image. The system generates, based on the image data, a plurality of folded focal planes matrices. For each respective pixel matrix, the system preprocesses pixel values in the respective focal pixel plane matrix to generate a respective preprocessed matrix, wherein the respective preprocessed matrix clusters together pixel values of the respective folded focal plane matrix based on the depth data for the image. The system generates phase functions based on a plurality of the preprocessed matrices. The system configures a spatial light modulator of the SLM device in accordance with the generated phase functions. The system then provides the plurality of the preprocessed matrices as input to the SLM device to generate for display a 3D representation of the received image data.

Abstract: Systems and methods for encoding/decoding a 3D image are provided. The system decomposes depth map into a plurality of component depth maps (CDMs) for a plurality of depth ranges, wherein each component depth map corresponds to a focal plane of a multiple focal plane (MFP) decomposition of the image data. The system generates a plurality of component depth map focal planes (CDMFPs) by combining each respective CDM with the depth map. The system scales data in each CDMFP by a respective scaling factor. The system generates for transmission a plurality of encoded scaled CDMFP data streams for the plurality of depth ranges, wherein each respective scaled CDMFP data stream is based at least in part on a respective scaled CDMFP.

Abstract: Systems and methods relate to viewing distance based resolution and/or quality control for video coding. An illustrative system receives, such as at a server, a distance between a user and a device. The system includes selecting, based at least in part on the distance between the user and the device, a resolution and/or quality level for each layer of a plurality of layers of an image. The system transmits a version of each layer at the selected quality level to the device to cause the device to reconstruct the image based on the transmitted version of each layer and generate for display the reconstructed image.

Abstract: Systems and methods are provided for using a Multiple Depth Plane (MDP) prediction in predictive coding. The system detects a camera viewpoint change between a current frame and a previous frame, decomposes a reconstructed depth map of the previous frame to a plurality of depth planes, adjusts the plurality of depth planes from a previous camera viewpoint to correspond with a current camera viewpoint, generates an MDP prediction by summing pixel values of the adjusted plurality of depth planes along a plurality of optical axes from the current camera viewpoint, determines an MDP prediction error between the MDP prediction and a depth map of the current frame, quantizes and codes the MDP prediction error, and transmits, to a receiver over a communication network, the camera viewpoint change and the coded quantized MDP prediction error for reconstruction of a depth map of the current frame.

Abstract: Systems and methods for reducing a number of focal planes used to display a three-dimensional object are disclosed herein. In an embodiment, data defining a three-dimensional image according to a first plurality of focal planes are received. Pixel luminance values from the first plurality of focal planes are mapped to a second plurality of focal planes comprising fewer focal planes than the first plurality of focal planes. Data is stored identifying initial focal distances of the mapped pixel luminance values in the first plurality of focal planes. The second plurality of focal planes are then displayed on a near eye device which uses the data identifying initial focal distances of the mapped pixel luminance values to adjust a wavelength of light produced by the second plurality of images to cause the pixels to appear at their original focal distances.

Abstract: System and methods are provided for using multi-format representation and coding of visual information. The system accesses an image data that comprises a texture data and a depth map; decomposes the depth map into a plurality of component depth maps (CDMs); and generates multiple focal planes (MFPs) comprising a plurality of focal planes. Each respective focal plane is based on the texture data and a respective CDM of the plurality of CDMs. The system selects a data subset including one or more of: (a) the texture data, (b) the depth map, (c) the plurality of CDMs, or (d) the plurality of focal planes; generates encoded data based on the selected data subset; and transmits, over a communication network, the encoded data to a client device to cause the client device to: generate for display or for further processing an image based on the encoded data.

Abstract: Systems and methods for encoding/decoding a 3D image are provided. The system accesses an image data that comprises a texture data and a depth map. The system decomposes the depth map into a plurality of component depth maps (CDMs) for a plurality of depth ranges, wherein each component depth map corresponds to a focal plane of multiple focal planes (MFPs) decomposition of the image data. The system generates a plurality of encoded CDM data streams for the plurality of depth ranges, wherein each respective CDM data stream is based at least in part on a respective CDM. The system then transmits the plurality of encoded CDM data streams to a client device to cause the client device to: (a) reconstruct the depth map, and (b) generate for display or for further processing an image based on the reconstructed depth map.

Abstract: Systems and methods for encoding/decoding a 3D image are provided. The system decomposes depth map into a plurality of component depth maps (CDMs) for a plurality of depth ranges, wherein each component depth map corresponds to a focal plane of a multiple focal plane (MFP) decomposition of the image data. The system generates a plurality of component depth map focal planes (CDMFPs) by combining each respective CDM with the depth map. The system scales data in each CDMFP by a respective scaling factor. The system generates for transmission a plurality of encoded scaled CDMFP data streams for the plurality of depth ranges, wherein each respective scaled CDMFP data stream is based at least in part on a respective scaled CDMFP.

Abstract: Systems and methods for rendering a 3D image are provided. The system receives texture and depth data (depth map) for an image. The system generates, based on the image data, a plurality of folded focal planes matrices. For each respective pixel matrix, the system preprocesses pixel values in the respective focal pixel plane matrix to generate a respective preprocessed matrix, wherein the respective preprocessed matrix clusters together pixel values of the respective folded focal plane matrix based on the depth data for the image. The system generates phase functions based on a plurality of the preprocessed matrices. The system configures a spatial light modulator of the SLM device in accordance with the generated phase functions. The system then provides the plurality of the preprocessed matrices as input to the SLM device to generate for display a 3D representation of the received image data.

Abstract: Systems and methods for rendering a 3D image are provided. The system receives image data including a texture matrix and a corresponding depth matrix. The system generates a first weight matrix based on a number of least significant bits of each depth value of the depth matrix. The system generating a second weight matrix that is a complement of the first weight matrix. The system generates the first folded focal plane matrix by applying weights of the first weight matrix to the texture matrix. The system generates the second folded focal plane matrix by applying weights of the second weight matrix to the texture matrix. The system provides the first and the second folded focal planes matrices to a display device for generating a 3D representation of the received texture matrix.