Abstract: Methods and devices for generating a multiplane image from a representation of a 3D scene are disclosed. Given a representation of the 3D scene, for example as a multiview image, a first MPI is generated by a convolutional neural network trained for generating a MPI according to a provided sampling of the depths of the 3D scene. The presence of objects in different layers, that is the complementary of the accumulated transparency of layers, is analyzed to resample the sampling of layers' depth. The new sampling is used by the CNN to generate a new MPI from a representation of the same 3D scene.

Abstract: Methods and apparatus for encoding and decoding a volumetric scene are disclosed. A set of attribute and geometry patches is obtained by projecting samples of the volumetric scene onto the patches according to projection parameters. If the geometry patch is comparable to a planar layer located at a constant depth according to the projection parameters, only the attribute patch is packed in an attribute atlas image and the depth value is encoded in metadata. Otherwise, both attribute and geometry patches are packed in an atlas. At the decoding, if metadata for an attribute patch indicates that its geometry may be determined from the projection parameters and a constant depth, the attributes are inverse projected on a planar layer. Otherwise, attributes are inverse projected according to the associated geometry patch.

Abstract: A method and a device for generating a data stream representative of a volumetric video generated from a sequence of multi-plane images are disclosed. A quantization law depending on an angular resolution used to generate patches from the multi-plane images is set up. For each intra-periods of the frame sequence, patches are generated for a given angular resolution. If generated patches do not fit in atlas images of a target size, the angular resolution is decreased and patches are generated again according to this new angular resolution.

Abstract: To represent a 3D scene, the MPI format uses a set of fronto-parallel planes. Different from MPI, the current MIV standard accepts a 3D scene represented as sequence input pairs of texture and depth pictures as input. To enable transmission of an MPI cube via the MIV-V3C standard, in one embodiment, an MPI cube is divided into empty regions and local MPI partitions that contain 3D objects. Each partition in the MPI cube can be projected to one or more patches. For a patch, the geometry is generated as well as the texture attribute and alpha attributes, and the alpha attributes may be represented as a peak and a width of an impulse. In another embodiment, an MPI RGBA layer of the MPI is cut into sub-images. Each sub-image may correspond to a patch, and the RGB and alpha information of the sub-image are assigned to the patch.

Abstract: Methods and devices are provided to encode and decode a data stream carrying data representative of a three-dimensional scene, the data stream comprising color pictures packed in a color image; depth pictures packed in a depth image; and a set of patch data items comprising de-projection data; data for retrieving a color picture in the color image and geometry data. Two types of geometry data are possible. The first type of data describes how to retrieve a depth picture in the depth image. The second type of data comprises an identifier of a parametric function and a list of parameter values for the identified parametric function.

Abstract: Methods and devices are provided to encode points of a sphere onto an image by projecting them as a function of an isotropic region guided mapping operator and/or to decode pixels of an image onto a sphere by de-projected them according to the same parametrized mapping operator. A region of interest is determined on the sphere and the angular resolution of pixels of the image is modulated as a function of their distance to the reference pixel corresponding to the center of the region of interest. Chosen angular resolutions and size of the region of interest are associated with the image and encoded in a stream.

Abstract: A sequence of three-dimension scenes is encoded as a video by an encoder and transmitted to a decoder which retrieves the sequence of 3D scenes. Points of a 3D scene visible from a determined point of view are encoded as a color image in a first track of the stream in order to be decodable independently from other tracks of the stream. The color image is compatible with a three degrees of freedom rendering. Depth information and depth and color of residual points of the scene are encoded in separate tracks of the stream and are decoded only in case the decoder is configured to decode the scene for a volumetric rendering.

Abstract: Methods and devices are provided to encode and decode a data stream carrying data representative of a three-dimensional scene, the data stream comprising color pictures packed in a color image; depth pictures packed in a depth image; and a set of patch data items comprising de-projection data; data for retrieving a color picture in the color image and geometry data. Two types of geometry data are possible. The first type of data describes how to retrieve a depth picture in the depth image. The second type of data comprises an identifier of a parametric function and a list of parameter values for the identified parametric function.

Abstract: Methods and devices for encoding, decoding and transmitting a three-dimensional scene initially represented as a multiplane image (MPI) are provided. Each layer of the MPI is split into patches based on the transparency component. Patches of a layer are grouped in a tile. The greater the depth of the layer, the greater the identifying number of the tile. When several tiles are packed in an atlas image, the same monotonic (i.e. ascending or descending) function according to depth applies to atlas numbers. At the decoding side, the current viewport to render is initially cleared and each decoded tile is sequentially blended over from the nearest one to the furthest due to the numbering of the set of atlases and tiles. Pixels of a patch under rendering are projected onto pixels of the viewport image according to the depth of the tile comprising the patch and metadata indicating the position of the patch in the layer of the MPI the patch has been clustered from.

Abstract: Methods and devices for encoding and decoding data representative of a 3D scene are disclosed. A set of first patches is generated from a first MVD content acquired from a first region of the 3D scene. A patch is a part of one of the views of the MVD content. A set of second patches is generated from a second MVD content acquired from a second region of the 3D scene. An atlas packing first and second patches is generated and associated with metadata indicating, for a patch of the atlas, whether the patch is a first or a second patch At the decoding side, first patches are used for rendering the viewport image and second patches are used for pre-processing or post-processing the viewport image.

Abstract: Methods, device and data stream format are disclosed in the present document for the encoding, the formatting and the decoding of depth information representative of a 3D scene. Compression and decompression of quantized values by a video codec leads to a value error. This error on values is particularly sensitive for depth encoding. The present invention proposes to encode and decode depth with a quantization function that minimize an angle error when a value error on quantized depth creates a location delta between the projected point and the de-projected point. The inverse of such a quantization function has to be encoded in metadata associated with the 3D scene, for example as a LUT, to be retrieved at the decoding, as such functions are not tractable.

Abstract: Methods and devices for encoding and decoding a data stream representative of a 3D volumetric scene comprising haptic features associated with objects of the 3D scene are disclosed. At the encoding, haptic features are associated with objects of the scene, for instance as haptic maps. Haptic components are stored in points of the 3D scene as color may be. These components are projected onto patch pictures which are packed in atlas images. At the decoding, haptic components are un-projected onto reconstructed points as color may be according to the depth component of pixels of the decoded atlases.

Abstract: Methods and devices for encoding and decoding data representative of a 3D scene are disclosed. A set of first patches is generated from a first MVD content acquired from a first region of the 3D scene. A patch is a part of one of the views of the MVD content. A set of second patches is generated from a second MVD content acquired from a second region of the 3D scene. An atlas packing first and second patches is generated and associated with metadata indicating, for a patch of the atlas, whether the patch is a first or a second patch At the decoding side, first patches are used for rendering the viewport image and second patches are used for pre-processing or post-processing the viewport image.

Abstract: At least one embodiment relates to a method and apparatus for encoding a volumetric video representing a scene, said encoding being based on patches representing the color and depth of a 2D projection of subparts of the scene, wherein a first patch is packed in a second patch for a given time interval lower than or equal to a time period along which the second patch is defined when said first patch can be packed in said second patch over said time interval. Decoding method and apparatus are also provided.

Abstract: Methods and devices are provided for encoding, transmitting and decoding 3DoF+ volumetric video. At the encoding stage one input view (among all the input ones) is selected to convey the viewport dependent light effect and its id is transmitted to the decoder as an extra metadata. On the decoder side, when patches coming from this selected view are available for the rendering of the viewport, they are preferentially used regarding the other candidates whatever the view to synthesize position.

Abstract: A sequence of three-dimension scenes is encoded as a video by an encoder and transmitted to a decoder which retrieves the sequence of 3D scenes. Points of a 3D scene visible from a determined point of view are encoded as a color image in a first track of the stream in order to be decodable independently from other tracks of the stream. The color image is compatible with a three degrees of freedom rendering. Depth information and depth and color of residual points of the scene are encoded in separate tracks of the stream and are decoded only in case the decoder is configured to decode the scene for a volumetric rendering.

Abstract: A patch-based atlas format in intra-periods of varying length is used to encode a volumetric video. A first atlas layout is built for a first sequence of 3D scenes. The number of 3D scenes in the sequence is chosen to fit the size of a Group of Pictures of the codec. A second sequence is iteratively set up by appending the next 3D scene of the sequence to encode while the number of patches of the layout built for this iterative second sequence is lower than or equal to the number of patches of the first layout. When iterations end, one of the layouts is selected to generate every atlas of the group. In such a way, size of metadata is decreased, and compression is enhanced.

Abstract: Methods, devices and data streams are proposed to encode, transport and decode 3D volumetric videos. The embodiments encompass the signaling of non-Lambertian patches together with their light reflection properties, so as to enable a ray-tracing based rendering engine to synthesize visually realistic virtual views with respect to light effects. A retro-compatible solution enabling conventional rendering engines to render such described 3D scene but without the advanced light effects is also described.

Abstract: Methods and devices for encoding and decoding 3D scenes of an immersive video are disclosed with the related data stream. Taking advantage of the fact that parts of the 3D scene does not change from one frame of the video to another, even when the cameras move and/or the lighting condition change, dynamic and static parts of the 3D scene are encoded in different atlases. In an embodiment, static patch atlases are repeated between two updates. In another embodiment, only updates are encoded and transmitted. The decoder maintains a memory associating entity of the 3D scene with a set of static patches that is updated with instructions linked to update static patch atlases. At each presentation time the static part of the 3D scene to decode is obtained from the memory.

Abstract: Methods and devices for generating a multiplane image from a representation of a 3D scene are disclosed. Given a representation of the 3D scene, for example as a multiview image, a first MPI is generated by a convolutional neural network trained for generating a MPI according to a provided sampling of the depths of the 3D scene. The presence of objects in different layers, that is the complementary of the accumulated transparency of layers, is analyzed to resample the sampling of layers' depth. The new sampling is used by the CNN to generate a new MPI from a representation of the same 3D scene.