Abstract: Methods, devices and data streams are proposed to change the transparency color on an image basis and transmit this information within the image. It is based on the fact that the useful information of a rectangle image with transparency is located in the center of the rectangle and on that a part of the fringe of the image, typically one of the four corners, is always transparent and can be used to transmit the transparency color for each image. The transparency color is coded on two components and the third component is used to transmit the transparency information. Possible color spaces for the representation of the image which separate luminance and chrominance information, as CEI XYZ or CEI xyY color spaces.

Abstract: Methods and devices for encoding and decoding a sequence of dense point clouds are disclosed. The encoding consists in obtaining a temporal prediction of the point cloud, compressing it and decompressing it. According to the present principles, the rendering attributes are mapped on the temporal prediction on one side and on the decompressed point cloud on the other side. The first one is subtracted from the second one to obtain a residue that is encoded aside with the compressed geometry. At the decoding, the inverse steps are performed.

Abstract: To encode and decode a point cloud, the point cloud is represented by an octree of nodes comprising a spatial location and a list of attribute values like color, reflectance, normal vectors, etc. An early termination decision workflow to disable the transform domain prediction is disclosed. According to the present principles, the distance to the tree root, that is the level of the node in the octree, is used as a parameter to predict when it is necessary to execute the intra frame prediction and, outlier nodes are removed before computing the prediction. In this embodiment, the memory byte consumption is reduced by four times the number of points.

Abstract: Methods, devices and data streams are proposed to encode, transport and decode 3D volumetric videos. The embodiments encompass the signaling of non-Lambertian patches together with their light reflection properties, so as to enable a ray-tracing based rendering engine to synthesize visually realistic virtual views with respect to light effects. A retro-compatible solution enabling conventional rendering engines to render such described 3D scene but without the advanced light effects is also described.

Abstract: Methods, device and data stream are provided to generate, transmit and decode volumetric scenes. According the present principles, rendering attribute values, like color values, are transmitted to the renderer to be used as a suggested attribute value for pixels for which this attribute is missing. For example, when rendering a viewport image of a volumetric scene, some pixels may miss a color information. In such a case, the present principles provide a suggested default color. These principles apply to various rendering attributes like reflectance, transparency, material or shininess. In an embodiment, an attribute value is associated with a region of the 3D space. In another embodiment, a quality level linked to the default value helps the renderer to select a filling method.

Abstract: Methods, device and data stream are provided to encode unaligned multi-view plus depth images associated with assistance metadata for generating depth maps aligned with the unaligned color views. At the encoding stage, for an unaligned color view, the contribution of each depth map is evaluated, and the objects represented are mapped. Picture patches are cut to be homogeneous in terms of objects and contributing views. Metadata describing this homogeneous cut are encoded together with the atlas. At the decoding stage, to generate a depth map aligned with the unaligned color view, only the depth maps referenced in the assistance metadata are warped.

Abstract: Methods, devices and streams are disclosed for encoding a depth atlas representative of the geometry of a volumetric video. Views to be encoded are analyzed to detect regions of the views with simple depth or color, that is regions for which the depth or color has a local variance lower than a given threshold. Resolution of such regions is reduced and the atlas comprises first regions in full resolution and downscaled second regions. Metadata indicating whether a patch is a downscaled region and, if so the downscaling factor, are associated with the atlas in the data stream. The decoder uses these metadata to compose the view from different patches.

Abstract: Methods, device and data stream are provided to encode unaligned multi-view plus depth images associated with assistance metadata for generating depth maps aligned with the unaligned color views. At the encoding stage, for an unaligned color view, the contribution of each depth map is evaluated. A subset of the most contributing depth maps is selected and metadata representative of this subset is encoded in the data stream of the unaligned multi-view plus depth images in association with the unaligned color view. At the decoding stage, to generate a depth map unaligned with the unaligned color view, only the depth maps referenced in the assistance metadata are warped.

Abstract: Systems and methods are provided in which a plurality of source views of a scene are encoded as an immersive video comprising a plurality of patches. The patches are segmented based on segmentation information from at least one of the source views. The segmentation information associates each of a plurality of regions of the source view with a respective entity. Information is also encoded that indicates which of the source views are associated with the segmentation information used to segment the patches. This information may be used to provide improved rendering, particularly where rendering is performed of less than all entities in the scene.

Abstract: Methods and apparatus for encoding and decoding a volumetric scene are disclosed. A set of attribute and geometry patches is obtained by projecting samples of the volumetric scene onto the patches according to projection parameters. If the geometry patch is comparable to a planar layer located at a constant depth according to the projection parameters, only the attribute patch is packed in an attribute atlas image and the depth value is encoded in metadata. Otherwise, both attribute and geometry patches are packed in an atlas. At the decoding, if metadata for an attribute patch indicates that its geometry may be determined from the projection parameters and a constant depth, the attributes are inverse projected on a planar layer. Otherwise, attributes are inverse projected according to the associated geometry patch.

Abstract: Methods and devices for encoding and decoding 3D scenes of an immersive video are disclosed with the related data stream. Taking advantage of the fact that parts of the 3D scene does not change from one frame of the video to another, even when the cameras move and/or the lighting condition change, dynamic and static parts of the 3D scene are encoded in different atlases. In an embodiment, static patch atlases are repeated between two updates. In another embodiment, only updates are encoded and transmitted. The decoder maintains a memory associating entity of the 3D scene with a set of static patches that is updated with instructions linked to update static patch atlases. At each presentation time the static part of the 3D scene to decode is obtained from the memory.

Abstract: A method and an apparatus for reconstructing at least one sequence of volumetric frames from a data stream are provided wherein the sequence of volumetric frames is temporally resampled and/or temporally aligned to a composition time frame using 3D motion information obtained for the volumetric frames. The method comprises decoding from the data stream the at least one sequence of volumetric frames, obtaining a 3D motion information representative of a displacement in a 3D space of points of volumetric frames of the at least one sequence, and displacing points of at least one volumetric frame of the sequence to a composition time frame using the 3D motion information.

Abstract: Methods, apparatus and data stream are described to encode, transmit and decode an atlas-based representation of a 3D scene based on a multiplane image (MPI) representation in which a depth component is encoded in each layer. Layers of the MPI are clustered on a transparency basis to generate texture, transparency and depth patch pictures. Patch pictures are packed in at least one atlas image. Metadata associating each patch to a layer and each layer to a depth and a depth quantization law are encoded in the data stream with the at least one atlas. At the decoding side, the MPI with a depth component is retrieved from the data stream and is used to render a viewport image from a viewpoint in the neighborhood of the center of the MPI.

Abstract: Methods, device and data stream are provided to encode unaligned multi-view plus depth images associated with assistance metadata for generating depth maps aligned with the unaligned color views. At the encoding stage, for an unaligned color view, the contribution of each depth map is evaluated. A subset of the most contributing depth maps is selected and metadata representative of this subset is encoded in the data stream of the unaligned multi-view plus depth images in association with the unaligned color view. At the decoding stage, to generate a depth map unaligned with the unaligned color view, only the depth maps referenced in the assistance metadata are warped.

Abstract: Methods and apparatus for encoding and decoding a volumetric scene are disclosed. A set of attribute and geometry patches is obtained by projecting samples of the volumetric scene onto the patches according to projection parameters. If the geometry patch is comparable to a planar layer located at a constant depth according to the projection parameters, only the attribute patch is packed in an attribute atlas image and the depth value is encoded in metadata. Otherwise, both attribute and geometry patches are packed in an atlas. At the decoding, if metadata for an attribute patch indicates that its geometry may be determined from the projection parameters and a constant depth, the attributes are inverse projected on a planar layer. Otherwise, attributes are inverse projected according to the associated geometry patch.

Abstract: Methods and devices are provided to encode and decode a data stream carrying data representative of a three-dimensional scene, the data stream comprising color pictures packed in a color image; depth pictures packed in a depth image; and a set of patch data items comprising de-projection data; data for retrieving a color picture in the color image and geometry data. Two types of geometry data are possible. The first type of data describes how to retrieve a depth picture in the depth image. The second type of data comprises an identifier of a parametric function and a list of parameter values for the identified parametric function.

Abstract: A sequence of three-dimension scenes is encoded as a video by an encoder and transmitted to a decoder which retrieves the sequence of 3D scenes. Points of a 3D scene visible from a determined point of view are encoded as a color image in a first track of the stream in order to be decodable independently from other tracks of the stream. The color image is compatible with a three degrees of freedom rendering. Depth information and depth and color of residual points of the scene are encoded in separate tracks of the stream and are decoded only in case the decoder is configured to decode the scene for a volumetric rendering.

Abstract: Methods and devices are provided to encode and decode a data stream carrying data representative of a three-dimensional scene, the data stream comprising color pictures packed in a color image; depth pictures packed in a depth image; and a set of patch data items comprising de-projection data; data for retrieving a color picture in the color image and geometry data. Two types of geometry data are possible. The first type of data describes how to retrieve a depth picture in the depth image. The second type of data comprises an identifier of a parametric function and a list of parameter values for the identified parametric function.

Abstract: Methods and devices for encoding, decoding and transmitting a three-dimensional scene initially represented as a multiplane image (MPI) are provided. Each layer of the MPI is split into patches based on the transparency component. Patches of a layer are grouped in a tile. The greater the depth of the layer, the greater the identifying number of the tile. When several tiles are packed in an atlas image, the same monotonic (i.e. ascending or descending) function according to depth applies to atlas numbers. At the decoding side, the current viewport to render is initially cleared and each decoded tile is sequentially blended over from the nearest one to the furthest due to the numbering of the set of atlases and tiles. Pixels of a patch under rendering are projected onto pixels of the viewport image according to the depth of the tile comprising the patch and metadata indicating the position of the patch in the layer of the MPI the patch has been clustered from.

Abstract: Methods, devices and stream are disclosed to encode and decode a scene (such as a point cloud) in the context of a patch-based transmission of a volumetric video content. Attributes of points of the scene are projected onto patches. Every point has a geometry attribute. For other attributes, like transparency of displacement attribute, some points may have no value. According to the present principles, each attribute is encoded in a different atlas with its own layout. This allow to save pixel rate in memory of the renderer.