Abstract: An encoding method comprises obtaining (101) an input set of volumetric image data, selecting (103) data from the image data for multiple views based on a visibility of the data from a respective viewpoint at a respective viewing direction and/or within a respective field of view such that a plurality of the views comprises only a part of the image data, encoding (105) each of the views as a separate output set (31), and generating (107) metadata which indicates the viewpoints. A decoding method comprises determining (121) a desired user viewpoint, obtaining (123) the metadata, selecting (125) one or more of the available viewpoints based on the desired user viewpoint, obtaining (127) one or more sets of image data in which one or more available views corresponding to the selected one or more available viewpoints have been encoded, and decoding (129) at least one of the one or more available views.

Abstract: A rendering device is provided for rendering a scene, wherein the rendering device is configured to display a video object or allow visual pass-through of an external environment to a viewer and to render the scene as augmentation of the video object or the external environment. The scene may be rendered in a hybrid manner, involving local rendering of a part of the scene by the rendering device and remote rendering of another part of the scene by a remote rendering system. Both the rendering device and the remote rendering system may have access to scene descriptor data which may be indicative of changes in a state of the scene over time. As such, the rendering client may request remote rendering of a part of the scene at a particular temporal state. e.g., to anticipate for network and/or processing latencies.

Abstract: A method of processing multi-view video data by an server is described, wherein the method may comprise receiving one or more first multi-view parameters from a client device, the one or more multi-view parameters being indicative of a first data format associated with a multi-view display device of the client device; receiving a request for a segment sent by the client device based on a manifest file, the segment request comprising a segment resource locator identifying a first source segment comprising first multi-view video data, the segment resource locator comprising one or more second multi-view parameters indicating that the first multi-view video data are of a second data format; generating a segment of the first data format based on the one or more first multi-view parameters, the first multi-view video data and, optionally, further multi-view video data associated with one or more further source segments; and, transmitting the generated segment to the client device.

Abstract: Apparatus and method are provided for decoding an encoded video bitstream with multiple frames for displaying a video on a transparent display. The method includes: receiving the encoded video bitstream, in which the video bitstream includes image data including multiple image values representing an image to be displayed and transparency data representing an intended transparency level of at least a part of the image to be displayed. For at least one frame of the video bitstream: the method further includes decoding received encoded image data to obtain a decoded image data; determining whether a current frame includes transparency data; in case the current frame includes transparency data: adjusting the decoded image data to obtain a transparency-adjusted image data depending on received transparency data.

Abstract: Decoding coded pictures of a video includes receiving, by a decoder apparatus, a bitstream comprising an encoded representation of first pictures of a first resolution version of the video and decoding the coded first pictures into first pictures, the first pictures being associated with corresponding second pictures of a second resolution version of the video, the first pictures being formed by sampling the corresponding second pictures using a sampling lattice. Signalling information received in the bitstream informs the decoder apparatus about a resolution change. The method includes identifying at least one of the first pictures as a first base picture and identifying one or more auxiliary pictures, the first base picture being associated with a corresponding second picture, and, each of the one or more auxiliary pictures being formed based on sampling the second picture using a different sampling lattice of the plurality of sampling lattices.

Abstract: A method of signaling encapsulated data representing a primary video sequence associated with an auxiliary video sequence, the primary and auxiliary video sequences resulting from image projections methods applied on signals captured by sensors, includes: writing visual contents of the primary and auxiliary video sequences as encapsulated data; and writing an alignment status as encapsulated data, the alignment status indicating whether the visual contents of the primary and auxiliary video sequences are aligned.

Abstract: A method of processing video data of a picture is described, the method including: allocating memory for a decoded picture in a decoded picture buffer, DPB, the decoded picture comprising pixels representing video data; receiving a bitstream comprising decoding units, DUs, and storing the DUs in a coded picture buffer, CRB, the DUs representing a coded picture that needs to be decoded into the decoded picture, each of the DUs representing a coded block of pixels; determining if, during decoding of the coded picture, at least one partial output can be performed, the at least one partial output including copying the one or more decoded DUs from the DPB to a data sink, while one or more DUs of the coded picture are not yet decoded and removed the CPB, the one or more decoded DUs representing a part of the decoded picture; and, or performing the at least one partial output if the processor determines that the at least one partial output can be performed, the performing including marking the one or more decoded DU

Abstract: A method for performing augmented reality (AR) content rendering, includes: obtaining virtual relighting effects based on virtual lighting information representative of at least one virtual light source; obtaining hybrid relighting effects based on the virtual lighting information and based on physical lighting information representative of at least one physical light source; and generating an AR video picture by aggregating a real-world scene relighted based on the virtual relighting effects and at least one virtual object relighted based on the hybrid relighting effects; and rendering the AR video picture.

Abstract: Methods and apparatus for processing an encoded representation of a sequence of pictures are disclosed, wherein the method includes: allocating memory for a current picture of the sequence of pictures in a decoded picture buffer, DPB; receiving a bitstream comprising an encoded representation of the current picture, and storing the encoded representation as a plurality of decoding units DUs, in a coded picture buffer, CPB, each of the plurality of DUs comprising an encoded set of pixels, wherein no coding dependency exists between coded pixels of different DUs of the current picture; receiving prioritization information associated with each of the plurality of DUs stored in the CPB, the prioritization information defining a decoding order in which the DUs of the current picture are being decoded; decoding one or more of said plurality of DU's into one or more decoded DU's, in accordance with said decoding order, each decoded DU comprising a set of pixels, each set of pixels being associated with a different a

Abstract: Methods and apparatus for processing an encoded representation of a sequence of pictures are disclosed, wherein the method includes: a processor of a decoding apparatus allocating memory for one or more pictures in a decoded picture buffer, DPB; the processor receiving a bitstream comprising an encoded representation of a current picture, the encoded representation of the current picture comprising a plurality of decoding units, DUs, each of the plurality of DUs comprising an encoded set of pixels; the processor storing the plurality of DUs of the current picture in a coded picture buffer, CPB; the processor decoding the plurality of DUs into decoded DUs of the current picture and storing the decoded DUs in the allocated memory of the DPB, the processor executing at least one partial output during the decoding of the plurality of DUs of the current picture, the performing of the at least one partial output including: copying one or more of the decoded DUs from the DPB to a data sink before all of the pluralit

Abstract: A method is implemented by an augmented reality (AR) device for performing AR content rendering. The method includes: transmitting, to a processing device external to the AR device, encoded source video picture data representative of a video picture of a real-world scene; receiving lighting information from the processing device; generating an AR video picture by: obtaining an aggregated video picture by aggregating the video picture and volumetric information representative of at least one virtual object; and incorporating, into the aggregated video picture, relighting effects associated with the at least one virtual object based on the received lighting information; and rendering the AR video picture.

Abstract: A method of processing a multi-view video by a client apparatus is described wherein, the multi-view video comprising a set of video streams is created by a set of cameras simultaneously capturing an object in a scene for a set of view angles and for one or more video resolutions, each video stream defining a view associated with a view angle of a camera relative to the object.

Abstract: One aspect of this disclosure relates a video processing device comprising a processor for processing a manifest file for video streaming for a user. The manifest file comprises at least a plurality of positions defined for a scene that are associated with pre-rendered omnidirectional or volumetric video segments stored on a server system. The manifest file may also contain a plurality of resource locators for retrieving omnidirectional or volumetric video segments from the server system. Each resource locator may be associated with a position defined for the scene. The video processing device may be configured to associate a position of the user with a first position for the scene in the manifest file to retrieve a first omnidirectional or volumetric video segment associated with the first position using a first resource locator from the manifest file.

Abstract: A method for encoding a video picture into a bitstream of encoded video picture data is provided. The video picture includes at least one component of samples representative of pixel values of a screen. The method includes: obtaining, from a transparency map, a modified video picture by assigning specific values (SV) to samples of one component (C) associated with transparent pixels, wherein the specific values (SV) are different of sample values of the component (C) associated with non-transparent pixels; and encoding the modified video picture.

Abstract: A method is described for forming output video frames by a decoder device, comprising: receiving a bitstream comprising bitstream parts representing encoded media data of video tiles and tile positioning information associated with the encoded video tiles; extracting tile positioning information from the bitstream, the tile positioning information comprising tile identifiers for identifying encoded video tiles, a tile identifier being associated with one or more boundary identifiers, each identifying a boundary of a video tile identified by a tile identifier; determining a tile map based on the boundary identifiers, the tile map representing a spatial layout of video tiles in an output video frame; and forming an output video frame, including identifying parts of the bitstream representing encoded media data of video tiles identified in the tile map, decoding the identified encoded media data and copying the decoded media data in the output video frame buffer according to spatial layout.

Abstract: A method of encapsulating point cloud data in encapsulating containers, includes writing point cloud data in at least one of the encapsulating containers; and for at least one spatial region of a spatial sensing coverage of a sensing device, writing first data on a presence of a sensing reliability score in one of the encapsulating containers; and if the first data indicates that a sensing reliability score is present in one encapsulating container, writing the sensing reliability score in one of the encapsulating containers.

Abstract: A 3D (three-dimensional) scene presenting method and associated devices use a shared reference document and a corresponding set of 3D update documents. When a presentation engine is triggered to access or present a target 3D scene among a set of target 3D scenes, it parses the shared reference document and the corresponding 3D scene update document among the set of 3D scene update documents to generate the target 3D scene description document describing the to-be accessed or presented 3D scene, renders the generated description document and updates the presenting 3D scene as the target scene.

Abstract: A rendering device is provided for rendering a scene, wherein the rendering device is configured to display a video object or allow visual pass-through of an external environment to a viewer and to render the scene as augmentation of the video object or the external environment. The scene may be rendered in a hybrid manner, involving local rendering of a part of the scene by the rendering device and remote rendering of another part of the scene by a remote rendering system. Both the rendering device and the remote rendering system may have access to scene descriptor data which may be indicative of changes in a state of the scene over time. As such, the rendering client may request remote rendering of a part of the scene at a particular temporal state. e.g., to anticipate for network and/or processing latencies.

Abstract: A method is described for forming output video frames by a decoder device, comprising: receiving a bitstream comprising bitstream parts representing encoded media data of video tiles and tile positioning information associated with the encoded video tiles; extracting tile positioning information from the bitstream, the tile positioning information comprising tile identifiers for identifying encoded video tiles, a tile identifier being associated with one or more boundary identifiers, each identifying a boundary of a video tile identified by a tile identifier; determining a tile map based on the boundary identifiers, the tile map representing a spatial layout of video tiles in an output video frame; and forming an output video frame, including identifying parts of the bitstream representing encoded media data of video tiles identified in the tile map, decoding the identified encoded media data and copying the decoded media data in the output video frame buffer according to spatial layout.

Abstract: A method is described for forming output video frames by a decoder device, comprising: receiving a bitstream comprising bitstream parts representing encoded media data of video tiles and tile positioning information associated with the encoded video tiles; extracting tile positioning information from the bitstream, the tile positioning information comprising tile identifiers for identifying encoded video tiles, a tile identifier being associated with one or more boundary identifiers, each identifying a boundary of a video tile identified by a tile identifier; determining a tile map based on the boundary identifiers, the tile map representing a spatial layout of video tiles in an output video frame; and forming an output video frame, including identifying parts of the bitstream representing encoded media data of video tiles identified in the tile map, decoding the identified encoded media data and copying the decoded media data in the output video frame buffer according to spatial layout.