Abstract: Concepts for encoding and decoding multi-view data for immersive video are disclosed. In an encoding method, metadata is generated comprising a field indicating if a patch data unit of the multi-view data comprises in-painted data for representing missing data. The generated metadata provides a means of distinguishing patch data units comprising original texture and depth data from patch data units comprising in-painted data (e.g. in-painted texture and depth data). The provision of such information within the metadata of immersive video may address problems associated with blending and pruned view reconstruction. Also provided are an encoder and a decoder for multi-view data for immersive video, and a corresponding bitstream, comprising metadata.

Abstract: A first apparatus comprises a first receiver (301) receiving images of a scene captured and a second receiver (303) receives 3D spatial data for the scene A view synthesis neural network (307) generates view shifted images for the scene for different view poses from the images and the spatial data. A neural network trainer (309) trains the view synthesis neural network (307) based on images of the scene for different view poses. A generator (305) generates an audiovisual data stream comprising: image data for the images, scene data representing the three dimensional spatial data, and coefficient data describing coefficients of the view synthesis neural network (307) after training. A second apparatus receives the audio visual data stream and sets a local neural network (403) based on the coefficient data. The local neural network (403) is then used to generate images of the scene for different view poses.

Abstract: An apparatus generates a data signal comprising three dimensional image data providing a representation of a three dimensional scene. The three dimensional image data includes at least one image providing visual data for the scene. The data signal further comprises a view dependency indication for an image region of the image where the view dependency indication is indicative of a degree of variation of one or more visual properties for scene points of the image region as a function of viewing direction. A rendering apparatus comprises a receiver (201) receiving the data signal and a renderer (203) generates a view image of the scene from a view pose from the three dimensional image data in dependence on the view dependency indication. Specifically, blending of contributions from different points of the scene to a given pixel of the view image may be dependent on the view dependency indication.

Abstract: A method for transmitting multi-view image frame data. The method comprises obtaining multi-view components representative of a scene generated from a plurality of sensors, wherein each multi-view component corresponds to a sensor and wherein at least one of the multi-view components includes a depth component and at least one of the multi-view components does not include a depth component. A virtual sensor pose is obtained for each sensor in a virtual scene, wherein the virtual scene is a virtual representation of the scene and wherein the virtual sensor pose is a virtual representation of the pose of the sensor in the scene when generating the corresponding multi-view component. Sensor parameter metadata is generated for the multi-view components, wherein the sensor parameter metadata contains extrinsic parameters for the multi-view components and the extrinsic parameters contain at least the virtual sensor pose of a sensor for each of the corresponding multi-view components.

Abstract: The invention provides a method for generating multilayer images. The method comprises receiving three-dimensional, 3D, source data of a scene and generating a plurality of layers at different depths in a virtual 3D scene, wherein each layer corresponds to a particular layer depth value relative to an origin point. Image data is generated for each layer based on the 3D source data, wherein the image data comprises texture data and transparency data, and displacement maps are generated for the layers based on the 3D source data and the depth values of the layers. The displacement maps define the depth of different points on a corresponding layer relative to the layer depth, wherein the generation of the displacement maps is restricted such that a depth surface, as defined by any displacement map, does not intersect any other depth surface in the virtual 3D scene. A depth surface is defined as the mapping of a displacement map onto the corresponding layer.

Abstract: A first apparatus comprises a first receiver (301) receiving images of a scene captured and a second receiver (303) receives 3D spatial data for the scene A view synthesis neural network (307) generates view shifted images for the scene for different view poses from the images and the spatial data. A neural network trainer (309) trains the view synthesis neural network (307) based on images of the scene for different view poses. A generator (305) generates an audiovisual data stream comprising: image data for the images, scene data representing the three dimensional spatial data, and coefficient data describing coefficients of the view synthesis neural network (307) after training. A second apparatus receives the audio visual data stream and sets a local neural network (403) based on the coefficient data. The local neural network (403) is then used to generate images of the scene for different view poses.

Abstract: Generating an image signal comprises a receiver (401) receiving source images representing a scene. A combined image generator (403) generates combined images from the source images. Each combined image is derived from only parts of at least two images of the source images. An evaluator (405) determines prediction quality measures for elements of the source images where the prediction quality measure for an element of a first source image is indicative of a difference between pixel values in the first source image and predicted pixel values for pixels in the element. The predicted pixel values are pixel values resulting from prediction of pixels from the combined images. A determiner (407) determines segments of the source images comprising elements for which the prediction quality measure is indicative of a difference above a threshold. An image signal generator (409) generates an image signal comprising image data representing the combined images and the segments of the source images.

Abstract: Methods of encoding and decoding depth data are disclosed. In an encoding method, depth values and occupancy data are both encoded into a depth map. The method adapts how the depth values and occupancy data are converted to map values in the depth map. For example, it may adaptively select a threshold, above or below which all values represent unoccupied pixels. By adapting how the depth and occupancy are encoded, based on analysis of the depth values, the method can enable more effective encoding and transmission of the depth data and occupancy data. The encoding method outputs metadata defining the adaptive encoding. This metadata can be used by a corresponding decoding method, to decode the map values. Also provided are an encoder and a decoder for depth data, and a corresponding bitstream, comprising a depth map and its associated metadata.

Abstract: Generating an image signal comprises a receiver that receives source images representing a scene. A combined image generator generates combined images from the source images. Each combined image is derived from source images. An evaluator determines prediction quality measures for elements of the source images where the prediction quality measure for an element of a source image is indicative of a difference between pixel values in the source image and predicted pixel values for pixels in the element. The predicted pixel values are pixel values resulting from prediction of pixels from the combined images. A determiner determines low-quality segments of the source images comprising elements for which the prediction quality measure is indicative of a difference between the pixels that is above a threshold. An image signal generator generates an image signal comprising image data representing the combined images and the low-quality segments of the source images.

Abstract: An apparatus comprises a receiver (301) receiving an image signal representing a scene. The image signal includes image data comprising a number of images where each image comprises pixels that represent an image property of the scene along a ray having a ray direction from a ray origin. The ray origins are different positions for at least some pixels. The image signal further comprises a plurality of parameters describing a variation of the ray origins and/or the ray directions for pixels as a function of pixel image positions. A renderer (303) renders images from the number of images based on the plurality of parameters.

Abstract: Methods of encoding and decoding depth data are disclosed. In an encoding method, depth values and occupancy data are both encoded into a depth map. The method adapts how the depth values and occupancy data are converted to map values in the depth map. For example, it may adaptively select a threshold, above or below which all values represent unoccupied pixels. By adapting how the depth and occupancy are encoded, based on analysis of the depth values, the method can enable more effective encoding and transmission of the depth data and occupancy data. The encoding method outputs metadata defining the adaptive encoding. This metadata can be used by a corresponding decoding method, to decode the map values. Also provided are an encoder and a decoder for depth data, and a corresponding bitstream, comprising a depth map and its associated metadata.

Abstract: In the field of wireless communications, approaches for a reallocation and reservation of resources for high priority communications, for a Quality of Service, QoS, feedback, and for handling certain events in a wireless communication network are described. Embodiments relate to the implementation of such approaches for entities of a wireless communication network or system performing sidelink communications like V2X Mode 3 or Mode 4 UEs. In particular, improved approaches for a reallocation and reservation of resources for high priority communications, for a QoS feedback and for handling certain events in a wireless communication network are presented.

Abstract: A distribution system comprises an audio server (101) for receiving incoming audio from remote clients (103) and for transmitting audio derived from the incoming audio to the remote clients (103). An audio apparatus comprises an audio a receiver (401) which receives data comprising: audio data for a plurality of audio components representing audio from a remote client of the plurality of remote clients; and proximity data for at least one of the audio components. The proximity data is indicative of proximity between remote clients. A generator (403) of the apparatus generates an audio mix from the audio components in response to the proximity data. For example, an audio component indicated to be proximal to a remote client may be excluded from an audio mix for that remote client.

Abstract: A method for transmitting multi-view image frame data. The method comprises obtaining multi-view components representative of a scene generated from a plurality of sensors, wherein each multi-view component corresponds to a sensor and wherein at least one of the multi-view components includes a depth component and at least one of the multi-view components does not include a depth component. A virtual sensor pose is obtained for each sensor in a virtual scene, wherein the virtual scene is a virtual representation of the scene and wherein the virtual sensor pose is a virtual representation of the pose of the sensor in the scene when generating the corresponding multi-view component. Sensor parameter metadata is generated for the multi-view components, wherein the sensor parameter metadata contains extrinsic parameters for the multi-view components and the extrinsic parameters contain at least the virtual sensor pose of a sensor for each of the corresponding multi-view components.

Abstract: A method for calibrating at least one of the six-degrees-of-freedom of all or part of cameras in a formation positioned for scene capturing, the method comprising a step of initial calibration before the scene capturing. The step comprises creating a reference video frame which comprises a reference image of a stationary reference object. During scene capturing the method further comprises a step of further calibration wherein the position of the reference image of the stationary reference object within a captured scene video frame is compared to the position of the reference image of the stationary reference object within the reference video frame, and a step adapting the at least one of the six-degrees-of-freedom of a multiple cameras of the formation if needed in order to get an improved scene capturing after the further calibration.

Abstract: A method for transitioning from a first set of video tracks, VT1, to a second set of video tracks, VT2, when rendering a multi-track video, wherein each video track has a corresponding rendering priority. The method comprises receiving an instruction to transition from a first set of first video tracks VT1 to a second set of second video tracks VT2, obtaining the video tracks VT2 and, if the video tracks VT2 are different to the video tracks VT1, applying a lowering function to the rendering priority of one or more of the video tracks in the first set of video tracks VT1 and/or an increase function to the rendering priority of one or more video tracks in the second set of video tracks VT2. The lowering function and the increase function decrease and increase the rendering priority over time respectively. The rendering priority is used in the determination of the weighting of a video track and/or elements of a video track used to render a multi-track video.

Abstract: The invention provides a method for generating multilayer images. The method comprises receiving three-dimensional, 3D, source data of a scene and generating a plurality of layers at different depths in a virtual 3D scene, wherein each layer corresponds to a particular layer depth value relative to an origin point. Image data is generated for each layer based on the 3D source data, wherein the image data comprises texture data and transparency data, and displacement maps are generated for the layers based on the 3D source data and the depth values of the layers. The displacement maps define the depth of different points on a corresponding layer relative to the layer depth, wherein the generation of the displacement maps is restricted such that a depth surface, as defined by any displacement map, does not intersect any other depth surface in the virtual 3D scene. A depth surface is defined as the mapping of a displacement map onto the corresponding layer.

Abstract: A method for preparing immersive video data prior to processing into an immersive video. The method comprises receiving immersive video data containing one or more images of a scene, the scene comprising one or more image regions, and obtaining a relevance factor for at least one of the image regions, the relevance factor being indicative of the relative importance of the image region to a viewer. The immersive video data is separated into one or more sets of region data, wherein each set of region data corresponds to the data of one or more image regions and, based on the relevance factor of an image region, a bitrate is selected at which the set of region data corresponding to the image region is to be sent to an external processing unit.

Abstract: A method for transitioning from a first set of video tracks, VT1, to a second set of video tracks, VT2, when rendering a multi-track video, wherein each video track has a corresponding rendering priority. The method comprises receiving an instruction to transition from a first set of first video tracks VT1 to a second set of second video tracks VT2, obtaining the video tracks VT2 and, if the video tracks VT2 are different to the video tracks VT1, applying a lowering function to the rendering priority of one or more of the video tracks in the first set of video tracks VT1 and/or an increase function to the rendering priority of one or more video tracks in the second set of video tracks VT2. The lowering function and the increase function decrease and increase the rendering priority over time respectively. The rendering priority is used in the determination of the weighting of a video track and/or elements of a video track used to render a multi-track video.

Abstract: An image synthesis apparatus comprises a first receiver (201) receiving three dimensional image data describing at least part of a three dimensional scene and second receiver (203) receiving a view pose for a viewer. An image region circuit (207) determines at least a first image region in the three dimensional image data and a depth circuit (209) determines a depth indication for the first image region from depth data of the three dimensional image data. A region circuit (211) determines a first region for the first image region. A view synthesis circuit (205) generates a view image from the three dimensional image data where the view image representing a view of the three dimensional scene from the view pose. The view synthesis circuit (205) is arranged to adapt a transparency for the first image region in the view image in response to the depth indication and a distance between the view pose and the first region.