Abstract: Processing of a compressed representation of a video signal is optimized for multiple tasks, such as object detection, viewing of displayed video, or other machine tasks. In one embodiment, multiple analysis stages and a single synthesis is performed as part of a coding/decoding operation with training of an encoder side analysis and, optionally, a corresponding machine task. In another embodiment, multiple synthesis operations are performed on the decoding side, so that respective analysis, synthesis, and task stages are optimized. Other embodiments comprise feeding decoded feature maps to tasks, predictive coding, and using hyperprior-based models.

Abstract: A processing module, or connector, adapts an output of a codec, or a decoded output, to a form suitable for an alternate task. In one embodiment, the output of a codec is used for a machine task and the connector adapts this output to a form suitable for a video display. In another embodiment, metadata accompanies the codec output, which can instruct the connector how to adapt the codec output for an alternate task. In other embodiments, the processing module performs averaging over a N×M window, or convolution.

Abstract: A method and an apparatus for decoding at least one part of at least one image is disclosed. The method comprises decoding at least one update parameter and modifying a deep neural network-based decoder based on said decoded update parameter. The method further comprises decoding at least one part of at least one image using at least said modified decoder.

Abstract: At least a method and an apparatus are presented for efficiently encoding or decoding video. For example, an intra prediction of an image block and side information are determined using at least one neural network from a context comprising pixels surrounding the image block. The side information allows a decoder to determine the intra prediction and is signaled for the decoding.

Abstract: Improved neural-network-based image and video coding techniques are presented, including hybrid techniques that include both tools of a host codec and neural-network-based tools. In these improved techniques, the host coding tools may include conventional video coding standards such H.266 (VVC). In an aspects, source frames may be partitioned and either host or neural-network-based tools may be selected per partition. Coding parameter decisions for a partition may be constrained based on the partitioning and coding tool selection. Rate control for host and neural network tools may be combined. Multi-stage processing of neural network output may use a checkerboard prediction pattern.

Abstract: The disclosure relates, according to a first aspect, to a method for compressing data including encoding at least one information representative of a use, during the compression, of a compressed sparse format. The disclosure relates, according to a second aspect, to a method for decompressing input data comprising obtaining information representative of zero or non-zero values in at least a part of the input data, and using only the non-zero values of the zero or non-zero values for a further processing of the part of the input data, based on the representative information. Corresponding devices, system, non-transitory program product, computer storage medium and signal are also disclosed.

Abstract: In one implementation, a picture is partitioned into multiple blocks, with uniform or different block sizes. Each block is compressed by an auto-encoder, which may comprise a deep neural network and entropy encoder. The compressed block may be reconstructed or decoded with another deep neural network. Quantization may be used in the encoder side, and de-quantization at the decoder side. When the block is encoded, neighboring blocks may be used as causal information. Latent information can also be used as input to a layer at the encoder or decoder. Vertical and horizontal position information can further be used to encode and decode the image block. A secondary network can be applied to the position information before it is used as input to a layer of the neural network at the encoder or decoder. To reduce blocking artifact, the block may be extended before being input to the encoder.

Abstract: At least a method and an apparatus are presented for efficiently encoding or decoding video. For example, a plurality of frames is provided to a motion estimator to produce an output comprising estimated motion information. The estimated motion information is provided to an auto-encoder or an auto-decoder to produce an output comprising reconstructed motion field. The reconstructed motion field and one or more decoded frames of the plurality of frames are provided to a deep neural network to produce an output comprising refined bi-directional motion field. The video is encoded or decoded based on the refined bi-directional motion field.

Abstract: Different implementations are described, particularly implementations for video encoding and decoding based on linear weighted intra prediction, also called matrix based intra prediction, are presented. Accordingly, for a block being encoded or decoded in linear weighted intra prediction, obtaining intra predicted samples from at least two matrix-vector products between at least two selected weight matrices of reduced size and a set of neighboring reference samples. Advantageously, such arrangement allows to reduce the amount of memory for storing data and to reduce the complexity of the intra prediction samples computation.

Abstract: For a picture with two or more color components, the prediction residuals for the first color component of a block to be encoded may be transformed with a first transform. The transform coefficients for the first color component may go through quantization, de-quantization and inverse transform to obtain reconstructed prediction residuals. Based on the reconstructed prediction residuals for the first color component, the phases of the transform basis function of the first transform can be adjusted to improve the sparsity of the transformed signal. The prediction residuals for the remaining color components may then be transformed with the adjusted transform. In order to determine the phase shift factor, the reconstructed prediction residuals for the first color component may be transformed with the first transform, adjusted by different candidate phase shift factors, and the candidate phase shift factor that provides a smallest sparsity measure can be selected for the block.

Abstract: A content stream comprising video and synchronized illumination data is based on a reference lighting setup from, for example, the site of the content creation. The content stream is received at a user location where the illumination data controls user lighting that is synchronized with the video data, so that when the video data is displayed the user's lighting is in synchronization with the video. In one embodiment, the illumination data is also synchronized with events of a game, so that a user playing games in a gaming environment will have his lighting synchronized with video and events of the game. In another embodiment, the content stream is embedded on a disk.

Abstract: A content stream comprising video and synchronized illumination data is based on a reference lighting setup from, for example, the site of the content creation. The content stream is received at a user location where the illumination data controls user lighting that is synchronized with the video data, so that when the video data is displayed the user's lighting is in synchronization with the video. In one embodiment, the illumination data is also synchronized with events of a game, so that a user playing games in a gaming environment will have his lighting synchronized with video and events of the game. In another embodiment, the content stream is embedded on a disk.

Abstract: For a picture with two or more color components, the prediction residuals for the first color component of a block to be encoded may be transformed with a first transform. The transform coefficients for the first color component may go through quantization, de-quantization and inverse transform to obtain reconstructed prediction residuals. Based on the reconstructed prediction residuals for the first color component, the phases of the transform basis function of the first transform can be adjusted to improve the sparsity of the transformed signal. The prediction residuals for the remaining color components may then be transformed with the adjusted transform. In order to determine the phase shift factor, the reconstructed prediction residuals for the first color component may be transformed with the first transform, adjusted by different candidate phase shift factors, and the candidate phase shift factor that provides a smallest sparsity measure can be selected for the block.

Abstract: A method for encoding a current image in an image set is disclosed. The method incudes accessing a reference image for the current image, determining at least one prediction image wherein a prediction image is obtained from a geometric transform applied to the reference image, the prediction image being a prediction for at least one region of current image, encoding the current image based on the at least a prediction image using block matching compensation, generating a bitstream comprising the encoded image, an item representative of the reference image and an item representative of the at least one geometric transform.