Abstract: Bi-prediction with un-equal weights varying spatially over a current block is achieved with weights which can be derived using a regression-based model computed with at least one template made of neighboring reconstructed samples. The bi-prediction can be used with geometric partition mode blocks with different motion vectors per sub-block. The regression aims at minimizing a distance between reconstructed samples of the template and prediction samples of the template derived from two uni-prediction templates.

Abstract: A video coding system for an image of a video representative of a virtual environment comprises an encoder and a decoder that perform a reconstruction of a differential picture based on a decoded picture buffer, wherein the decoded picture buffer comprises a least a picture based on a reference differential picture and on a second picture generated by a three-dimensional renderer based on the information representative of the virtual environment. An encoding method and a decoding method are described and comprise adding the reconstructed differential picture is to the second picture.

Abstract: The cross-component model generally describes a relation between the luma samples and associated chroma samples. Using the cross-component model parameters derived based on a neighboring region, the chroma samples of the current block can be predicted from the collated reconstructed luma samples. Because the cross-component model parameters are derived at both the encoder and decoder, these model parameters need not to be signaled explicitly and therefore can reduce signaling overhead. In one implementation, the model parameters of across-component model are obtained that models a relation between at least a temporal prediction of a luma sample (PredY) and a temporal prediction of a chroma sample (PredCb/PredCr). Encoding and decoding method based on the cross-component residual/prediction model are further disclosed.

Abstract: At least a method and an apparatus are presented for efficiently encoding or decoding video by applying a neural network-based processing to a tensor of input data to generate a tensor of output data. For example, the quantization of the tensors is limited to a scaling by a power of 2. For example, the tensor product layer, the bias addition layer and the activation are fused to reduce the number of operations and increase the available bits to represent the values.

Abstract: In-loop filters are commonly used to filter reconstructed video paths within a digital video encoder and decoder. Commonly used filters are deblocking filters and adaptive loop filters. Embodiments provide filter classification techniques for determination of such in-loop filters with adaptability to depend on criteria comprising spatial distance from a block edge, features across block edges, selection of samples to be filtered and filter size or shape. In one embodiment, deblocking characteristics are combined with adaptive loop filters in one filtering step.

Abstract: A method and an apparatus for encoding or decoding a video are provided. A luma component of a block of the video is reconstructed. The video block being coded in an inter coding mode, a cross-component prediction mode for the video block is determined and one or more syntax elements are encoded for the video block, the one or more syntax elements providing for deriving the cross-component prediction mode on the decoder side from among a set of cross-component prediction modes. The chroma component of the video block is encoded based on the reconstructed luma component and the determined cross-component prediction mode.

Abstract: Systems, methods, and instrumentalities are disclosed for predicting and/or reordering the prediction boundary, such as a geometric partition mode (GPM) boundary for a coding block. For example, a device may determine a geometric boundary prediction for a current block based on a geometric boundary associated with a neighboring block of the current block. The device may encode or decode the current block based at least in part on the determined geometric boundary prediction. The neighboring block of the current block may be a spatial neighbor of the current block. The neighboring block of the current block may be a temporal neighbor of the current block.

Abstract: An iterative training of neural networks for video coding and decoding using intra prediction is provided that finds a tradeoff between an extreme genericity and an extreme specialization to a codec for the trained neural networks. At the first iteration, the set of neural networks is trained following a partitioning approach. Then, for several iterations, the set of neural networks is inserted into the codec, and pairs of a block and its context are extracted from the partitioning of images via the codec with a single additional neural network-based mode then, the neural networks are retrained on these pairs. This way, from the second iteration, the neural networks learn an intra prediction diverging from that in the codec while still being valuable for the codec in terms of rate-distortion performance.

Abstract: A method comprising: obtaining an initial list of motion vector predictors candidates for a current block; obtaining bi-prediction candidates from the initial list, each bi-prediction candidate comprising two motion vectors, a first motion vector of the two motion vectors pointing on a first prediction block in a first reference picture and a second motion vector of the two motion vectors pointing on a second prediction block in a second reference picture; calculating a score for bi-prediction candidates of the initial list; and, obtaining a motion vector predictor for the motion information of the current block based on the calculated scores; wherein each score is value representative of a difference between a first template and a second template, the first template comprising samples of the first prediction block and the second template comprising samples of the second prediction block corresponding spatially to the samples of the first template.

Abstract: System, methods, and instrumentalities are disclosed for reducing intra prediction auto-regressive (AR) complexity. A device may obtain an auto-regressive prediction model for a current block. The device may obtain a set of reference samples. The device may determine, in parallel, a first prediction sample and a second prediction sample of the current block based on the set of reference samples and the auto-regressive prediction model. The device may decode the current block based on the first prediction sample and the second prediction sample.

Abstract: Template based derivation of intra sub-partitioning (ISP) mode is used to improve coding efficiency of intra predicted coded video blocks. The template is formed from surrounding reconstructed top and left samples. In one embodiment, ISP is performed independently from template based intra mode derivation (TIMD). The derivation of an intra sub-block partition mode is determined from template pixels using a current prediction mode for either an upper or left template. In another embodiment, ISP and TIMD are combined. In a further embodiment, a flag is signaled to communicate template based intra mode derivation and intra sub-block partition usage.

Abstract: A method for reducing a latency in a gaming application comprising: obtaining (305B) a first frame, said first frame being representative of a first action performed by a user in the gaming application; obtaining (500) information representative of a second action performed by the user in the gaming application, said second action following the first action; and, predicting (500) a second frame corresponding to the second action from data comprising at least the first frame and the information representative of a second action using a neural network.

Abstract: In one implementation, the prediction of a block is filtered using a filter learned on a template. To learn the filter, a first template is generated from a set of decoded samples in an area neighboring to the current block, and a second template is generated from a set of predicted samples in the same neighboring area, where the set of predicted samples can be obtained based on the prediction parameters used to predict the current block or from stored prediction samples for neighboring block(s) associated with the template area. The filter parameters are calculated by minimizing a loss function between the set of decoded samples and a set of filtered predicted samples. The filter can be a convolutional 7-tap filter, a convolution 5-tap filter, or a piecewise linear function.

Abstract: Systems, methods, and instrumentalities are disclosed herein for video encoding and/or decoding using intra combined intra-intra prediction mode. In examples, a video decoder/encoder may obtain a first prediction signal and a second intra prediction signal for a block. The first prediction signal may be associated with a first intra prediction mode. The second prediction signal may be associated with a second intra prediction mode. The decoder/encoder may generate a prediction sample by determining a first weight associated with the first intra prediction signal and a second weight associated with the second intra prediction signal. The first weight and the second weight may be determined by a first neighboring block prediction mode and a second neighboring block prediction mode. The decoder/encoder may derive a residual sample based on the predicted sample. The block may be decoded or encoded based on the derived residual sample.

Abstract: In one implementation, the prediction of a block is filtered using a filter learned on a template. To learn the filter, a first template is generated from a set of decoded samples in an area neighboring to the current block, and a second template is generated from a set of predicted samples in the same neighboring area, where the set of predicted samples are obtained based on the intra prediction mode used to predict the current block. The filter parameters are calculated by minimizing a loss function between the set of decoded samples and a set of filtered predicted samples. To reduce the computation complexity, the intra filtering may be only enabled when the intra prediction mode is from the MPM list, is obtained from TIMD or DIMD, or if at least an intra mode in the template area is a close neighbor to the current intra prediction mode.

Abstract: Systems, methods, and instrumentalities are disclosed for storing a post processing order in dedicated code in a header of a region (e.g., a slice) and/or a picture. A video encoding and/or decoding device may include a processor configured to determine a post-filter order associated with a sub-region of a region or a picture. The post-filter order may be associated with at least two filters. The post-filter order may indicate a post-filter processing order. The device may send the post-filter order in a region header or a picture header. The device may generate a coding cost and/or a distortion. The coding cost and/or the distortion may be associated with the post-filter processing order.

Abstract: Intra sub-block partitioning (ISP) is changed to improve coding in certain instances. In one embodiment, when the ISP direction and Intra Prediction Mode (IPM) are both oriented in one direction, it is proposed to change the splitting for the sub-partitions. Depending on the embodiment, the IPMs considered vertical can be all IPMs with index greater than value (34), or the IPMs considered vertical could be only IPMs with an index value greater than (50) to only account for modes that only use the above reference samples. In another embodiment, sub-partitions could have an additional split in either direction.

Abstract: Flexible methods to use a combination of coding modes while keeping good performance tradeoff are implemented with embodiments that determine coding mode information. In one embodiment, determination is made of activation of a first coding mode, and upon determination of whether a second coding mode is used, coding or decoding using the determined modes occurs. Other embodiments enable use of multiple reference lines, intra sub-block partitioning, matrix intra prediction, and low frequency non-separable transforms.

Abstract: Apparatuses and methods are disclosed including techniques for encoding and decoding video data. The disclosed techniques include obtaining video data, including data representing a video data region, and then computing models used for cross-component based prediction of chroma samples from the video data region. The computing of models includes accumulating concurrently data of multiple models using a single loop through reference samples selected from the video data. The accumulated data of the models are generated based on the reference samples according to their classification into respective classes. The models are then applied using a single loop through samples of the video region, the models applied to predict the chroma samples according to their classification into the respective classes.

Abstract: Described are methods and apparatuses for encoding/decoding a picture comprising predicting at least one block, wherein the predicting comprises performing motion compensation and local illumination compensation based on a reference block, the local illumination compensation including applying a linear model based on sums of absolute differences of neighboring reconstructed samples and corresponding reference samples of the reference block, wherein the neighboring reconstructed samples and corresponding reference samples of the reference block are co-located according to an L-shape substantially adjacent to the block to be predicted, the L-shape comprising a row of pixels located to the top side of the predicted block and a column of pixels located to the left side of the predicted block, the co-location being determined according a motion vector of the predicted block.