Abstract: A method and an apparatus for video processing are provided. The method includes that a decoding terminal receives a plurality of coded video frames coded using one or more generative adversarial networks (GANs), receives network parameters related to the one or more GANs, and decodes the plurality of coded video frames using GANs based on the network parameters. Further, the one or more GANs respectively implement one or more video coding functions including reference-frame coding, motion-compensated frame prediction, and residue-frame coding.

Abstract: A neural network system, a method and an apparatus for image compression are provided. The neural network may include a generator including an encoder, an entropy estimator, and a decoder, where the encoder receives an input image and generates an encoder output, a plurality of quantized feature entries are obtained based on the encoder output outputted at a last encoder block, the entropy estimator receives the plurality of quantized feature entries and calculates an entropy loss based on the plurality of quantized feature entries, and the decoder receives the plurality of quantized feature entries and generates a reconstructed image. Furthermore, the neural network may include a discriminator that determines whether the reconstructed image different from the input image based on a discriminator loss. Moreover, the generator may determine whether content of the reconstructed image matches content of the input image based on a generator loss including the entropy loss.

Abstract: A method, an apparatus, and a non-transitory computer-readable storage medium for video compression using a generative adversarial network (GAN) are provided. The method includes obtaining, by a generator of the GAN, a reconstructed target frame based on a reference frame and a raw target frame to be reconstructed; concatenating, by a transformer-based discriminator of the GAN, the reference frame, the raw target frame and the reconstructed target frame to obtain a paired data; determining, by the transformer-based discriminator of the GAN, whether the paired data is real or fake to guide reconstruction of the raw target frame; and determining a generator loss and a transformer-based discriminator loss, and performing gradient back propagation and updating network parameters of the GAN based on the generator loss and the transformer-based discriminator loss.

Abstract: A method and an apparatus for video processing are provided. The method includes that a decoding terminal receives a plurality of coded video frames coded using one or more generative adversarial networks (GANs), receives network parameters related to the one or more GANs, and decodes the plurality of coded video frames using GANs based on the network parameters. Further, the one or more GANs respectively implement one or more video coding functions including reference-frame coding, motion-compensated frame prediction, and residue-frame coding.

Abstract: A class-specific neural network for video compressed sensing and methods for training and testing the class-specific neural network are provided. The class-specific neural network includes a Gaussian-mixture model (GMM) and a plurality of encoders, where the GMM classifies video frame blocks with a plurality of clusters and assigns the video frame blocks to the plurality of clusters. Further, the plurality of encoders receive the video frame blocks and generate a plurality of compressed-sensed frame block vectors, where the plurality of encoders correspond to the plurality of clusters.

Abstract: A method for detecting moving objects in video frames, an apparatus and a non-transitory computer-readable storage medium thereof are provided. The method includes that: an encoder in a 3-dimenional (3D) separable convolutional neural network with multi-input multi-output (3DS_MM) receives a first input including multiple video frames, where the encoder includes a plurality of encoder layers including 3D separable convolutional neural network (CNN) layers; the encoder generates a first encoder output; and a decoder in the 3DS_MM receives the first encoder output and generates a first output including multiple first binary masks related to the first input, where the decoder includes a plurality of decoder layers comprising 3D separable transposed CNN layers.

Abstract: An apparatus including a memory operably coupled to a processor. The processor is configured to determine whether to use an intra smoothing filter for a rectangular prediction unit (PU), wherein a width of the rectangular PU is different from a height of the rectangular PU.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing intra coding predictions. An intra-coding process applied to pixels in a frame of media is determined. The intra-coding process is determined whether to corresponding to at least one of most probable modes. In response to determining the intra-coding process does not correspond to the at least one of the most probable modes, four angular prediction modes are extracted from a list of prediction modes. A set of candidates based on the four angular prediction modes are determined. A pre-defined order of the set of candidates is determined, wherein each candidate mode of the set of candidate modes is included in a ranked order and signaled with a particular fixed length coding, and wherein a length of the particular fixed length coding increases based on the pre-defined order of the set of candidate modes.

Abstract: An apparatus including a memory operably coupled to a processor. The processor is configured to determine whether to use an intra smoothing filter for a rectangular prediction unit (PU), wherein a width of the rectangular PU is different from a height of the rectangular PU.

Abstract: An apparatus including a memory operably coupled to a processor. The processor is configured to select an intra smoothing filter for a rectangular prediction unit (PU) based on a lookup table (LUT) used for square PUs, wherein a width of the rectangular PU is different from a height of the rectangular PU.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing intra coding predictions. An intra-coding process applied to pixels in a frame of media is determined. The intra-coding process is determined whether to corresponding to at least one of most probable modes. In response to determining the intra-coding process does not correspond to the at least one of the most probable modes, four angular prediction modes are extracted from a list of prediction modes. A set of candidates based on the four angular prediction modes are determined. A pre-defined order of the set of candidates is determined, wherein each candidate mode of the set of candidate modes is included in a ranked order and signaled with a particular fixed length coding, and wherein a length of the particular fixed length coding increases based on the pre-defined order of the set of candidate modes.

Abstract: System and method embodiments for image coding are disclosed. In an embodiment, a method in a data processing system for image encoding includes determining a sparsity constraint according to a dimension of an input image signal. The method also includes iteratively determining a plurality of approximations to the input image signal. Each iteration provides an approximation of the input image signal. Each approximation includes a set of dictionary element indices and coefficients. The dictionary is an over-complete dictionary. Iterations of the determining step are terminated when a number of iterations is equal to the sparsity constraint. The method also includes selecting one of the plurality of approximations according to a minimum rate-distortion cost. The method also includes determining an encoded image signal according to non-zero coefficients and corresponding indices for each non-zero coefficient in the selected approximation.

Abstract: An apparatus comprises a receiver configured to receive video views comprising a reference view and a current view, wherein the reference view comprises a reference block and the current view comprises a current block, and a processor coupled to the receiver and configured to determine neighboring reference pixels associated with the reference block, determine neighboring current pixels associated with the current block, determine a first positional pairing between the neighboring reference pixels and the neighboring current pixels, determine a second positional pairing between the neighboring reference pixels and the neighboring current pixels, and determine an optimal pairing from between the first positional pairing and the second positional pairing.

Abstract: System and method embodiments are provided for achieving improved View Synthesis Distortion (VSD) calculation and more accurate distortion estimation of encoded video frames. An embodiment method includes obtaining a depth map value for a video frame and determining a weighting factor for depth distortion in accordance with the depth map value. The weighting factor maps a pixel range of the depth map value to an output function having higher values for closer image objects and lower values for farther image objects. The VSD for the video frame is then calculated as a function of absolute horizontal texture gradients weighted by a depth distortion value and the weighting factor determined in accordance with the depth map value.

Abstract: A method, an apparatus and a decoder for decoding a block of a depth map are provided. An ordered list of decoding modes is obtained, wherein the ordered list of decoding modes comprises a plurality of decoding modes each of which is capable of being used for decoding of the block. A plurality of depth modeling modes (DMMs) each of which is capable of being used for decoding of the block are obtained. And whether a DMM of the plurality of DMMs is to be added into the ordered list of decoding modes in accordance with a decision condition is determined.

Abstract: A video codec configured to receive a current block and a plurality of neighboring pixels, wherein the current block comprises a first partition and a second partition, select one or more reference pixels from the plurality of neighboring pixels, and predict a plurality of pixels located in the second partition based on the reference pixels.

Abstract: An apparatus including a memory operably coupled to a processor. The processor is configured to select an intra smoothing filter for a rectangular prediction unit (PU) based on a lookup table (LUT) used for square PUs, wherein a width of the rectangular PU is different from a height of the rectangular PU.

Abstract: A method for coding a coding unit that is coded with a single sample value is provided. The method selects a coding pattern from at least two predetermined coding patterns, each of which includes a plurality of boundary neighboring samples of the coding unit that have been reconstructed, and decodes the coding unit according to a value of at least one of the plurality of boundary neighboring samples of the selected coding pattern that is available.

Abstract: A method for reducing a computational load in high efficiency video coding includes generating a full rate distortion calculation list of selected intra coding modes where the intra coding modes including intra prediction modes and depth modeling modes. A rate distortion cost is determined, with a segment-wise depth coding mode being disabled, for each intra prediction mode in the full rate distortion calculation list and a smallest rate distortion cost intra prediction mode is selected. A rate distortion cost for a particular intra prediction mode is calculated with the segment-wise depth coding mode enabled. After comparison, one of the particular intra prediction mode and the smallest rate distortion cost intra prediction mode having the smallest rate distortion cost is applied to a prediction unit.

Abstract: Depth based block partitioning in high efficiency video coding is provided by partitioning a video image block into different partitions using a binary segmentation mask. A determination is made whether to filter pixels at a boundary between the partitions. A particular pixel is not filtered in response to each adjacent pixel in vertical and horizontal planes in relation to the particular pixel having a same value. The particular pixel is filtered in response to any adjacent pixel in the vertical and horizontal planes in relation to the particular pixel having a different value than any other adjacent pixel in the vertical and horizontal planes in relation to the particular pixel. Pixels are filtered pursuant to a filtering process in response to a filtering determination.