Abstract: A proposed intermediate way of handling the renderable portion of the first view results in more efficient coding. Instead of omitting the coding of the renderable portion completely, even more efficient coding of multi-view signals entails merely suppressing the coding of the residual signal within the renderable portion, whereas the prediction parameter coding still takes place from the non-renderable portion of the multi-view signal across the renderable portion so that prediction parameters for the renderable portion may be exploited for predicting parameters for the non-renderable portion. The additional coding rate for transmitting the prediction parameters for the renderable portion may be kept low as this merely aims at forming a continuation of the parameter history across the renderable portion to serve as a basis for prediction parameters of other portions of the multi-view signal.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.

Abstract: A picture of a sequence of pictures is decoded by deriving partitioning information from the data stream, subdividing the picture into coding blocks according to the partitioning information, and decoding the picture in units of the coding blocks. Filtering information is derived from the data stream, which indicates a subdivision of the picture into filtering blocks. The subdivision into filtering blocks is used for filtering the picture.

Abstract: A scalable video decoder is described which is configured to reconstruct a base layer signal from a coded data stream to obtain a reconstructed base layer signal; and reconstruct an enhancement layer signal including spatially or temporally predicting a portion of an enhancement layer signal, currently to be reconstructed, from an already reconstructed portion of the enhancement layer signal to obtain an enhancement layer internal prediction signal; forming, at the portion currently to be reconstructed, a weighted average of an inter-layer prediction signal obtained from the reconstructed base layer signal, and the enhancement layer internal prediction signal to obtain an enhancement layer prediction signal such that a weighting between the inter-layer prediction signal and the enhancement layer internal prediction signal varies over different spatial frequency components; and predictively reconstructing the enhancement layer signal using the enhancement layer prediction signal.

Abstract: A method for decoding a predetermined block of a picture using intra-prediction by reading, for each of predetermined intra-prediction blocks, from a data stream, a mode index identifying a matrix-based intra-prediction mode is disclosed. Samples of each predetermined intra-prediction block are predicted by computing a matrix-vector product between an input vector derived from reference samples and a prediction matrix associated with the identified matrix-based intra-prediction mode (k) and associating components of an output vector obtained by the matrix-vector product onto sample positions of the predetermined block. Further predetermined intra-predicted blocks of the picture are predicted to obtain a prediction signal. A transformation flag is decoded from the data stream using context adaptive binary arithmetic coding, and a prediction residual is decoded and re-transformed using a reverse transformation based on the transformation flag to obtain a prediction residual signal.

Abstract: A video decoder includes one or more processors that are configured to: determine that a picture is in a 4:4:4 color sampling format, determine, based at least in part on an index signaled in a data stream, a matrix-based intra prediction (MIP) mode, decode a luma block of the picture using the MIP mode, determine, for a chroma block of the picture, whether a coding tree of the chroma block is a single tree, select an intra prediction mode for decoding the chroma block, and decode the chroma block using the selected intra prediction mode. The selected intra prediction mode is the MIP mode in response to the coding tree of the chroma block being the single tree, and the selected intra prediction mode is a planar intra prediction mode in response to the coding tree of the chroma block not being the single tree.

Abstract: Methods, non-transitory computer readable media, apparatuses, and systems for high-resolution image generation using diffusion models include obtaining a prompt and generating, using a first diffusion model, a predicted denoised image at a first resolution based on the prompt. The predicted denoised image is generated at a first intermediate diffusion step of the first diffusion model. The predicted denoised image is upsampled to obtain an upsampled denoised image at a second resolution that is higher than the first resolution. A second diffusion model then generates an output image at the second resolution based on the prompt and the upsampled denoised image.

Abstract: A method, apparatus, non-transitory computer readable medium, apparatus, and system for video generation include first obtaining a training set including a training video. Then, embodiments initialize a video generation model, sample a subnet architecture from an architecture search space, and a identify a subset of the weights of the video generation model based on the sampled subnet architecture. Subsequently, embodiments train, based on the training video, a subnet of the video generation model to generate synthetic video data. The subnet includes a subset of the weights of the video generation model.

Abstract: The present invention concerns an apparatus configured to partition a picture into leaf blocks using recursive multi-tree partitioning, block-based encode the picture into a data stream using the partitioning of the picture into the leaf blocks, wherein the apparatus is configured to, in partitioning the picture into the leaf blocks, for a predetermined block which extends beyond a boundary of the picture, reduce an available set of split modes depending on a position at which the boundary of the picture crosses the predetermined block in order to obtain a reduced set of one or more split modes, wherein the apparatus is configured to signal a selected split mode in the data stream.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.

Abstract: A method for decoding a predetermined block of a picture using intra-prediction by reading, for each of predetermined intra-prediction blocks, from a data stream, a mode index identifying a matrix-based intra-prediction mode is disclosed. Samples of each predetermined intra-prediction block are predicted by computing a matrix-vector product between an input vector derived from reference samples and a prediction matrix associated with the identified matrix-based intra-prediction mode (k)—and associating components of an output vector obtained by the matrix-vector product onto sample positions of the predetermined block. Further predetermined intra-predicted blocks of the picture are predicted to obtain a prediction signal. A transformation flag is decoded from the data stream using context adaptive binary arithmetic coding, and a prediction residual is decoded and re-transformed using a reverse transformation based on the transformation flag to obtain a prediction residual signal.

Abstract: A block-based decoder is configured to partition a picture of more than one color component and of a color sampling format, according to which each color component is equally sampled, into blocks using a partitioning scheme in which the picture is equally partitioned with respect to each color component. A first color component is decoded in units of the blocks with selecting, for each of intra-predicted first color component blocks, one out of a first set of intra-prediction modes. The first set comprises matrix-based intra prediction modes according to each of which a block inner is predicted by deriving a sample value vector out of neighboring reference samples, computing a matrix-vector product between the sample value vector and a prediction matrix associated with the respective matrix-based intra prediction mode to obtain a prediction vector, and predicting samples in the block inner on the basis of the prediction vector.

Abstract: Video codec for supporting temporal inter-prediction, configured to perform padding of an area of a referenced portion of a reference picture which extends beyond a border of the reference picture, which referenced portion is referenced by an inter predicted block of a current picture by selecting one of a plurality of intra-prediction modes, and padding the area using the selected intra-prediction mode.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for combining digital images. In particular, in one or more embodiments, the disclosed systems combine latent codes of a source digital image and a target digital image utilizing a blending network to determine a combined latent encoding and generate a combined digital image from the combined latent encoding utilizing a generative neural network. In some embodiments, the disclosed systems determine an intersection face mask between the source digital image and the combined digital image utilizing a face segmentation network and combine the source digital image and the combined digital image utilizing the intersection face mask to generate a blended digital image.

Abstract: Techniques for block wise encoding and decoding may be applied to encoders, decoders, and methods for encoding or decoding and include assigning, based on a first signalization in the data stream, the predetermined block to the first set or the second set, sorting the assigned set of intra-prediction modes according to intra-prediction modes used for neighboring blocks, neighboring the predetermined block, to obtain a list of intra prediction modes, deriving, for the predetermined block, from the data stream, an index into the list of intra prediction modes, predicting the predetermined block using an intra prediction mode onto which the index points. The decoder or encoder, if the assigned set is the first set of intra-prediction modes, in sorting the assigned set, uses a second mapping which maps each intra-prediction mode of the second set of prediction modes onto a representative one in the first set of intra-prediction modes.

Abstract: A video decoder for decoding an encoded video signal including encoded picture data and indication data of a picture of a video to reconstruct the picture of the video is provided. The video decoder includes an interface configured for receiving the encoded video signal, and a data decoder configured for reconstructing the picture of the video by decoding the encoded picture data using the indication data. The picture is partitioned into a plurality of coding areas. One or more coding areas of the plurality of coding areas include two or more coding tree units of the plurality of coding tree units, wherein each coding area of the one or more coding areas which includes two or more coding tree units exhibits a coding order for the two or more coding tree units of the coding area.

Abstract: The present invention concerns an apparatus configured to partition a picture into leaf blocks using recursive multi-tree partitioning, block-based encode the picture into a data stream using the partitioning of the picture into the leaf blocks, wherein the apparatus is configured to, in partitioning the picture into the leaf blocks, for a predetermined block which extends beyond a boundary of the picture, reduce an available set of split modes depending on a position at which the boundary of the picture crosses the predetermined block in order to obtain a reduced set of one or more split modes, wherein the apparatus is configured to signal a selected split mode in the data stream.

Abstract: An apparatus for block-based predictive decoding of a picture comprises a combiner configured to combine a residual signal a predetermined block of the picture and a reference signal for the predetermined block so as to obtain a first spectrum, the residual signal correcting a prediction error of a prediction of the predetermined block of the picture; a reducer configured to perform thresholding on the first spectrum to obtain a second spectrum so that coefficients below a threshold value are set to a predefined value; an extractor configured to obtain from the second spectrum a modified version of the residual signal; and a reconstructor block configured to decode the predetermined block of the picture from the data stream on the basis of the modified version of the residual signal.

Abstract: Techniques for block wise encoding and decoding may be applied to encoders, decoders, and methods for encoding or decoding. In one example, there is provided a technique to decode or encode a predetermined block of the picture by assigning, based on a first signalization in the data stream, the predetermined block to a first set or a second set; sorting the assigned set of intra-prediction modes according to intra-prediction modes used for neighboring blocks, neighboring the predetermined block, to obtain a list of intra prediction modes; deriving, for the predetermined block, from the data stream, an index into the list of intra prediction modes; and predicting the predetermined block using an intra prediction mode onto which the index points.