Abstract: Various embodiments provide an apparatus, a method and a computer program product for end-to-end learned predictive coding of media frames. An example apparatus includes at least one processor; and at least one non-transitory memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform: encode or decode one or more media frames for at least one neural network; wherein an inter-frame codec is applied to at least one media frame of the one or more media frames; and wherein a first decoded reference frame and a second decoded reference frame refer to reference frames for the at least one media frame.

Abstract: An example method includes: receiving an input extended block comprising an input block and input margins, wherein the input block is derived from an image or a video frame, and wherein the input block and the input margins are input to a filter, and wherein the input margins comprise pixels of the image or the video frame; filtering the input extended block to obtain a filtered extended block comprising a filtered block and filtered margins; receiving other blocks, wherein the other blocks are derived from the image or video frame, and wherein at least one pixel of the other blocks overlaps with at least one pixel of the filtered margins of the filtered extended block; and blending the filtered extended block with the other blocks, wherein an operation to blend the filtered extended block with the other blocks is applied to two or more overlapping pixels

Abstract: Example apparatus include circuitry for: obtaining audio content representing at least one audio space; enabling at least one digital signal processing operation to render the audio content such that the rendered audio content includes at least one target response for the at least one audio space wherein the enabling of the at least one digital signal processing operation to render the audio content is controlled based on obtaining the at least one target response for the at least one audio space. When the obtained target response is known the circuitry obtains at least one parameter for the at least one digital signal processing operation. When the obtained target response is unknown the circuitry obtains at least one parameter for a neural network and determines at least one parameter for the at least one digital signal processing operation.

Abstract: A method, apparatus and computer program is described comprising: generating or obtaining at least one augmented reality image for presentation to a user, the augmented reality images comprising one or more virtual objects, displays or devices of an augmented reality system; attenuating real-world audio of the augmented reality system; and controlling the provision of audio to the user in at least one audio focus or beamform direction relating to at least one selected real-world object of the augmented reality system.

Abstract: A method comprising: obtaining (400), in a first apparatus (500), media content, encoding (402), in a neural data compression network of the first apparatus (500), the media content wherein one or more parameters of the neural data compression network are determined based on a type of at least one analysis task to be performed on the media content; and transmitting (404) the encoded media content to a second apparatus (502).

Abstract: A method is provided for defining a metadata box of a neural network representation (NNR) item data, wherein the NNR item data comprises an NNR bitstream; and defining an association between the NNR item data and an NNR configuration by using a configuration item property, wherein the NNR configuration item property comprises information about stored NNR item data. Corresponding apparatuses and computer program products are also provided.

Abstract: A method is provided for computing predetermined loss terms based on original data and decoded data; training one or more neural networks of a system by using the predetermined loss terms; updating weights for one or more of other loss terms; and determining trade-offs between predetermined objectives of the system. Corresponding apparatuses and computer program products are also provided.

Abstract: Various embodiments provide an apparatus, a method, and a computer program product. An example method includes determining one or more quantization parameters (quantizers) based at least on one or more of the following: a mean absolute value computed based on a set of parameters of a neural network comprising a parameter; a maximum absolute value computed based on a set of activations of the neural network comprising an activation; a number of parameters in the set of parameters of the neural network comprising the parameter; or a maximum absolute value computed based on an output value computed based on the parameter and the activation; and quantizing at least one of the parameter or the activation based at least on the one or more quantization parameters.

Abstract: A method comprising: obtaining a configuration of at least one neural network comprising a plurality of intra-prediction mode agnostic layers and one or more intra-prediction mode specific layers, the one or more intra-prediction mode specific layers corresponding to different intra-prediction modes; obtaining at least one input video frame comprising a plurality of blocks; determining to encode one or more blocks using intra prediction; determining an intra-prediction mode for each of said one or more blocks; grouping blocks having same intra-prediction mode into groups, each group being assigned with a computation path among the plurality of intra-prediction mode agnostic and the one or more intra-prediction mode specific layers; training the plurality of intra-prediction mode agnostic and/or the one or more intra-prediction mode specific layers of the neural networks based on a training loss between an output of the neural networks relating to a group of blocks and ground-truth blocks, wherein the ground-t

Abstract: An apparatus including circuitry configured for: in a first-person perspective mediated reality state, rendering mediated reality content as content distributed across a first area of a user's field of view, wherein a point of view of a user determines a point of view within a three-dimensional virtual space and determines at least part of the content distributed across the first area of a user's field of view as a virtual scene; responding to at least one user gesture to enter a spatially consolidated state; and in the spatially consolidated state, rendering the mediated reality content as content distributed across a second area of a user's field of view, wherein the second area is smaller than the first area and the point of view of the user does not determine the content distributed across the second area of a user's field of view.

Abstract: Various embodiments provide an apparatus, a method, and a computer program product. The apparatus includes at least one processor; and at least one non-transitory memory comprising computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform: determine a plurality of weights used to calculate a weighted loss based at least on a performance of a plurality of neural networks on one or more training samples; and jointly train the plurality of neural networks, wherein at each training iteration the plurality of neural networks are trained based at least on the weighted loss.

Abstract: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: maintain a first parameter update tree that tracks residuals of weight updates of a machine learning model; maintain a second parameter update tree that tracks the weight updates of the machine learning model; pass the first parameter update tree and the residuals to an encoder; receive a first bitstream generated for the residuals from the encoder; pass the second parameter update tree and the weight updates to the encoder; receive a second bitstream generated for the weight updates from the encoder; and determine whether to signal to a decoder the first bitstream generated for the residuals or the second bitstream generated for the weight updates.

Abstract: In example embodiments, an apparatus, a method, and a computer program product are provided. An example apparatus include processing circuitry; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the processing circuitry, cause the apparatus at least to: overfit a neural network on each media item, from a batch of media items, for a number of iterations to obtain an overfitted neural network model for the each media item; evaluate the overfitted neural network model on the each media item to obtain evaluation errors; and update parameters of the neural network to be based on the evaluation errors.

Abstract: An apparatus comprising: at least one processor; and at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: receive a signal, the signal comprising a sparse signal; perform residual coding on the signal; perform predictive coding on the signal; determine a residual, the residual comprising a residual of the signal and a base signal or a residual of an approximation and the base signal, the approximation being an approximation of the signal; and determine whether to transmit the residual or the signal over a communication channel.

Abstract: In example embodiments, an apparatus, a method, and a computer program product are provided. The apparatus includes at least one processor; and at least one non-transitory memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform: encode or decode a high-level bitstream syntax for at least one neural network; wherein the high-level bitstream syntax comprises at least one information unit, wherein the at least one information unit comprises syntax definitions for the at least one neural network or a portion of the at least one neural network; and wherein a serialized bitstream comprises one or more of the at least one information units.

Abstract: Various embodiments provide an apparatus, a method, and a computer program product. An example apparatus includes at least one processor; and at least one non-transitory memory comprising computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform: determine a subset of parameters to overfit from a set of candidate parameters of decoder side neural network to be overfitted (OPs); wherein the subset of parameters to overfit is smaller than the set of candidate parameters to be overfitted; and overfit the determined subset of parameters.

Abstract: Various embodiments provide an apparatus, a method, and a computer program product. The apparatus includes at least one processor; and at least one non-transitory memory comprising computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform: organize plurality of decoders side neural networks based on one or more task categories or one or more tasks; and select a decoder side neural network based at least on the one or more task categories or the one or more task.

Abstract: Various embodiments provide an apparatus, a method, and a computer program product. 1. An apparatus incudes at least one processor; and at least one non-transitory memory includes computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to: perform an overfitting operation, at an encoder side, to obtain an overfitted probability model, wherein overfitting comprises one or more training operations applied to a probability model, wherein one or more parameters of the probability model are trained; use the overfitted probability model to provide probability estimates to a lossless codec or a substantially lossless codec for encoding data or a portion of the data; and signal information to a decoder on whether to perform the overfitting operation at the decoder side.

Abstract: Media content is received for streaming to a user device. A neural network is trained based on a first portion of the media content. Weights of the neural network are updated to overfit the first portion of the media content to provide a first overfitted neural network. The neural network or the first overfitted neural network is trained based on a second portion of the media content. Weights of the neural network or the first overfitted neural network are updated to overfit the second portion of the media content to provide a second overfitted neural network. The first portion and the second portion of the media content are sent with associations to the first overfitted neural network and the second overfitted to the user equipment.

Abstract: Various embodiments provide an apparatus, a method, and a computer program product. An example apparatus includes at least one processor; and at least one non-transitory memory comprising computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to iteratively perform following until a stopping criterion is met: provide a finetuning driving content (FDC) or a content derived from FDC to a decoder side neural network (DSNN); compute an output of the DSNN as a processed FDC; compute a loss based on the processed FDC and an approximated ground truth data (AGT) associated with the FDC; compute an update to the DSNN; and apply the computed update to the DSNN.