Abstract: A device includes a memory and one or more processors coupled to the memory and configured to execute instructions from the memory. Execution of the instructions causes the one or more processors to combine two or more data portions to generate input data for a decoder network. A first data portion of the two or more data portions is based on a first encoding of a data sample by a multiple description coding network and content of a second data portion of the two or more data portions depends on whether data based on a second encoding of the data sample by the multiple description coding network is available. Execution of the instructions also causes the one or more processors to obtain, from the decoder network, output data based on the input data and to generate a representation of the data sample based on the output data.

Abstract: Systems and techniques are described for coding audio signals. For example, a voice decoder can generate, using a first neural network, an excitation signal for at least one sample of an audio signal at least in part by performing a non-linear operation based on one or more inputs to the first neural network, the excitation signal being configured to excite a learned linear filter. The voice decoder can further generate, using the learned linear filter and the excitation signal, at least one sample of a reconstructed audio signal. For example, a second neural network can be used to generate coefficients for one or more learned linear filters, which receive as input the excitation signal generated by the first neural network trained to perform the non-linear operation.

Abstract: Systems and techniques are described for coding audio signals. For example, a voice decoder can generate, using a neural network, an excitation signal for at least one sample of an audio signal based on one or more inputs to the neural network, the excitation signal being configured to excite a linear predictive coding (LPC) filter. The voice decoder can further generate, using the LPC filter based on the excitation signal, at least one sample of a reconstructed audio signal. For example, the neural network can generate coefficients for one or more linear time-varying filters (e.g., a linear time-varying harmonic filter and a linear time-varying noise filter). The voice decoder can use the one or more linear time-varying filters including the generated coefficients to generate the excitation signal.

Abstract: Systems and techniques are provided for processing image data. According to some aspects, a computing device can determine an optical flow between a current frame having a first resolution and a first previous frame having the first resolution. The computing device can warp a second previous frame having a second resolution based on the determined optical flow to generate a warped previous frame having the second resolution, the second resolution being higher than the first resolution. The computing device can process, using a diffusion machine learning model, a noise frame, the current frame, and the warped previous frame to generate an output frame having the second resolution.

Abstract: Systems and techniques are described for audio coding. An audio system receives feature(s) corresponding an audio signal, for example from an encoder and/or a speech synthesis engine. The audio system generates an excitation signal, such as a harmonic signal and/or a noise signal, based on the feature(s). The audio system uses a filterbank to generate band-specific signals from the excitation signal. The band-specific signals correspond to frequency bands. The audio system inputs the feature(s) into a machine learning (ML) filter estimator to generate parameter(s) associated with linear filter(s). The audio system inputs the feature(s) into a voicing estimator to generate gain value(s). The audio system generates an output audio signal based on modification of the band-specific signals, application of the linear filter(s) according to the parameter(s), and amplification using the gain amplifier(s) according to the gain value(s).

Abstract: Systems and techniques are described herein for processing video data. For example, a machine-learning based stereo video coding system can obtain video data including at least a right-view image of a right view of a scene and a left-view image of a left view of the scene. The machine-learning based stereo video coding system can compress the right-view image and the left-view image in parallel to generate a latent representation of the right-view image and the left-view image. The right-view image and the left-view image can be compressed in parallel based on inter-view information between the right-view image and the left-view image, determined using one or more parallel autoencoders.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for encoding content using a neural network. An example method generally includes encoding video content into a latent space representation through an encoder implemented by a first machine learning model. A code is generated by upsampling the latent space representation of the video content. A prior is calculated based on a conditional probability of obtaining the upsampled latent space representation conditioned by the latent space representation of the video content. A compressed version of the video content is generated based on a probabilistic model implemented by a second machine learning model, the generated code, and the calculated prior, and the compressed version of the video content is output for transmission.

Abstract: An example computing device may include memory and one or more processors. The one or more processors may be configured to parallel entropy decode encoded video data from a received bitstream to generate entropy decoded data. The one or more processors may be configured to predict a motion vector based on the entropy decoded data. The one or more processors may be configured to decode a motion vector residual from the entropy decoded data. The one or more processors may be configured to add the motion vector residual and motion vector. The one or more processors may be configured to warp previous reconstructed video data with an overlapped block-based warp function using the motion vector to generate predicted current video data. The one or more processors may be configured to sum the predicted current video data with a residual block to generate current reconstructed video data.

Abstract: Systems and techniques are described for processing image data using a residual model that can be configured with an adjustable number of sampling steps. For example, a process can include obtaining a latent representation of an image and processing, using a decoder of a machine learning model, the latent representation of the image to generate an initial reconstructed image. The process can further include processing, using the residual model, the initial reconstructed image and noise data to predict a plurality of predictions of a residual over a number of sampling steps. The residual represents a difference between the image and the initial reconstructed image. The process can include obtaining, from the plurality of predictions of the residual, a final residual representing the difference between the image and the initial reconstructed image. The process can further include combining the initial reconstructed image and the residual to generate a final reconstructed image.

Abstract: A device includes one or more processors configured to generate, at an encoder portion of an autoencoder, first output data at least partially based on first input data and to generate, at a decoder portion or the autoencoder, a representation of the first input data at least partially based on the first output data. The one or more processors are configured to generate, at the encoder portion, second output data based on second input data and first state data and to generate, at the decoder portion, a representation of the second input data based on the second output data and second state data. Each of the first state data and the second state data correspond to the state of the decoder portion resulting from generation of the representation of the first input data. The first and second input data correspond to sequential values of a signal to be encoded.

Abstract: Techniques are described for coding audio signals. For example, using a neural network, a residual signal is generated for a sample of an audio signal based on inputs to the neural network. The residual signal is configured to excite a long-term prediction filter and/or a short-term prediction filter. Using the long-term prediction filter and/or the short-term prediction filter, a sample of a reconstructed audio signal is determined. The sample of the reconstructed audio signal is determined based on the residual signal generated using the neural network for the sample of the audio signal.

Abstract: Techniques are described herein for coding video content using recurrent-based machine learning tools. A device can include a neural network system including encoder and decoder portions. The encoder portion can generate output data for the current time step of operation of the neural network system based on an input video frame for a current time step of operation of the neural network system, reconstructed motion estimation data from a previous time step of operation, reconstructed residual data from the previous time step of operation, and recurrent state data from at least one recurrent layer of a decoder portion of the neural network system from the previous time step of operation. A decoder portion of the neural network system can generate, based on the output data and recurrent state data from the previous time step of operation, a reconstructed video frame for the current time step of operation.

Abstract: Techniques are described herein for coding video content using recurrent-based machine learning tools. A device can include a neural network system including encoder and decoder portions. The encoder portion can generate output data for the current time step of operation of the neural network system based on an input video frame for a current time step of operation of the neural network system, reconstructed motion estimation data from a previous time step of operation, reconstructed residual data from the previous time step of operation, and recurrent state data from at least one recurrent layer of a decoder portion of the neural network system from the previous time step of operation. A decoder portion of the neural network system can generate, based on the output data and recurrent state data from the previous time step of operation, a reconstructed video frame for the current time step of operation.

Abstract: A device includes one or more processors configured to generate, at an encoder portion of an autoencoder, first output data at least partially based on first input data and to generate, at a decoder portion or the autoencoder, a representation of the first input data at least partially based on the first output data. The one or more processors are configured to generate, at the encoder portion, second output data based on second input data and first state data and to generate, at the decoder portion, a representation of the second input data based on the second output data and second state data. Each of the first state data and the second state data correspond to the state of the decoder portion resulting from generation of the representation of the first input data. The first and second input data correspond to sequential values of a signal to be encoded.

Abstract: Techniques are described for coding audio signals. For example, using a neural network, a residual signal is generated for a sample of an audio signal based on inputs to the neural network. The residual signal is configured to excite a long-term prediction filter and/or a short-term prediction filter. Using the long-term prediction filter and/or the short-term prediction filter, a sample of a reconstructed audio signal is determined. The sample of the reconstructed audio signal is determined based on the residual signal generated using the neural network for the sample of the audio signal.