Abstract: Certain aspects of the present disclosure provide techniques for compressing content using a neural network. An example method generally includes receiving content for compression. The content is encoded into a first latent code space through an encoder implemented by an artificial neural network trained to generate a latent space representation of the content. A first compressed version of the encoded content is generated using a first quantization bin size of a series of quantization bin sizes. A refined compressed version of the encoded content is generated by scaling the first compressed version of the encoded content into one or more second quantization bin sizes smaller than the first quantization bin size, conditioned at least on a value of the first compressed version of the encoded content. The refined compressed version of the encoded content is output for transmission.

Abstract: A method for generating a causal graph includes receiving a data set including observation data and intervention data corresponding to multiple variables. A probability distribution is determined for each variable based on the observation data. A likelihood of including each edge in the graph is computed based on the probability distribution and the intervention data. Each edge is a causal connection between variables of the multiple variables. The graph is generated based on the likelihood of including each edge. The graph may be updated by iteratively repeating the determination of the probability distribution and the computing of the likelihood of including each edge.

Abstract: Certain aspects of the present disclosure are directed to methods and apparatus for compressing video content using deep generative models. One example method generally includes receiving video content for compression. The received video content is generally encoded into a latent code space through an auto-encoder, which may be implemented by a first artificial neural network. A compressed version of the encoded video content is generally generated through a trained probabilistic model, which may be implemented by a second artificial neural network, and output for transmission.

Abstract: A processor-implemented method includes observing an environment via one or more sensors associated with a robotic device. The processor-implemented method also includes generating, via an inference model, a belief of the environment based on data associated with prior actions of the robotic device in the environment. The processor-implemented method further includes controlling the robotic device to perform an action in the environment based on generating the belief.

Abstract: A computer-implemented method for operating an artificial neural network (ANN) includes receiving an input by the ANN. The ANN generates a latent representation of the input. The latent representation is communicated according to a bit rate based on a learned latent scaling parameter. The latent scaling parameter is learned based on a channel index and a tradeoff parameter value that corresponds to a value that balances the bit rate and a distortion.

Abstract: Techniques are described for compressing data using machine learning systems and tuning machine learning systems for compressing the data. An example process can include receiving, by a neural network compression system (e.g., trained on a training dataset), input data for compression by the neural network compression system. The process can include determining a set of updates for the neural network compression system, the set of updates including updated model parameters tuned using the input data. The process can include generating, by the neural network compression system using a latent prior, a first bitstream including a compressed version of the input data. The process can further include generating, by the neural network compression system using the latent prior and a model prior, a second bitstream including a compressed version of the updated model parameters. The process can include outputting the first bitstream and the second bitstream for transmission to a receiver.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for improved machine learning. Input data comprising a plurality of points in multidimensional space is accessed. An edge connecting a first point and a second point of the plurality of points is identified, and the edge is mapped to a defined axis in the multidimensional space by applying a group element to the edge. An intermediate feature is generated by processing the mapped edge using a neural network. An output feature is generated by applying an inverse of the group element to the intermediate feature, and an output inference is generated based at least in part on the output feature.

Abstract: Techniques are described for processing video data, such as by performing learned bidirectional coding using a unidirectional coding system and an interpolated reference frame. For example, a process can include obtaining a first reference frame and a second reference frame. The process can include generating a third reference frame at least in part by performing interpolation between the first reference frame and the second reference frame. The process can include performing unidirectional inter-prediction on an input frame based on the third reference frame, such as by estimating motion between an input frame and the third reference frame, and generating a warped frame at least in part by warping one or more pixels of the third reference frame based on the estimated motion. The process can include generating, based on the warped frame and a predicted residual, a reconstructed frame representing the input frame, the reconstructed frame including a bidirectionally-predicted frame.

Abstract: Techniques are described for processing video data, such as by performing learned bidirectional coding using a unidirectional coding system and an interpolated reference frame. For example, a process can include obtaining a first reference frame and a second reference frame. The process can include generating a third reference frame at least in part by performing interpolation between the first reference frame and the second reference frame. The process can include performing unidirectional inter-prediction on an input frame based on the third reference frame, such as by estimating motion between an input frame and the third reference frame, and generating a warped frame at least in part by warping one or more pixels of the third reference frame based on the estimated motion. The process can include generating, based on the warped frame and a predicted residual, a reconstructed frame representing the input frame, the reconstructed frame including a bidirectionally-predicted frame.

Abstract: A method of image compression includes receiving an image. Multiple quantized latent representations are generated to represent features of the image. Each of the quantized latent representations has a different resolution and is generated at staggered timings. Each of the later generated quantized latent representations is conditioned on each of the prior generated quantized latent representations. The multiple quantized latent representations are decoded to reconstruct the image.

Abstract: Certain aspects of the present disclosure are directed to methods and apparatus for compressing video content using deep generative models. One example method generally includes receiving video content for compression. The received video content is generally encoded into a latent code space through an encoder, which may be implemented by a first artificial neural network. A compressed version of the encoded video content is generally generated through a trained probabilistic model, which may be implemented by a second artificial neural network, and output for transmission.

Abstract: Certain aspects of the present disclosure are directed to methods and apparatus for compressing video content using deep generative models. One example method generally includes receiving video content for compression. The received video content is generally encoded into a latent code space through an encoder, which may be implemented by a first artificial neural network. A compressed version of the encoded video content is generally generated through a trained probabilistic model, which may be implemented by a second artificial neural network, and output for transmission.

Abstract: A processor-implemented method for video compression using an artificial neural network (ANN) includes receiving a video via the ANN. The ANN extracts a first set of features of a current frame of the video and a second set of features of a reference frame of the video. The ANN determines an estimate of correlation features between the first set of features of the current frame and the second set of features of the reference frame. The estimate of the correlation features are encoded and transmitted to a receiver.

Abstract: A processor-implemented method for image compression using an artificial neural network (ANN) includes receiving, at an encoder of the ANN, an image and a spatial segmentation map corresponding to the image. The spatial segmentation map indicates one or more regions of interest. The encoder compresses the image according to a controllable spatial bit allocation. The controllable spatial bit allocation is based on a learned quantization bin size.

Abstract: Systems and techniques are described for encoding and/or decoding data based on motion estimation that applies variable-scale warping. An encoding device can receive an input frame and a reference frame that depict a scene at different times. The encoding device can generate an optical flow identifying movements in the scene between the two frames. The encoding device can generate a weight map identifying how finely or coarsely the reference frame can be warped for input frame prediction. The encoding device can generate encoded video data based on the optical flow and the weight map. A decoding device can generate a reconstructed optical flow and a reconstructed weight map from the encoded data. A decoding device can generate a prediction frame by warping the reference frame based on the reconstructed optical flow and the reconstructed weight map. The decoding device can generate a reconstructed input frame based on the prediction frame.

Abstract: Systems and techniques are described herein for processing media data using a neural network system. For instance, a process can include obtaining a latent representation of a frame of encoded image data and generating, by a plurality of decoder transformer layers of a decoder sub-network using the latent representation of the frame of encoded image data as input, a frame of decoded image data. At least one decoder transformer layer of the plurality of decoder transformer layers includes: one or more transformer blocks for generating one or more patches of features and determine self-attention locally within one or more window partitions and shifted window partitions applied over the one or more patches; and a patch un-merging engine for decreasing a respective size of each patch of the one or more patches.

Abstract: Techniques are described for compressing data using machine learning systems. An example process can include receiving input data for compression by a neural network compression system. The process can include determining, based on the input data, a set of updated model parameters for the neural network compression system, wherein the set of updated model parameters is selected from a subspace of model parameters. The process can include generating at least one bitstream including a compressed version of the input data and a compressed version of one or more subspace coordinates that correspond to the set of updated model parameters. The process can include outputting the at least one bitstream for transmission to a receiver.

Abstract: A device includes one or more processors configured to generate, at an encoder portion of an autoencoder, first output data at least partially based on first input data and to generate, at a decoder portion or the autoencoder, a representation of the first input data at least partially based on the first output data. The one or more processors are configured to generate, at the encoder portion, second output data based on second input data and first state data and to generate, at the decoder portion, a representation of the second input data based on the second output data and second state data. Each of the first state data and the second state data correspond to the state of the decoder portion resulting from generation of the representation of the first input data. The first and second input data correspond to sequential values of a signal to be encoded.

Abstract: Techniques are described for compressing and decompressing data using machine learning systems. An example process can include receiving a plurality of images for compression by a neural network compression system. The process can include determining, based on a first image from the plurality of images, a first plurality of weight values associated with a first model of the neural network compression system. The process can include generating a first bitstream comprising a compressed version of the first plurality of weight values. The process can include outputting the first bitstream for transmission to a receiver.

Abstract: Certain aspects of the present disclosure are directed to methods and apparatus for compressing video content using deep generative models. One example method generally includes receiving video content for compression. The received video content is generally encoded into a latent code space through an auto-encoder, which may be implemented by a first artificial neural network. A compressed version of the encoded video content is generally generated through a trained probabilistic model, which may be implemented by a second artificial neural network, and output for transmission.