Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output sequence of discrete tokens using a diffusion model. In one aspect, a method includes initializing the output sequence by assigning a respective embedding to each of the plurality of output positions; repeatedly performing the following at each of multiple reverse diffusion steps: a current continuous representation of the output sequence; processing a diffusion model input that comprises the current continuous representation using the diffusion model to generate a diffusion model output; processing the respective initial scores using a softmax function to generate, for each of the plurality of output positions, a probability distribution over the plurality of embeddings in the vocabulary of embeddings; and updating the continuous representation of the output sequence using the probability distributions and the vocabulary of embeddings.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output sequence that includes a respective token selected from a vocabulary of tokens at each of multiple output positions. In one aspect, one of the methods includes obtaining an initial output sequence, the initial output sequence comprising a mask token at each of at least a subset of the multiple output positions; repeatedly performing the following at each of multiple update iterations: obtaining an intermediate representation of the output sequence; generate a diffusion model output that comprises, for each of the multiple output positions, a respective score for each token in at least a subset of the vocabulary of tokens; determining, for each output position in the output sequence that is occupied by a mask token, a masked probability; selecting a subset of the multiple output positions; and generating an updated intermediate representation.

Abstract: Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for training a classification machine-learning model. The system obtains calibration training examples and prediction training examples, determines a threshold value based on the calibration training examples, generates data characterizing predicted confidence sets based on the threshold value and the prediction training examples, and update model parameters based at least on the predicted confidence sets.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating one or more differentiable order statistics for a vector of scores. In one aspect, a method comprises: obtaining the vector of scores, wherein each position in the vector of scores is associated with a respective index from a set of indices; obtaining a plurality of pairs of indices; generating a respective swapping probability for each pair of indices based on the vector of scores; generating, for each pair of indices, a respective soft-swapping matrix for the pair of indices as a combination of: (i) an identity matrix, and (ii) an exchange matrix, wherein the exchange matrix is weighted in the combination by the swapping probability for the pair of indices; and generating the one or more differentiable order statistics for the vector of scores using the soft-swapping matrices.