Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a sequence of data elements that includes a respective data element at each position in a sequence of positions. In one aspect, a method includes: for each position after a first position in the sequence of positions: obtaining a current sequence of data element embeddings that includes a respective data element embedding of each data element at a position that precedes the current position, obtaining a sequence of latent embeddings, and processing: (i) the current sequence of data element embeddings, and (ii) the sequence of latent embeddings, using a neural network to generate the data element at the current position. The neural network includes a sequence of neural network blocks including: (i) a cross-attention block, (ii) one or more self-attention blocks, and (iii) an output block.

Abstract: Implementations are described herein for training and using machine learning to determine mappings between matching nodes of graphs representing predecessor source code snippets and graphs representing successor source code snippets. In various implementations, first and second graphs may be obtained, wherein the first graph represents a predecessor source code snippet and the second graph represents a successor source code snippet. The first graph and the second graph may be applied as inputs across a trained machine learning model to generate node similarity measures between individual nodes of the first graph and nodes of the second graph. Based on the node similarity measures, a mapping may be determined across the first and second graphs between pairs of matching nodes.

Abstract: Implementations are described herein for training and using machine learning to determine mappings between matching nodes of graphs representing predecessor source code snippets and graphs representing successor source code snippets. In various implementations, first and second graphs may be obtained, wherein the first graph represents a predecessor source code snippet and the second graph represents a successor source code snippet. The first graph and the second graph may be applied as inputs across a trained machine learning model to generate node similarity measures between individual nodes of the first graph and nodes of the second graph. Based on the node similarity measures, a mapping may be determined across the first and second graphs between pairs of matching nodes.