Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing a sequence-to-sequence model that is recurrent in depth while employing self-attention to combine information from different parts of sequences.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing a sequence to sequence model that is recurrent in depth while employing self-attention to combine information from different parts of sequences.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for semantically-consistent image style transfer. One of the methods includes: receiving an input source domain image; processing the source domain image using one or more source domain low-level encoder neural network layers to generate a low-level representation; processing the low-level representation using one more high-level encoder neural network layers to generate an embedding of the input source domain image; processing the embedding using one or more high-level decoder neural network layers to generate a high-level feature representation of features of the input source domain image; and processing the high-level feature representation of the features of the input source domain image using one or more target domain low-level decoder neural network layers to generate an output target domain image that is from the target domain but that has similar semantics to the input source domain image.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining a predicted structure of a protein that is specified by an amino acid sequence. In one aspect, a method comprises: obtaining an initial embedding and initial values of structure parameters for each amino acid in the amino acid sequence, wherein the structure parameters for each amino acid comprise location parameters that specify a predicted three-dimensional spatial location of the amino acid in the structure of the protein; and processing a network input comprising the initial embedding and the initial values of the structure parameters for each amino acid in the amino acid sequence using a folding neural network to generate a network output comprising final values of the structure parameters for each amino acid in the amino acid sequence.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing a sequence to sequence model that is recurrent in depth while employing self-attention to combine information from different parts of sequences.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for semantically-consistent image style transfer. One of the methods includes: receiving an input source domain image; processing the source domain image using one or more source domain low-level encoder neural network layers to generate a low-level representation; processing the low-level representation using one more high-level encoder neural network layers to generate an embedding of the input source domain image; processing the embedding using one or more high-level decoder neural network layers to generate a high-level feature representation of features of the input source domain image; and processing the high-level feature representation of the features of the input source domain image using one or more target domain low-level decoder neural network layers to generate an output target domain image that is from the target domain but that has similar semantics to the input source domain image.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing a sequence to sequence model that is recurrent in depth while employing self-attention to combine information from different parts of sequences.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing a sequence to sequence model that is recurrent in depth while employing self-attention to combine information from different parts of sequences.