Abstract: Methods, systems, and apparatuses, including computer programs encoded on computer storage media, for generating a summary of a set of content items using a language model neural network. In particular, the described techniques include processing data characterizing a set of content items and the respective relative prominence data for each of the set of content items using a language model neural network to generate the summary of the set of content items. Because the relative prominence data is included in the input to the language model neural network, the summary will reflect the relative prominence represented for each of the content items.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent interacting with an environment. In one aspect, a method comprises: receiving an observation that characterizes the environment; receiving a conditioning input that characterizes a task to be performed by the agent in the environment; for each of a plurality of sub-regions of the observation, generating an observation patch embedding of the sub-region; generating a conditioning input embedding of the conditioning input; processing the observation patch embeddings and the conditioning input embedding to generate a policy output that defines an action to be performed by the agent in response to the observation, wherein the processing comprises applying a linear attention mechanism over the observation patch embeddings and the conditioning input embedding; selecting an action to be performed by the agent using the policy output; and causing the agent to perform the selected action.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating descriptions of digital components. In one aspect, a method includes receiving data indicating a query received from a client device of a user. An initial digital component is obtained. Search history data that includes a set of related past queries received from the user is obtained. Updated text related to the first resource is generated by conditioning a language model with one or more contextual inputs that cause the language model to generate one or more outputs that include the updated text, the one or more contextual inputs characterizing one or more of the first query, data related to the initial digital component, the sequence of related past queries, or one or more tasks to be performed by the language model. An updated digital component that depicts the updated text is generated and provided.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training neural networks. One of the methods for training a neural network configured to perform a machine learning task includes performing, at each of a plurality of iterations: performing a training step to obtain respective new gradients of a loss function; for each network parameter: generating an optimizer network input; processing the optimizer network input using an optimizer neural network, wherein the processing comprises, for each cell: generating a cell input for the cell; and processing the cell input for the cell to generate a cell output, wherein the processing comprises: obtaining latent embeddings from the cell input; generating the cell output from the hidden state; and determining an update to the hidden state; and generating an optimizer network output defining an update for the network parameter; and applying the update to the network parameter.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing network inputs using an attention neural network that has one or more sparse attention sub-layers. Each sparse attention sub-layer is configured to apply a sparse attention mechanism that attends differently for input positions that are in a first proper subset of the input positions in the input to the sub-layer than for positions that are not in the first proper subset.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing network inputs using an attention neural network that has one or more sparse attention sub-layers. Each sparse attention sub-layer is configured to apply a sparse attention mechanism that attends differently for input positions that are in a first proper subset of the input positions in the input to the sub-layer than for positions that are not in the first proper subset.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing network inputs using an attention neural network that has one or more sparse attention sub-layers. Each sparse attention sub-layer is configured to apply a sparse attention mechanism that attends differently for input positions that are in a first proper subset of the input positions in the input to the sub-layer than for positions that are not in the first proper subset.

Abstract: Systems and associated methods provide a cluster-level semi-supervision model for inter-active clustering. Embodiments accept user provided semi-supervision for updating cluster descriptions and assignment of data items to clusters. Assignment feedback re-assigns data items among existing clusters, while cluster description feedback helps to position existing cluster centers more meaningfully. The feedback can continue until the user is satisfied with the clustering achieved or one or more predetermined stopping criteria have been reached.

Abstract: Systems and associated methods provide a cluster-level semi-supervision model for inter-active clustering. Embodiments accept user provided semi-supervision for updating cluster descriptions and assignment of data items to clusters. Assignment feedback re-assigns data items among existing clusters, while cluster description feedback helps to position existing cluster centers more meaningfully. The feedback can continue until the user is satisfied with the clustering achieved or one or more predetermined stopping criteria have been reached.