Abstract: The present disclosure is directed to generative models for datasets constrained by marginal constraints. One method includes receiving a request to generate a target dataset based on a marginal constraint for a source dataset. A first object occurs at a source frequency in the source dataset. The marginal constraint indicates a target frequency for the first object. The source dataset encodes a set of co-occurrence frequencies for a plurality of object pairs. A source generative model is accessed. The source generative model includes a first module and a second module that are trained on the source dataset. The second module is updated based on the marginal constraint. An adapted generative model is generated that includes the first module and the updated second module. The target dataset is generated based on the adapted generative model. The first object occurs at the target frequency in the target dataset. The target dataset encodes the set of co-occurrence frequencies for the plurality of object pairs.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating data defining a graph. In one aspect, a method comprises: sequentially generating a respective edge set for each node in the graph, wherein for each of a plurality of nodes after a first node, generating the edge set for the node comprises: receiving a context embedding for the node that summarizes a respective edge set for each node that precedes the node; generating, based on the context embedding for the node: (i) a respective edge set for the node, and (ii) a respective embedding of the edge set for the node; generating a context embedding for a next node in the ordering of the nodes using the embedding of the edge set for the node; and adding the set of edges defined by the edge set for the node to the graph.

Abstract: Example embodiments of aspects of the present disclosure provide an example computer-implemented method for improved prompting of a machine-learned model. The example method can include obtaining an instructive sequence descriptive of an instructive query, an instructive response, and an instructive trace of intermediate states from the instructive query to the instructive response. The example method can include inputting, to a machine-learned model, the instructive sequence and an operative query, wherein the machine-learned model is configured to process the operative query with attention over the instructive sequence. The example method can include generating, using the machine-learned model and responsive to the operative query, an operative response.

Abstract: Provided are computing systems, methods, and platforms for negative sampling in knowledge graphs with improved efficiency. A knowledge graph comprising entities and links between the entities can be obtained. A query computation graph comprising nodes and edges can be generated based on the knowledge graph. The nodes of the query computation graph can include anchor nodes, a root node, and intermediate nodes positioned in paths between the anchor nodes and the root node. A node cut of a query of the query computation graph can be determined and can include at least one node that cuts at least one path between each anchor node and the root node of the query computation graph. Negative samples can be identified by bidirectionally traversing the query computation graph in a first direction from the anchor nodes to the node cut and in a second direction from the root node to the node cut.

Abstract: An example method for pretraining a machine-learned model is provided. The example method includes obtaining a plurality of different combinations of configuration parameters of a pretraining objective framework. The example method includes generating, using the pretraining objective framework, a plurality of corrupted training examples from one or more training examples, wherein the plurality of corrupted training examples are respectively generated according to the plurality of different combinations. The example method includes inputting the plurality of corrupted training examples into the machine-learned model, wherein the machine-learned model is configured to generate uncorrupted subportions corresponding to corrupted subportions of the corrupted training examples. The example method includes obtaining, from the machine-learned model, a plurality of outputs respectively generated by the machine-learned model based on the plurality of corrupted training examples.

Abstract: The present disclosure is directed to machine learning model architectures which provide full attention capability in each attention head while maintaining low computation and memory complexity. Specifically, according to one aspect of the present disclosure, example attention models provided herein can treat the self-attention mechanism as a conditional expectation over embeddings at each location and approximate the conditional distribution with a structured factorization. Each location can attend to all other locations, either via direct attention, or through indirect attention to group representations, which are again conditional expectations of embeddings from corresponding local regions.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating data defining a graph. In one aspect, a method comprises: sequentially generating a respective edge set for each node in the graph, wherein for each of a plurality of nodes after a first node, generating the edge set for the node comprises: receiving a context embedding for the node that summarizes a respective edge set for each node that precedes the node; generating, based on the context embedding for the node: (i) a respective edge set for the node, and (ii) a respective embedding of the edge set for the node; generating a context embedding for a next node in the ordering of the nodes using the embedding of the edge set for the node; and adding the set of edges defined by the edge set for the node to the graph.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generative modelling of an exchangeable sets. Methods can include obtaining a dataset of training observations. Each training observation is an exchangeable set that includes a plurality of data points. Each training observations is processed using a first neural network to generate parameters of a first probability distribution based on which a latent variable is sampled. The latent variable is processed using a second neural network to generate a new observation that includes a plurality of data points. The training observation and the new observation is processed using an energy neural network to generate an estimate of an energy of the training observation and the new observation. The energy neural network is then trained to optimize an objective function that measures the difference between the estimate of the energy of the training observation and the new observation.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network used to select actions to be performed by a reinforcement learning agent interacting with an environment. In one aspect, a method includes obtaining path data defining a path through the environment traversed by the agent. A consistency error is determined for the path from a combined reward, first and last soft-max state values, and a path likelihood. A value update for the current values of the policy neural network parameters is determined from at least the consistency error. The value update is used to adjust the current values of the policy neural network parameters.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining a control policy for an agent interacting with an environment. One of the methods includes updating the control policy using policy-consistent backups using Q learning. To determine a policy-consistent backup, the system determining a policy-consistent backup for the control policy at the current observation—current action pair, comprising: for each of a plurality of actions in a set of possible actions that can be performed by the agent, identifying Q values assigned by the control policy to next observation—action pairs by the control policy and justified by at least one of the information sets; pruning, from the identified Q values, any Q values that are justified only by information sets that are not policy-class consistent; and determining, from the reward and only the identified Q values that were not pruned, the policy-consistent backup.

Abstract: Methods and systems for learning policies using sparse and underspecified rewards. One of the methods includes training the policy jointly with an auxiliary reward function having a plurality of auxiliary reward parameters, the auxiliary reward function being configured to map, in accordance with the auxiliary reward parameters, trajectory features of at least a trajectory to an auxiliary reward value that indicates how well the trajectory performed a task in response to a context input.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network used to select actions to be performed by a reinforcement learning agent interacting with an environment. In one aspect, a method includes obtaining path data defining a path through the environment traversed by the agent. A consistency error is determined for the path from a combined reward, first and last soft-max state values, and a path likelihood. A value update for the current values of the policy neural network parameters is determined from at least the consistency error. The value update is used to adjust the current values of the policy neural network parameters.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network used to select actions to be performed by a reinforcement learning agent interacting with an environment. In one aspect, a method includes obtaining path data defining a path through the environment traversed by the agent. A consistency error is determined for the path from a combined reward, first and last soft-max state values, and a path likelihood. A value update for the current values of the policy neural network parameters is determined from at least the consistency error. The value update is used to adjust the current values of the policy neural network parameters.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network used to select actions to be performed by a reinforcement learning agent interacting with an environment. In one aspect, a method includes obtaining path data defining a path through the environment traversed by the agent. A consistency error is determined for the path from a combined reward, first and last soft-max state values, and a path likelihood. A value update for the current values of the policy neural network parameters is determined from at least the consistency error. The value update is used to adjust the current values of the policy neural network parameters.

Abstract: A method includes obtaining data identifying a machine learning model to be trained to perform a machine learning task, the machine learning model being configured to receive an input example and to process the input example in accordance with current values of a plurality of model parameters to generate a model output for the input example; obtaining initial training data for training the machine learning model, the initial training data comprising a plurality of training examples and, for each training example, a ground truth output that should be generated by the machine learning model by processing the training example; generating modified training data from the initial training data; and training the machine learning model on the modified training data.