Abstract: There is disclosed a computer-implemented method for training a neural network. The method comprises determining a gradient associated with a parameter of the neural network. The method further comprises determining a ratio of a gradient norm to parameter norm and comparing the ratio to a threshold. In response to determining that the ratio exceeds the threshold, the value of the gradient is reduced such that the ratio is equal to or below the threshold. The value of the parameter is updated based upon the reduced gradient value.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for privacy-sensitive training of a neural network. In one aspect, a method includes training a set of neural network parameters of the neural network on a set of training data over multiple training iterations to optimize an objective function. Each training iteration includes: sampling a batch of network inputs from the set of training data; determining a clipped gradient for each network input in the batch of network inputs; and updating the neural network parameters using the clipped gradients for the network inputs in the batch of network inputs.

Abstract: Computer-implemented methods for determining missing content in a database are provided. The database may contain a plurality of known embedded sentences and their relationship to content. In one aspect, a method includes receiving new queries and generating new embedded sentences from said new queries. The method also includes determining whether the new embedded sentences are similar to known embedded sentences. The method also includes generating a message indicating that new embedded sentence is not linked to content. Systems are also provided.

Abstract: Computer-implemented methods for retrieving content in response to receiving a natural language query are provided. In one aspect, a method includes receiving a natural language query submitted by a user using a user interface, generating an embedded sentence from said query, determining a similarity between the embedded sentence derived from the received natural language query and embedded sentences from queries saved in a database comprising a fixed mapping of responses to saved queries expressed as the embedded sentences, retrieving a response for an embedded sentence determined to be similar to one of the saved queries, and providing the response to the user via the user interface. Systems are also provided.