Patents by Inventor Sven Adrian Gowal
Sven Adrian Gowal has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11847414Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a text classification machine learning model. One of the methods includes training a model having a plurality of parameters and configured to generate a classification of a text sample comprising a plurality of words by processing a model input that includes a combined feature representation of the plurality of words in the text sample, wherein the training comprises receiving a text sample and a target classification for the text sample; generating a plurality of perturbed combined feature representations; determining, based on the plurality of perturbed combined feature representations, a region in the embedding space; and determining an update to the parameters based on an adversarial objective that encourages the model to assign the target classification for the text sample for all of the combined feature representations in the region in the embedding space.Type: GrantFiled: April 23, 2021Date of Patent: December 19, 2023Assignee: DeepMind Technologies LimitedInventors: Krishnamurthy Dvijotham, Anton Zhernov, Sven Adrian Gowal, Conrad Grobler, Robert Stanforth
-
Publication number: 20230316729Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for processing a network input using a trained neural network with network parameters to generate an output for a machine learning task. The training includes: receiving a set of training examples each including a training network input and a reference output; for each training iteration, generating a corrupted network input for each training network input using a corruption neural network; updating perturbation parameters of the corruption neural network using a first objective function based on the corrupted network inputs; generating an updated corrupted network input for each training network input based on the updated perturbation parameters; and generating a network output for each updated corrupted network input using the neural network; for each training example, updating the network parameters using a second objective function based on the network output and the reference output.Type: ApplicationFiled: April 1, 2022Publication date: October 5, 2023Inventors: Dan-Andrei Calian, Sven Adrian Gowal, Timothy Arthur Mann, András György
-
Patent number: 11775830Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes processing each training input using the neural network and in accordance with the current values of the network parameters to generate a network output for the training input; computing a respective loss for each of the training inputs by evaluating a loss function; identifying, from a plurality of possible perturbations, a maximally non-linear perturbation; and determining an update to the current values of the parameters of the neural network by performing an iteration of a neural network training procedure to decrease the respective losses for the training inputs and to decrease the non-linearity of the loss function for the identified maximally non-linear perturbation.Type: GrantFiled: December 12, 2022Date of Patent: October 3, 2023Assignee: DeepMind Technologies LimitedInventors: Chongli Qin, Sven Adrian Gowal, Soham De, Robert Stanforth, James Martens, Krishnamurthy Dvijotham, Dilip Krishnan, Alhussein Fawzi
-
Publication number: 20230252286Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes processing each training input using the neural network and in accordance with the current values of the network parameters to generate a network output for the training input; computing a respective loss for each of the training inputs by evaluating a loss function; identifying, from a plurality of possible perturbations, a maximally non-linear perturbation; and determining an update to the current values of the parameters of the neural network by performing an iteration of a neural network training procedure to decrease the respective losses for the training inputs and to decrease the non-linearity of the loss function for the identified maximally non-linear perturbation.Type: ApplicationFiled: December 12, 2022Publication date: August 10, 2023Inventors: Chongli Qin, Sven Adrian Gowal, Soham De, Robert Stanforth, James Martens, Krishnamurthy Dvijotham, Dilip Krishnan, Alhussein Fawzi
-
Publication number: 20230244912Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for learning from delayed outcomes using neural networks. One of the methods includes receiving an input observation; generating, from the input observation, an output label distribution over possible labels for the input observation at a final time, comprising: processing the input observation using a first neural network configured to process the input observation to generate a distribution over possible values for an intermediate indicator at a first time earlier than the final time; generating, from the distribution, an input value for the intermediate indicator; and processing the input value for the intermediate indicator using a second neural network configured to process the input value for the intermediate indicator to determine the output label distribution over possible values for the input observation at the final time; and providing an output derived from the output label distribution.Type: ApplicationFiled: April 6, 2023Publication date: August 3, 2023Inventors: Huiyi Hu, Ray Jiang, Timothy Arthur Mann, Sven Adrian Gowal, Balaji Lakshminarayanan, András György
-
Patent number: 11714994Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for learning from delayed outcomes using neural networks. One of the methods includes receiving an input observation; generating, from the input observation, an output label distribution over possible labels for the input observation at a final time, comprising: processing the input observation using a first neural network configured to process the input observation to generate a distribution over possible values for an intermediate indicator at a first time earlier than the final time; generating, from the distribution, an input value for the intermediate indicator; and processing the input value for the intermediate indicator using a second neural network configured to process the input value for the intermediate indicator to determine the output label distribution over possible values for the input observation at the final time; and providing an output derived from the output label distribution.Type: GrantFiled: March 11, 2019Date of Patent: August 1, 2023Assignee: DeepMind Technologies LimitedInventors: Huiyi Hu, Ray Jiang, Timothy Arthur Mann, Sven Adrian Gowal, Balaji Lakshminarayanan, András György
-
Patent number: 11526755Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes processing each training input using the neural network and in accordance with the current values of the network parameters to generate a network output for the training input; computing a respective loss for each of the training inputs by evaluating a loss function; identifying, from a plurality of possible perturbations, a maximally non-linear perturbation; and determining an update to the current values of the parameters of the neural network by performing an iteration of a neural network training procedure to decrease the respective losses for the training inputs and to decrease the non-linearity of the loss function for the identified maximally non-linear perturbation.Type: GrantFiled: May 22, 2020Date of Patent: December 13, 2022Assignee: DeepMind Technologies LimitedInventors: Chongli Qin, Sven Adrian Gowal, Soham De, Robert Stanforth, James Martens, Krishnamurthy Dvijotham, Dilip Krishnan, Alhussein Fawzi
-
Publication number: 20210334459Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a text classification machine learning model. One of the methods includes training a model having a plurality of parameters and configured to generate a classification of a text sample comprising a plurality of words by processing a model input that includes a combined feature representation of the plurality of words in the text sample, wherein the training comprises receiving a text sample and a target classification for the text sample; generating a plurality of perturbed combined feature representations; determining, based on the plurality of perturbed combined feature representations, a region in the embedding space; and determining an update to the parameters based on an adversarial objective that encourages the model to assign the target classification for the text sample for all of the combined feature representations in the region in the embedding space.Type: ApplicationFiled: April 23, 2021Publication date: October 28, 2021Inventors: Krishnamurthy Dvijotham, Anton Zhernov, Sven Adrian Gowal, Conrad Grobler, Robert Stanforth
-
Publication number: 20190279076Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for learning from delayed outcomes using neural networks. One of the methods includes receiving an input observation; generating, from the input observation, an output label distribution over possible labels for the input observation at a final time, comprising: processing the input observation using a first neural network configured to process the input observation to generate a distribution over possible values for an intermediate indicator at a first time earlier than the final time; generating, from the distribution, an input value for the intermediate indicator; and processing the input value for the intermediate indicator using a second neural network configured to process the input value for the intermediate indicator to determine the output label distribution over possible values for the input observation at the final time; and providing an output derived from the output label distribution.Type: ApplicationFiled: March 11, 2019Publication date: September 12, 2019Inventors: Huiyi Hu, Ray Jiang, Timothy Arthur Mann, Sven Adrian Gowal, Balaji Lakshminarayanan, Andras Gyorgy