Patents Assigned to DeepMind Technologies
  • Patent number: 12147899
    Abstract: Methods, systems and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network. One of the methods includes receiving an observation characterizing a current state of the environment; determining a target network output for the observation by performing a look ahead search of possible future states of the environment starting from the current state until the environment reaches a possible future state that satisfies one or more termination criteria, wherein the look ahead search is guided by the neural network in accordance with current values of the network parameters; selecting an action to be performed by the agent in response to the observation using the target network output generated by performing the look ahead search; and storing, in an exploration history data store, the target network output in association with the observation for use in updating the current values of the network parameters.
    Type: Grant
    Filed: December 4, 2023
    Date of Patent: November 19, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Karen Simonyan, David Silver, Julian Schrittwieser
  • Patent number: 12141677
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for prediction of an outcome related to an environment. In one aspect, a system comprises a state representation neural network that is configured to: receive an observation characterizing a state of an environment being interacted with by an agent and process the observation to generate an internal state representation of the environment state; a prediction neural network that is configured to receive a current internal state representation of a current environment state and process the current internal state representation to generate a predicted subsequent state representation of a subsequent state of the environment and a predicted reward for the subsequent state; and a value prediction neural network that is configured to receive a current internal state representation of a current environment state and process the current internal state representation to generate a value prediction.
    Type: Grant
    Filed: June 25, 2020
    Date of Patent: November 12, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: David Silver, Tom Schaul, Matteo Hessel, Hado Philip van Hasselt
  • Patent number: 12141691
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output examples using neural networks. Each output example includes multiple N-bit output values. To generate a given N-bit output value, a first recurrent input comprising the preceding N-bit output value is processed using a recurrent neural network and in accordance with a hidden state to generate a first score distribution. Then, values for the first half of the N bits are selected. A second recurrent input comprising (i) the preceding N-bit output value and (ii) the values for the first half of the N bits are processed using the recurrent neural network and in accordance with the same hidden state to generate a second score distribution. The values for the second half of the N bits of the output value are then selected using the second score distribution.
    Type: Grant
    Filed: February 11, 2019
    Date of Patent: November 12, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Nal Emmerich Kalchbrenner, Karen Simonyan, Erich Konrad Elsen
  • Patent number: 12131243
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating data specifying a three-dimensional mesh of an object using an auto-regressive neural network.
    Type: Grant
    Filed: February 8, 2021
    Date of Patent: October 29, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Charlie Thomas Curtis Nash, Iaroslav Ganin, Seyed Mohammadali Eslami, Peter William Battaglia
  • Patent number: 12131248
    Abstract: There is described a neural network system for generating a graph, the graph comprising a set of nodes and edges. The system comprises one or more neural networks configured to represent a probability distribution over sequences of node generating decisions and/or edge generating decisions, and one or more computers configured to sample the probability distribution represented by the one or more neural networks to generate a graph.
    Type: Grant
    Filed: May 8, 2023
    Date of Patent: October 29, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Yujia Li, Christopher James Dyer, Oriol Vinyals
  • Patent number: 12124938
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for learning from delayed outcomes using neural networks. One of the methods includes receiving an input observation; generating, from the input observation, an output label distribution over possible labels for the input observation at a final time, comprising: processing the input observation using a first neural network configured to process the input observation to generate a distribution over possible values for an intermediate indicator at a first time earlier than the final time; generating, from the distribution, an input value for the intermediate indicator; and processing the input value for the intermediate indicator using a second neural network configured to process the input value for the intermediate indicator to determine the output label distribution over possible values for the input observation at the final time; and providing an output derived from the output label distribution.
    Type: Grant
    Filed: April 6, 2023
    Date of Patent: October 22, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Huiyi Hu, Ray Jiang, Timothy Arthur Mann, Sven Adrian Gowal, Balaji Lakshminarayanan, András György
  • Patent number: 12100477
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining a predicted structure of a protein that is specified by an amino acid sequence. In one aspect, a method comprises: obtaining a multiple sequence alignment for the protein; determining, from the multiple sequence alignment and for each pair of amino acids in the amino acid sequence of the protein, a respective initial embedding of the pair of amino acids; processing the initial embeddings of the pairs of amino acids using a pair embedding neural network comprising a plurality of self-attention neural network layers to generate a final embedding of each pair of amino acids; and determining the predicted structure of the protein based on the final embedding of each pair of amino acids.
    Type: Grant
    Filed: December 1, 2020
    Date of Patent: September 24, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: John Jumper, Andrew W. Senior, Richard Andrew Evans, Russell James Bates, Mikhail Figurnov, Alexander Pritzel, Timothy Frederick Goldie Green
  • Patent number: 12099928
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the methods includes providing an output derived from the neural network output for the time step as a system output for the time step; maintaining a current state of the external memory; determining, from the neural network output for the time step, memory state parameters for the time step; updating the current state of the external memory using the memory state parameters for the time step; reading data from the external memory in accordance with the updated state of the external memory; and combining the data read from the external memory with a system input for the next time step to generate the neural network input for the next time step.
    Type: Grant
    Filed: February 24, 2023
    Date of Patent: September 24, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Edward Thomas Grefenstette, Karl Moritz Hermann, Mustafa Suleyman, Philip Blunsom
  • Patent number: 12094474
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for verifying the provenance of a digital object generated by a neural network, such as an image or audio object. Also methods, systems, and apparatus, including computer programs, for training a watermarking neural network and a watermark decoding neural network. The described techniques make efficient use of computing resources and are robust to attack.
    Type: Grant
    Filed: November 15, 2023
    Date of Patent: September 17, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Sven Adrian Gowal, Christopher Gamble, Florian Nils Stimberg, Sylvestre-Alvise Guglielmo Rebuffi, Sree Meghana Thotakuri, Jamie Hayes, Ian Goodfellow, Rudy Bunel, Miklós Zsigmond Horváth, David Stutz, Olivia Anne Wiles
  • Patent number: 12088823
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for encoding video comprising a sequence of video frames. In one aspect, a method comprises for one or more of the video frames: obtaining a feature embedding for the video frame; processing the feature embedding using a rate control machine learning model to generate a respective score for each of multiple quantization parameter values; selecting a quantization parameter value using the scores; determining a cumulative amount of data required to represent: (i) an encoded representation of the video frame and (ii) encoded representations of each preceding video frame; determining, based on the cumulative amount of data, that a feedback control criterion for the video frame is satisfied; updating the selected quantization parameter value; and processing the video frame using an encoding model to generate the encoded representation of the video frame.
    Type: Grant
    Filed: November 3, 2021
    Date of Patent: September 10, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Chenjie Gu, Hongzi Mao, Ching-Han Chiang, Cheng Chen, Jingning Han, Ching Yin Derek Pang, Rene Andre Claus, Marisabel Guevara Hechtman, Daniel James Visentin, Christopher Sigurd Fougner, Charles Booth Schaff, Nishant Patil, Alejandro Ramirez Bellido
  • Patent number: 12086714
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network used to select actions performed by a reinforcement learning agent interacting with an environment. In one aspect, a method includes maintaining a replay memory, where the replay memory stores pieces of experience data generated as a result of the reinforcement learning agent interacting with the environment. Each piece of experience data is associated with a respective expected learning progress measure that is a measure of an expected amount of progress made in the training of the neural network if the neural network is trained on the piece of experience data. The method further includes selecting a piece of experience data from the replay memory by prioritizing for selection pieces of experience data having relatively higher expected learning progress measures and training the neural network on the selected piece of experience data.
    Type: Grant
    Filed: January 30, 2023
    Date of Patent: September 10, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Tom Schaul, John Quan, David Silver
  • Patent number: 12073304
    Abstract: Methods, systems, and apparatus for classifying a new example using a comparison set of comparison examples. One method includes maintaining a comparison set, the comparison set including comparison examples and a respective label vector for each of the comparison examples, each label vector including a respective score for each label in a predetermined set of labels; receiving a new example; determining a respective attention weight for each comparison example by applying a neural network attention mechanism to the new example and to the comparison examples; and generating a respective label score for each label in the predetermined set of labels from, for each of the comparison examples, the respective attention weight for the comparison example and the respective label vector for the comparison example, in which the respective label score for each of the labels represents a likelihood that the label is a correct label for the new example.
    Type: Grant
    Filed: June 16, 2023
    Date of Patent: August 27, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Charles Blundell, Oriol Vinyals
  • Patent number: 12067732
    Abstract: A computer-implemented neural network system for decomposing input video data. A video data input receives a sequence of video image frames. The sequence is encoded, using a 3D spatio-temporal encoder neural network, into a set of latent variables representing a compressed version of the sequence. A 3D spatio-temporal decoder neural network processes the set of latent variables to generate two or more sets of decomposed video data; these may be stored, communicated, and/or made available to a user interface. Input video including undesired features such as reflections, shadows, and occlusions may thus be decomposed into two or more video sequences, one in which the undesired features are suppressed, and another containing the undesired features.
    Type: Grant
    Filed: November 20, 2019
    Date of Patent: August 20, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Joao Carreira, Jean-Baptiste Alayrac, Andrew Zisserman
  • Patent number: 12067491
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network having a plurality of policy parameters and used to select actions to be performed by an agent to control the agent to perform a particular task while interacting with one or more other agents in an environment. In one aspect, the method includes: maintaining data specifying a pool of candidate action selection policies; maintaining data specifying respective matchmaking policy; and training the policy neural network using a reinforcement learning technique to update the policy parameters. The policy parameters define policies to be used in controlling the agent to perform the particular task.
    Type: Grant
    Filed: April 6, 2023
    Date of Patent: August 20, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: David Silver, Oriol Vinyals, Maxwell Elliot Jaderberg
  • Patent number: 12061964
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent. One of the methods includes sampling a behavior modulation in accordance with a current probability distribution; for each of one or more time steps: processing an input comprising an observation characterizing a current state of the environment at the time step using an action selection neural network to generate a respective action score for each action in a set of possible actions that can be performed by the agent; modifying the action scores using the sampled behavior modulation; and selecting the action to be performed by the agent at the time step based on the modified action scores; determining a fitness measure corresponding to the sampled behavior modulation; and updating the current probability distribution over the set of possible behavior modulations using the fitness measure corresponding to the behavior modulation.
    Type: Grant
    Filed: September 25, 2020
    Date of Patent: August 13, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Tom Schaul, Diana Luiza Borsa, Fengning Ding, David Szepesvari, Georg Ostrovski, Simon Osindero, William Clinton Dabney
  • Patent number: 12056593
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting an action to be performed by a reinforcement learning agent interacting with an environment. A current observation characterizing a current state of the environment is received. For each action in a set of multiple actions that can be performed by the agent to interact with the environment, a probability distribution is determined over possible Q returns for the action-current observation pair. For each action, a measure of central tendency of the possible Q returns with respect to the probability distributions for the action-current observation pair is determined. An action to be performed by the agent in response to the current observation is selected using the measures of central tendency.
    Type: Grant
    Filed: November 16, 2020
    Date of Patent: August 6, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Marc Gendron-Bellemare, William Clinton Dabney
  • Patent number: 12046249
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for bandwidth extension. One of the methods includes obtaining a low-resolution version of an input, the low-resolution version of the input comprising a first number of samples at a first sample rate over a first time period; and generating, from the low-resolution version of the input, a high-resolution version of the input comprising a second, larger number of samples at a second, higher sample rate over the first time period. Generating the high-resolution version includes generating a representation of the low-resolution version of the input; processing the representation of the low-resolution version of the input through a conditioning neural network to generate a conditioning input; and processing the conditioning input using a generative neural network to generate the high/resolution version of the input.
    Type: Grant
    Filed: April 30, 2020
    Date of Patent: July 23, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Ioannis Alexandros Assael, Thomas Chadwick Walters, Archit Gupta, Brendan Shillingford
  • Patent number: 12032523
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for compressed sensing using neural networks. One of the methods includes receiving an input measurement of an input data item; for each of one or more optimization steps: processing a latent representation using a generator neural network to generate a candidate reconstructed data item, processing the candidate reconstructed data item using a measurement neural network to generate a measurement of the candidate reconstructed data item, and updating the latent representation to reduce an error between the measurement and the input measurement; and processing the latent representation after the one or more optimization steps using the generator neural network to generate a reconstruction of the input data item.
    Type: Grant
    Filed: March 13, 2020
    Date of Patent: July 9, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Yan Wu, Timothy Paul Lillicrap, Mihaela Rosca
  • Patent number: 12033055
    Abstract: A system including an attention neural network that is configured to receive an input sequence and to process the input sequence to generate an output is described. The attention neural network includes: an attention block configured to receive a query input, a key input, and a value input that are derived from an attention block input. The attention block includes an attention neural network layer configured to: receive an attention layer input derived from the query input, the key input, and the value input, and apply an attention mechanism to the query input, the key input, and the value input to generate an attention layer output for the attention neural network layer; and a gating neural network layer configured to apply a gating mechanism to the attention block input and the attention layer output of the attention neural network layer to generate a gated attention output.
    Type: Grant
    Filed: September 7, 2020
    Date of Patent: July 9, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Emilio Parisotto, Hasuk Song, Jack William Rae, Siddhant Madhu Jayakumar, Maxwell Elliot Jaderberg, Razvan Pascanu, Caglar Gulcehre
  • Patent number: 12032869
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating audio output samples predicted to be communicated by a user. One example system includes a first user device having a first user. The first user device initiates a communication session between the first user and a second user of a second user device. The first user device obtains a neural network model of the second user. The neural network model is trained to generate, conditioned on audio input samples received up to a current time step, an audio output sample predicted to be communicated by the second user at a next time step. The user device repeatedly provides received audio input samples as input to the neural network model and plays audio output samples generated by the neural network model in place of received audio input samples communicated by the second user.
    Type: Grant
    Filed: August 12, 2022
    Date of Patent: July 9, 2024
    Assignee: DeepMind Technologies Limited
    Inventors: Jakob Nicolaus Foerster, Ioannis Alexandros Assael