Patents by Inventor Alexander Benjamin Graves
Alexander Benjamin Graves has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11783182Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for asynchronous deep reinforcement learning. One of the systems includes a plurality of workers, wherein each worker is configured to operate independently of each other worker, and wherein each worker is associated with a respective actor that interacts with a respective replica of the environment during the training of the deep neural network.Type: GrantFiled: February 8, 2021Date of Patent: October 10, 2023Assignee: DeepMind Technologies LimitedInventors: Volodymyr Mnih, Adrià Puigdomènech Badia, Alexander Benjamin Graves, Timothy James Alexander Harley, David Silver, Koray Kavukcuoglu
-
Patent number: 11715009Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network including a first subnetwork followed by a second subnetwork on training inputs by optimizing an objective function. In one aspect, a method includes processing a training input using the neural network to generate a training model output, including processing a subnetwork input for the training input using the first subnetwork to generate a subnetwork activation for the training input in accordance with current values of parameters of the first subnetwork, and providing the subnetwork activation as input to the second subnetwork; determining a synthetic gradient of the objective function for the first subnetwork by processing the subnetwork activation using a synthetic gradient model in accordance with current values of parameters of the synthetic gradient model; and updating the current values of the parameters of the first subnetwork using the synthetic gradient.Type: GrantFiled: May 19, 2017Date of Patent: August 1, 2023Assignee: DeepMind Technologies LimitedInventors: Oriol Vinyals, Alexander Benjamin Graves, Wojciech Czarnecki, Koray Kavukcuoglu, Simon Osindero, Maxwell Elliot Jaderberg
-
Publication number: 20220261647Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for asynchronous deep reinforcement learning. One of the systems includes a plurality of workers, wherein each worker is configured to operate independently of each other worker, and wherein each worker is associated with a respective actor that interacts with a respective replica of the environment during the training of the deep neural network.Type: ApplicationFiled: April 29, 2022Publication date: August 18, 2022Inventors: Volodymyr Mnih, Adrià Puigdomènech Badia, Alexander Benjamin Graves, Timothy James Alexander Harley, David Silver, Koray Kavukcuoglu
-
Patent number: 11334792Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for asynchronous deep reinforcement learning. One of the systems includes a plurality of workers, wherein each worker is configured to operate independently of each other worker, and wherein each worker is associated with a respective actor that interacts with a respective replica of the environment during the training of the deep neural network.Type: GrantFiled: May 3, 2019Date of Patent: May 17, 2022Assignee: DeepMind Technologies LimitedInventors: Volodymyr Mnih, Adria Puigdomenech Badia, Alexander Benjamin Graves, Timothy James Alexander Harley, David Silver, Koray Kavukcuoglu
-
Patent number: 11210579Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the methods includes providing an output derived from a first portion of a neural network output as a system output; determining one or more sets of writing weights for each of a plurality of locations in an external memory; writing data defined by a third portion of the neural network output to the external memory in accordance with the sets of writing weights; determining one or more sets of reading weights for each of the plurality of locations in the external memory from a fourth portion of the neural network output; reading data from the external memory in accordance with the sets of reading weights; and combining the data read from the external memory with a next system input to generate the next neural network input.Type: GrantFiled: March 26, 2020Date of Patent: December 28, 2021Assignee: DeepMind Technologies LimitedInventors: Alexander Benjamin Graves, Ivo Danihelka, Gregory Duncan Wayne
-
Patent number: 11151443Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the systems includes a sparse memory access subsystem that is configured to perform operations comprising generating a sparse set of reading weights that includes a respective reading weight for each of the plurality of locations in the external memory using the read key, reading data from the plurality of locations in the external memory in accordance with the sparse set of reading weights, generating a set of writing weights that includes a respective writing weight for each of the plurality of locations in the external memory, and writing the write vector to the plurality of locations in the external memory in accordance with the writing weights.Type: GrantFiled: February 3, 2017Date of Patent: October 19, 2021Assignee: DeepMind Technologies LimitedInventors: Ivo Danihelka, Gregory Duncan Wayne, Fu-min Wang, Edward Thomas Grefenstette, Jack William Rae, Alexander Benjamin Graves, Timothy Paul Lillicrap, Timothy James Alexander Harley, Jonathan James Hunt
-
Patent number: 11080594Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory using reinforcement learning. One of the methods includes providing an output derived from the system output portion of the neural network output as a system output in the sequence of system outputs; selecting a memory access process from a predetermined set of memory access processes for accessing the external memory from the reinforcement learning portion of the neural network output; writing and reading data from locations in the external memory in accordance with the selected memory access process using the differentiable portion of the neural network output; and combining the data read from the external memory with a next system input in the sequence of system inputs to generate a next neural network input in the sequence of neural network inputs.Type: GrantFiled: December 30, 2016Date of Patent: August 3, 2021Assignee: DeepMind Technologies LimitedInventors: Ilya Sutskever, Ivo Danihelka, Alexander Benjamin Graves, Gregory Duncan Wayne, Wojciech Zaremba
-
Publication number: 20210166127Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for asynchronous deep reinforcement learning. One of the systems includes a plurality of workers, wherein each worker is configured to operate independently of each other worker, and wherein each worker is associated with a respective actor that interacts with a respective replica of the environment during the training of the deep neural network.Type: ApplicationFiled: February 8, 2021Publication date: June 3, 2021Inventors: Volodymyr Mnih, Adrià Puigdomènech Badia, Alexander Benjamin Graves, Timothy James Alexander Harley, David Silver, Koray Kavukcuoglu
-
Publication number: 20210150355Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a machine learning model. In one aspect, a method includes receiving training data for training the machine learning model on a plurality of tasks, where each task includes multiple batches of training data. A task is selected in accordance with a current task selection policy. A batch of training data is selected from the selected task. The machine learning model is trained on the selected batch of training data to determine updated values of the model parameters. A learning progress measure that represents a progress of the training of the machine learning model as a result of training the machine learning model on the selected batch of training data is determined. The current task selection policy is updated using the learning progress measure.Type: ApplicationFiled: January 27, 2021Publication date: May 20, 2021Inventors: Marc Gendron-Bellemare, Jacob Lee Menick, Alexander Benjamin Graves, Koray Kavukcuoglu, Remi Munos
-
Patent number: 11010663Abstract: Systems, methods, and apparatus, including computer programs encoded on a computer storage medium, related to associative long short-term memory (LSTM) neural network layers configured to maintain N copies of an internal state for the associative LSTM layer, N being an integer greater than one. In one aspect, a system includes a recurrent neural network including an associative LSTM layer, wherein the associative LSTM layer is configured to, for each time step, receive a layer input, update each of the N copies of the internal state using the layer input for the time step and a layer output generated by the associative LSTM layer for a preceding time step, and generate a layer output for the time step using the N updated copies of the internal state.Type: GrantFiled: December 30, 2016Date of Patent: May 18, 2021Assignee: DeepMind Technologies LimitedInventors: Ivo Danihelka, Nal Emmerich Kalchbrenner, Gregory Duncan Wayne, Benigno Uría-Martínez, Alexander Benjamin Graves
-
Publication number: 20210117801Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the systems includes a memory interface subsystem that is configured to perform operations comprising determining a respective content-based weight for each of a plurality of locations in an external memory; determining a respective allocation weight for each of the plurality of locations in the external memory; determining a respective final writing weight for each of the plurality of locations in the external memory from the respective content-based weight for the location and the respective allocation weight for the location; and writing data defined by the write vector to the external memory in accordance with the final writing weights.Type: ApplicationFiled: November 9, 2020Publication date: April 22, 2021Inventors: Alexander Benjamin Graves, Ivo Danihelka, Timothy James Alexander Harley, Malcolm Kevin Campbell Reynolds, Gregory Duncan Wayne
-
Patent number: 10936946Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for asynchronous deep reinforcement learning. One of the systems includes a plurality of workers, wherein each worker is configured to operate independently of each other worker, and wherein each worker is associated with a respective actor that interacts with a respective replica of the environment during the training of the deep neural network.Type: GrantFiled: November 11, 2016Date of Patent: March 2, 2021Assignee: DeepMind Technologies LimitedInventors: Volodymyr Mnih, Adrià Puigdomènech Badia, Alexander Benjamin Graves, Timothy James Alexander Harley, David Silver, Koray Kavukcuoglu
-
Patent number: 10936949Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a machine learning model. In one aspect, a method includes receiving training data for training the machine learning model on a plurality of tasks, where each task includes multiple batches of training data. A task is selected in accordance with a current task selection policy. A batch of training data is selected from the selected task. The machine learning model is trained on the selected batch of training data to determine updated values of the model parameters. A learning progress measure that represents a progress of the training of the machine learning model as a result of training the machine learning model on the selected batch of training data is determined. The current task selection policy is updated using the learning progress measure.Type: GrantFiled: July 10, 2019Date of Patent: March 2, 2021Assignee: DeepMind Technologies LimitedInventors: Marc Gendron-Bellemare, Jacob Lee Menick, Alexander Benjamin Graves, Koray Kavukcuoglu, Remi Munos
-
Publication number: 20210004677Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an encoder neural network, a decoder neural network, and a prior neural network, and using the trained networks for generative modeling, data compression, and data decompression. In one aspect, a method comprises: providing a given observation as input to the encoder neural network to generate parameters of an encoding probability distribution; determining an updated code for the given observation; selecting a code that is assigned to an additional observation; providing the code assigned to the additional observation as input to the prior neural network to generate parameters of a prior probability distribution; sampling latent variables from the encoding probability distribution; providing the latent variables as input to the decoder neural network to generate parameters of an observation probability distribution; and determining gradients of a loss function.Type: ApplicationFiled: February 11, 2019Publication date: January 7, 2021Inventors: Jacob Lee Menick, Alexander Benjamin Graves
-
Patent number: 10832134Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the systems includes a memory interface subsystem that is configured to perform operations comprising determining a respective content-based weight for each of a plurality of locations in an external memory; determining a respective allocation weight for each of the plurality of locations in the external memory; determining a respective final writing weight for each of the plurality of locations in the external memory from the respective content-based weight for the location and the respective allocation weight for the location; and writing data defined by the write vector to the external memory in accordance with the final writing weights.Type: GrantFiled: December 9, 2016Date of Patent: November 10, 2020Assignee: DeepMind Technologies LimitedInventors: Alexander Benjamin Graves, Ivo Danihelka, Timothy James Alexander Harley, Malcolm Kevin Campbell Reynolds, Gregory Duncan Wayne
-
Publication number: 20200320396Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network including a first subnetwork followed by a second subnetwork on training inputs by optimizing an objective function. In one aspect, a method includes processing a training input using the neural network to generate a training model output, including processing a subnetwork input for the training input using the first subnetwork to generate a subnetwork activation for the training input in accordance with current values of parameters of the first subnetwork, and providing the subnetwork activation as input to the second subnetwork; determining a synthetic gradient of the objective function for the first subnetwork by processing the subnetwork activation using a synthetic gradient model in accordance with current values of parameters of the synthetic gradient model; and updating the current values of the parameters of the first subnetwork using the synthetic gradient.Type: ApplicationFiled: May 19, 2017Publication date: October 8, 2020Applicant: Deepmind Technologies LimitedInventors: Oriol VINYALS, Alexander Benjamin GRAVES, Wojciech CZARNECKI, Koray KAVUKCUOGLU, Simon OSINDERO, Maxwell Elliot JADERBERG
-
Publication number: 20200226446Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the methods includes providing an output derived from a first portion of a neural network output as a system output; determining one or more sets of writing weights for each of a plurality of locations in an external memory; writing data defined by a third portion of the neural network output to the external memory in accordance with the sets of writing weights; determining one or more sets of reading weights for each of the plurality of locations in the external memory from a fourth portion of the neural network output; reading data from the external memory in accordance with the sets of reading weights; and combining the data read from the external memory with a next system input to generate the next neural network input.Type: ApplicationFiled: March 26, 2020Publication date: July 16, 2020Applicant: DeepMind Technologies LimitedInventors: Alexander Benjamin Graves, Ivo Danihelka, Gregory Duncan Wayne
-
Patent number: 10691997Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks to generate additional outputs. One of the systems includes a neural network and a sequence processing subsystem, wherein the sequence processing subsystem is configured to perform operations comprising, for each of the system inputs in a sequence of system inputs: receiving the system input; generating an initial neural network input from the system input; causing the neural network to process the initial neural network input to generate an initial neural network output for the system input; and determining, from a first portion of the initial neural network output for the system input, whether or not to cause the neural network to generate one or more additional neural network outputs for the system input.Type: GrantFiled: December 21, 2015Date of Patent: June 23, 2020Assignee: DeepMind Technologies LimitedInventors: Alexander Benjamin Graves, Ivo Danihelka, Gregory Duncan Wayne
-
Patent number: 10650302Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the methods includes providing an output derived from a first portion of a neural network output as a system output; determining one or more sets of writing weights for each of a plurality of locations in an external memory; writing data defined by a third portion of the neural network output to the external memory in accordance with the sets of writing weights; determining one or more sets of reading weights for each of the plurality of locations in the external memory from a fourth portion of the neural network output; reading data from the external memory in accordance with the sets of reading weights; and combining the data read from the external memory with a next system input to generate the next neural network input.Type: GrantFiled: October 16, 2015Date of Patent: May 12, 2020Assignee: DeepMind Technologies LimitedInventors: Alexander Benjamin Graves, Ivo Danihelka, Gregory Duncan Wayne
-
Patent number: 10482373Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing grid Long Short-Term Memory (LSTM) neural networks that includes a plurality of N-LSTM blocks arranged in an N-dimensional grid. Each N-LSTM block is configured to: receive N input hidden vectors, the N input hidden vectors each corresponding to a respective one of the N dimensions; receive N input memory vectors, the N input memory vectors each corresponding to a respective one of the N dimensions; and, for each of the dimensions, apply a respective transform for the dimension to the memory hidden vector corresponding to the dimension and the input hidden vector corresponding to the dimension to generate a new hidden vector corresponding to the dimension and a new memory vector corresponding to the dimension.Type: GrantFiled: June 6, 2016Date of Patent: November 19, 2019Assignee: DeepMind Technologies LimitedInventors: Nal Emmerich Kalchbrenner, Ivo Danihelka, Alexander Benjamin Graves