Patents by Inventor Scott Ellison Reed
Scott Ellison Reed has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11966839Abstract: A system comprising a causal convolutional neural network to autoregressively generate a succession of values of a data item conditioned upon previously generated values of the data item. The system includes support memory for a set of support data patches each of which comprises an encoding of an example data item. A soft attention mechanism attends to one or more patches when generating the current item value. The soft attention mechanism determines a set of scores for the support data patches, for example in the form of a soft attention query vector dependent upon the previously generated values of the data item. The soft attention query vector is used to query the memory. When generating the value of the data item at a current iteration layers of the causal convolutional neural network are conditioned upon the support data patches weighted by the scores.Type: GrantFiled: October 25, 2018Date of Patent: April 23, 2024Assignee: DeepMind Technologies LimitedInventors: Aaron Gerard Antonius van den Oord, Yutian Chen, Danilo Jimenez Rezende, Oriol Vinyals, Joao Ferdinando Gomes de Freitas, Scott Ellison Reed
-
Publication number: 20240042600Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-driven robotic control. One of the methods includes maintaining robot experience data; obtaining annotation data; training, on the annotation data, a reward model; generating task-specific training data for the particular task, comprising, for each experience in a second subset of the experiences in the robot experience data: processing the observation in the experience using the trained reward model to generate a reward prediction, and associating the reward prediction with the experience; and training a policy neural network on the task-specific training data for the particular task, wherein the policy neural network is configured to receive a network input comprising an observation and to generate a policy output that defines a control policy for a robot performing the particular task.Type: ApplicationFiled: June 8, 2023Publication date: February 8, 2024Inventors: Serkan Cabi, Ziyu Wang, Alexander Novikov, Ksenia Konyushkova, Sergio Gomez Colmenarejo, Scott Ellison Reed, Misha Man Ray Denil, Jonathan Karl Scholz, Oleg O. Sushkov, Rae Chan Jeong, David Barker, David Budden, Mel Vecerik, Yusuf Aytar, Joao Ferdinando Gomes de Freitas
-
Patent number: 11803746Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for neural programming. One of the methods includes processing a current neural network input using a core recurrent neural network to generate a neural network output; determining, from the neural network output, whether or not to end a currently invoked program and to return to a calling program from the set of programs; determining, from the neural network output, a next program to be called; determining, from the neural network output, contents of arguments to the next program to be called; receiving a representation of a current state of the environment; and generating a next neural network input from an embedding for the next program to be called and the representation of the current state of the environment.Type: GrantFiled: April 27, 2020Date of Patent: October 31, 2023Assignee: DeepMind Technologies LimitedInventors: Scott Ellison Reed, Joao Ferdinando Gomes de Freitas
-
Patent number: 11734797Abstract: A method of generating an output image having an output resolution of N pixels×N pixels, each pixel in the output image having a respective color value for each of a plurality of color channels, the method comprising: obtaining a low-resolution version of the output image; and upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations: obtaining a current version of the output image having a current K×K resolution; and processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having a 2K×2K resolution.Type: GrantFiled: May 23, 2022Date of Patent: August 22, 2023Assignee: DeepMind Technologies LimitedInventors: Nal Emmerich Kalchbrenner, Daniel Belov, Sergio Gomez Colmenarejo, Aaron Gerard Antonius van den Oord, Ziyu Wang, Joao Ferdinando Gomes de Freitas, Scott Ellison Reed
-
Publication number: 20230256593Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for off-line learning using a reward prediction model. One of the methods includes obtaining robot experience data; training, on a first subset of the robot experience data, a reward prediction model that receives a reward input comprising an input observation and generates as output a reward prediction that is a prediction Neural Network of a task-specific reward for the particular task that should be assigned to the input observation; processing experiences in the robot experience data using the trained reward prediction model to generate a respective reward prediction for each of the processed experiences; and training a policy neural network on (i) the processed experiences and (ii) the respective reward predictions for the processed experiences.Type: ApplicationFiled: July 27, 2021Publication date: August 17, 2023Inventors: Konrad Zolna, Scott Ellison Reed
-
Patent number: 11712799Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-driven robotic control. One of the methods includes maintaining robot experience data; obtaining annotation data; training, on the annotation data, a reward model; generating task-specific training data for the particular task, comprising, for each experience in a second subset of the experiences in the robot experience data: processing the observation in the experience using the trained reward model to generate a reward prediction, and associating the reward prediction with the experience; and training a policy neural network on the task-specific training data for the particular task, wherein the policy neural network is configured to receive a network input comprising an observation and to generate a policy output that defines a control policy for a robot performing the particular task.Type: GrantFiled: September 14, 2020Date of Patent: August 1, 2023Assignee: DeepMind Technologies LimitedInventors: Serkan Cabi, Ziyu Wang, Alexander Novikov, Ksenia Konyushkova, Sergio Gomez Colmenarejo, Scott Ellison Reed, Misha Man Ray Denil, Jonathan Karl Scholz, Oleg O. Sushkov, Rae Chan Jeong, David Barker, David Budden, Mel Vecerik, Yusuf Aytar, Joao Ferdinando Gomes de Freitas
-
Patent number: 11663441Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection policy neural network, wherein the action selection policy neural network is configured to process an observation characterizing a state of an environment to generate an action selection policy output, wherein the action selection policy output is used to select an action to be performed by an agent interacting with an environment. In one aspect, a method comprises: obtaining an observation characterizing a state of the environment subsequent to the agent performing a selected action; generating a latent representation of the observation; processing the latent representation of the observation using a discriminator neural network to generate an imitation score; determining a reward from the imitation score; and adjusting the current values of the action selection policy neural network parameters based on the reward using a reinforcement learning training technique.Type: GrantFiled: September 27, 2019Date of Patent: May 30, 2023Assignee: DeepMind Technologies LimitedInventors: Scott Ellison Reed, Yusuf Aytar, Ziyu Wang, Tom Paine, Sergio Gomez Colmenarejo, David Budden, Tobias Pfaff, Aaron Gerard Antonius van den Oord, Oriol Vinyals, Alexander Novikov
-
Publication number: 20230061411Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent to interact with an environment using an action selection neural network. In one aspect, a method comprises, at each time step in a sequence of time steps: generating a current representation of a state of a task being performed by the agent in the environment as of the current time step as a sequence of data elements; autoregressively generating a sequence of data elements representing a current action to be performed by the agent at the current time step; and after autoregressively generating the sequence of data elements representing the current action, causing the agent to perform the current action at the current time step.Type: ApplicationFiled: August 24, 2021Publication date: March 2, 2023Inventors: Tom Erez, Alexander Novikov, Emilio Parisotto, Jack William Rae, Konrad Zolna, Misha Man Ray Denil, Joao Ferdinando Gomes de Freitas, Oriol Vinyals, Scott Ellison Reed, Sergio Gomez, Ashley Deloris Edwards, Jacob Bruce, Gabriel Barth-Maron
-
Publication number: 20220284546Abstract: A method of generating an output image having an output resolution of N pixels×N pixels, each pixel in the output image having a respective color value for each of a plurality of color channels, the method comprising: obtaining a low-resolution version of the output image; and upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations: obtaining a current version of the output image having a current K×K resolution; and processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having a 2K×2K resolution.Type: ApplicationFiled: May 23, 2022Publication date: September 8, 2022Inventors: Nal Emmerich Kalchbrenner, Daniel Belov, Sergio Gomez Colmenarejo, Aaron Gerard Antonius van den Oord, Ziyu Wang, Joao Ferdinando Gomes de Freitas, Scott Ellison Reed
-
Publication number: 20220261639Abstract: A method is proposed of training a neural network to generate action data for controlling an agent to perform a task in an environment. The method includes obtaining, for each of a plurality of performances of the task, one or more first tuple datasets, each first tuple dataset comprising state data characterizing a state of the environment at a corresponding time during the performance of the task; and a concurrent process of training the neural network and a discriminator network. The training process comprises a plurality of neural network update steps and a plurality of discriminator network update steps.Type: ApplicationFiled: July 16, 2020Publication date: August 18, 2022Inventors: Konrad Zolna, Scott Ellison Reed, Ziyu Wang, Alexander Novikov, Sergio Gomez Colmenarejo, Joao Ferdinando Gomes de Freitas, David Budden, Serkan Cabi
-
Patent number: 11361403Abstract: A method of generating an output image having an output resolution of N pixels×N pixels, each pixel in the output image having a respective color value for each of a plurality of color channels, the method comprising: obtaining a low-resolution version of the output image; and upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations: obtaining a current version of the output image having a current K×K resolution; and processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having a 2K×2K resolution.Type: GrantFiled: February 26, 2018Date of Patent: June 14, 2022Assignee: DeepMind Technologies LimitedInventors: Nal Emmerich Kalchbrenner, Daniel Belov, Sergio Gomez Colmenarejo, Aaron Gerard Antonius van den Oord, Ziyu Wang, Joao Ferdinando Gomes de Freitas, Scott Ellison Reed
-
Patent number: 11355097Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an adaptive audio-generation model. One of the methods includes generating an adaptive audio-generation model including learning a plurality of embedding vectors and parameter values of a neural network using training data comprising first text and audio data representing a plurality of different individual speakers speaking portions of the first text, wherein the plurality of embedding vectors represent respective voice characteristics of the plurality of different individual speakers.Type: GrantFiled: October 1, 2020Date of Patent: June 7, 2022Assignee: DeepMind Technologies LimitedInventors: Yutian Chen, Scott Ellison Reed, Aaron Gerard Antonius van den Oord, Oriol Vinyals, Heiga Zen, Ioannis Alexandros Assael, Brendan Shillingford, Joao Ferdinando Gomes de Freitas
-
Publication number: 20210078169Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-driven robotic control. One of the methods includes maintaining robot experience data; obtaining annotation data; training, on the annotation data, a reward model; generating task-specific training data for the particular task, comprising, for each experience in a second subset of the experiences in the robot experience data: processing the observation in the experience using the trained reward model to generate a reward prediction, and associating the reward prediction with the experience; and training a policy neural network on the task-specific training data for the particular task, wherein the policy neural network is configured to receive a network input comprising an observation and to generate a policy output that defines a control policy for a robot performing the particular task.Type: ApplicationFiled: September 14, 2020Publication date: March 18, 2021Inventors: Serkan Cabi, Ziyu Wang, Alexander Novikov, Ksenia Konyushkova, Sergio Gomez Colmenarejo, Scott Ellison Reed, Misha Man Ray Denil, Jonathan Karl Scholz, Oleg O. Sushkov, Rae Chan Jeong, David Barker, David Budden, Mel Vecerik, Yusuf Aytar, Joao Ferdinando Gomes de Freitas
-
Publication number: 20210027425Abstract: A method of generating an output image having an output resolution of N pixels×N pixels, each pixel in the output image having a respective color value for each of a plurality of color channels, the method comprising: obtaining a low-resolution version of the output image; and upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations: obtaining a current version of the output image having a current K×K resolution; and processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having a 2K×2K resolution.Type: ApplicationFiled: February 26, 2018Publication date: January 28, 2021Inventors: Nal Emmerich Kalchbrenner, Daniel Belov, Sergio Gomez Colmenarejo, Aaron Gerard Antonius van den Oord, Ziyu Wang, Joao Gomes de Freitas, Scott Ellison Reed
-
Publication number: 20210020160Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an adaptive audio-generation model. One of the methods includes generating an adaptive audio-generation model including learning a plurality of embedding vectors and parameter values of a neural network using training data comprising first text and audio data representing a plurality of different individual speakers speaking portions of the first text, wherein the plurality of embedding vectors represent respective voice characteristics of the plurality of different individual speakers.Type: ApplicationFiled: October 1, 2020Publication date: January 21, 2021Inventors: Yutian Chen, Scott Ellison Reed, Aaron Gerard Antonius van den Oord, Oriol Vinyals, Heiga Zen, Ioannis Alexandros Assael, Brendan Shillingford, Joao Ferdinando Gomes de Freitas
-
Patent number: 10810993Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an adaptive audio-generation model. One of the methods includes generating an adaptive audio-generation model including learning a plurality of embedding vectors and parameter values of a neural network using training data comprising first text and audio data representing a plurality of different individual speakers speaking portions of the first text, wherein the plurality of embedding vectors represent respective voice characteristics of the plurality of different individual speakers.Type: GrantFiled: October 28, 2019Date of Patent: October 20, 2020Assignee: DeepMind Technologies LimitedInventors: Yutian Chen, Scott Ellison Reed, Aaron Gerard Antonius van den Oord, Oriol Vinyals, Heiga Zen, Ioannis Alexandros Assael, Brendan Shillingford, Joao Ferdinando Gomes de Freitas
-
Publication number: 20200327413Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for neural programming. One of the methods includes processing a current neural network input using a core recurrent neural network to generate a neural network output; determining, from the neural network output, whether or not to end a currently invoked program and to return to a calling program from the set of programs; determining, from the neural network output, a next program to be called; determining, from the neural network output, contents of arguments to the next program to be called; receiving a representation of a current state of the environment; and generating a next neural network input from an embedding for the next program to be called and the representation of the current state of the environment.Type: ApplicationFiled: April 27, 2020Publication date: October 15, 2020Inventors: Scott Ellison Reed, Joao Ferdinando Gomes de Freitas
-
Publication number: 20200250528Abstract: A system comprising a causal convolutional neural network to autoregressively generate a succession of values of a data item conditioned upon previously generated values of the data item. The system includes support memory for a set of support data patches each of which comprises an encoding of an example data item. A soft attention mechanism attends to one or more patches when generating the current item value. The soft attention mechanism determines a set of scores for the support data patches, for example in the form of a soft attention query vector dependent upon the previously generated values of the data item. The soft attention query vector is used to query the memory. When generating the value of the data item at a current iteration layers of the causal convolutional neural network are conditioned upon the support data patches weighted by the scores.Type: ApplicationFiled: October 25, 2018Publication date: August 6, 2020Inventors: Aaron Gerard Antonius van den Oord, Yutian Chen, Danilo Jimenez Rezende, Oriol Vinyals, Joao Ferdinando Gomes de Freitas, Scott Ellison Reed
-
Publication number: 20200135172Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an adaptive audio-generation model. One of the methods includes generating an adaptive audio-generation model including learning a plurality of embedding vectors and parameter values of a neural network using training data comprising first text and audio data representing a plurality of different individual speakers speaking portions of the first text, wherein the plurality of embedding vectors represent respective voice characteristics of the plurality of different individual speakers.Type: ApplicationFiled: October 28, 2019Publication date: April 30, 2020Inventors: Yutian Chen, Scott Ellison Reed, Aaron Gerard Antonius van den Oord, Oriol Vinyals, Heiga Zen, Ioannis Alexandros Assael, Brendan Shillingford, Joao Ferdinando Gomes de Freitas
-
Patent number: 10635974Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for neural programming. One of the methods includes processing a current neural network input using a core recurrent neural network to generate a neural network output; determining, from the neural network output, whether or not to end a currently invoked program and to return to a calling program from the set of programs; determining, from the neural network output, a next program to be called; determining, from the neural network output, contents of arguments to the next program to be called; receiving a representation of a current state of the environment; and generating a next neural network input from an embedding for the next program to be called and the representation of the current state of the environment.Type: GrantFiled: November 11, 2016Date of Patent: April 28, 2020Assignee: DeepMind Technologies LimitedInventors: Scott Ellison Reed, Joao Ferdinando Gomes de Freitas