Patents by Inventor William Clinton Dabney
William Clinton Dabney has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240135182Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting an action to be performed by a reinforcement learning agent interacting with an environment. In one aspect, a method comprises: receiving a current observation; for each action of a plurality of actions: randomly sampling one or more probability values; for each probability value: processing the action, the current observation, and the probability value using a quantile function network to generate an estimated quantile value for the probability value with respect to a probability distribution over possible returns that would result from the agent performing the action in response to the current observation; determining a measure of central tendency of the one or more estimated quantile values; and selecting an action to be performed by the agent in response to the current observation using the measures of central tendency for the actions.Type: ApplicationFiled: December 15, 2023Publication date: April 25, 2024Inventors: Georg Ostrovski, William Clinton Dabney
-
Patent number: 11887000Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting an action to be performed by a reinforcement learning agent interacting with an environment. In one aspect, a method comprises: receiving a current observation; for each action of a plurality of actions: randomly sampling one or more probability values; for each probability value: processing the action, the current observation, and the probability value using a quantile function network to generate an estimated quantile value for the probability value with respect to a probability distribution over possible returns that would result from the agent performing the action in response to the current observation; determining a measure of central tendency of the one or more estimated quantile values; and selecting an action to be performed by the agent in response to the current observation using the measures of central tendency for the actions.Type: GrantFiled: February 15, 2023Date of Patent: January 30, 2024Assignee: DeepMind Technologies LimitedInventors: Georg Ostrovski, William Clinton Dabney
-
Patent number: 11755879Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing and storing inputs for use in a neural network. One of the methods includes receiving input data for storage in a memory system comprising a first set of memory blocks, the memory blocks having an associated order; passing the input data to a highest ordered memory block; for each memory block for which there is a lower ordered memory block: applying a filter function to data currently stored by the memory block to generate filtered data and passing the filtered data to a lower ordered memory block; and for each memory block: combining the data currently stored in the memory block with the data passed to the memory block to generate updated data, and storing the updated data in the memory block.Type: GrantFiled: February 11, 2019Date of Patent: September 12, 2023Assignee: DeepMind Technologies LimitedInventors: Razvan Pascanu, William Clinton Dabney, Thomas Stepleton
-
Publication number: 20230196108Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting an action to be performed by a reinforcement learning agent interacting with an environment. In one aspect, a method comprises: receiving a current observation; for each action of a plurality of actions: randomly sampling one or more probability values; for each probability value: processing the action, the current observation, and the probability value using a quantile function network to generate an estimated quantile value for the probability value with respect to a probability distribution over possible returns that would result from the agent performing the action in response to the current observation; determining a measure of central tendency of the one or more estimated quantile values; and selecting an action to be performed by the agent in response to the current observation using the measures of central tendency for the actions.Type: ApplicationFiled: February 15, 2023Publication date: June 22, 2023Inventors: Georg Ostrovski, William Clinton Dabney
-
Patent number: 11610118Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting an action to be performed by a reinforcement learning agent interacting with an environment. In one aspect, a method comprises: receiving a current observation; for each action of a plurality of actions: randomly sampling one or more probability values; for each probability value: processing the action, the current observation, and the probability value using a quantile function network to generate an estimated quantile value for the probability value with respect to a probability distribution over possible returns that would result from the agent performing the action in response to the current observation; determining a measure of central tendency of the one or more estimated quantile values; and selecting an action to be performed by the agent in response to the current observation using the measures of central tendency for the actions.Type: GrantFiled: February 11, 2019Date of Patent: March 21, 2023Assignee: DeepMind Technologies LimitedInventors: Georg Ostrovski, William Clinton Dabney
-
Publication number: 20210089908Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent. One of the methods includes sampling a behavior modulation in accordance with a current probability distribution; for each of one or more time steps: processing an input comprising an observation characterizing a current state of the environment at the time step using an action selection neural network to generate a respective action score for each action in a set of possible actions that can be performed by the agent; modifying the action scores using the sampled behavior modulation; and selecting the action to be performed by the agent at the time step based on the modified action scores; determining a fitness measure corresponding to the sampled behavior modulation; and updating the current probability distribution over the set of possible behavior modulations using the fitness measure corresponding to the behavior modulation.Type: ApplicationFiled: September 25, 2020Publication date: March 25, 2021Inventors: Tom Schaul, Diana Luiza Borsa, Fengning Ding, David Szepesvari, Georg Ostrovski, Simon Osindero, William Clinton Dabney
-
Publication number: 20210064970Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting an action to be performed by a reinforcement learning agent interacting with an environment. A current observation characterizing a current state of the environment is received. For each action in a set of multiple actions that can be performed by the agent to interact with the environment, a probability distribution is determined over possible Q returns for the action-current observation pair. For each action, a measure of central tendency of the possible Q returns with respect to the probability distributions for the action-current observation pair is determined. An action to be performed by the agent in response to the current observation is selected using the measures of central tendency.Type: ApplicationFiled: November 16, 2020Publication date: March 4, 2021Inventors: Marc Gendron-Bellemare, William Clinton Dabney
-
Patent number: 10860920Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting an action to be performed by a reinforcement learning agent interacting with an environment. A current observation characterizing a current state of the environment is received. For each action in a set of multiple actions that can be performed by the agent to interact with the environment, a probability distribution is determined over possible Q returns for the action-current observation pair. For each action, a measure of central tendency of the possible Q returns with respect to the probability distributions for the action-current observation pair is determined. An action to be performed by the agent in response to the current observation is selected using the measures of central tendency.Type: GrantFiled: July 10, 2019Date of Patent: December 8, 2020Assignee: DeepMind Technologies LimitedInventors: Marc Gendron-Bellemare, William Clinton Dabney
-
Publication number: 20200364557Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting an action to be performed by a reinforcement learning agent interacting with an environment. In one aspect, a method comprises: receiving a current observation; for each action of a plurality of actions: randomly sampling one or more probability values; for each probability value: processing the action, the current observation, and the probability value using a quantile function network to generate an estimated quantile value for the probability value with respect to a probability distribution over possible returns that would result from the agent performing the action in response to the current observation; determining a measure of central tendency of the one or more estimated quantile values; and selecting an action to be performed by the agent in response to the current observation using the measures of central tendency for the actions.Type: ApplicationFiled: February 11, 2019Publication date: November 19, 2020Inventors: Georg Ostrovski, William Clinton Dabney
-
Patent number: 10755177Abstract: A voice user interface (VUI) system use collaborative filtering to expand its own knowledge base. The system is designed to improve the accuracy and performance of the Natural Language Understanding (NLU) processing that underlies VUIs. The system leverages the knowledge of system users to crowdsource new information.Type: GrantFiled: December 31, 2015Date of Patent: August 25, 2020Assignee: AMAZON TECHNOLOGIES, INC.Inventors: William Clinton Dabney, Arpit Gupta, Faisal Ladhak, Markus Dreyer, Anjishnu Kumar
-
Publication number: 20190332923Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting an action to be performed by a reinforcement learning agent interacting with an environment. A current observation characterizing a current state of the environment is received. For each action in a set of multiple actions that can be performed by the agent to interact with the environment, a probability distribution is determined over possible Q returns for the action-current observation pair. For each action, a measure of central tendency of the possible Q returns with respect to the probability distributions for the action-current observation pair is determined. An action to be performed by the agent in response to the current observation is selected using the measures of central tendency.Type: ApplicationFiled: July 10, 2019Publication date: October 31, 2019Inventors: Marc Gendron-Bellemare, William Clinton Dabney
-
Publication number: 20190251419Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing and storing inputs for use in a neural network. One of the methods includes receiving input data for storage in a memory system comprising a first set of memory blocks, the memory blocks having an associated order; passing the input data to a highest ordered memory block; for each memory block for which there is a lower ordered memory block: applying a filter function to data currently stored by the memory block to generate filtered data and passing the filtered data to a lower ordered memory block; and for each memory block: combining the data currently stored in the memory block with the data passed to the memory block to generate updated data, and storing the updated data in the memory block.Type: ApplicationFiled: February 11, 2019Publication date: August 15, 2019Inventors: Razvan Pascanu, William Clinton Dabney, Thomas Stepleton
-
Patent number: 10070244Abstract: An audio system has multiple loudspeaker devices to produce sound corresponding to different channels of a multi-channel audio signal such as a surround sound audio signal. The loudspeaker devices may have speech recognition capabilities. In response to a command spoken by a user, the loudspeaker devices automatically determine their positions and configure themselves to receive appropriate channels based on the positions. In order to determine the positions, a first of the loudspeaker devices analyzes sound representing the user command to determine the position of the first loudspeaker device relative to the user. The first loudspeaker also produces responsive speech indicating to the user that the loudspeaker devices have been or are being configured. The other loudspeaker devices analyze the sound representing the responsive speech to determine their positions relative to the first loudspeaker device and report their positions to the first loudspeaker device.Type: GrantFiled: September 30, 2015Date of Patent: September 4, 2018Assignee: Amazon Technologies, Inc.Inventor: William Clinton Dabney