Patents by Inventor Aleksandra Faust
Aleksandra Faust has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11972339Abstract: Implementations relate to using deep reinforcement learning to train a model that can be utilized, at each of a plurality of time steps, to determine a corresponding robotic action for completing a robotic task. Implementations additionally or alternatively relate to utilization of such a model in controlling a robot. The robotic action determined at a given time step utilizing such a model can be based on: current sensor data associated with the robot for the given time step, and free-form natural language input provided by a user. The free-form natural language input can direct the robot to accomplish a particular task, optionally with reference to one or more intermediary steps for accomplishing the particular task. For example, the free-form natural language input can direct the robot to navigate to a particular landmark, with reference to one or more intermediary landmarks to be encountered in navigating to the particular landmark.Type: GrantFiled: March 22, 2019Date of Patent: April 30, 2024Assignee: GOOGLE LLCInventors: Pararth Shah, Dilek Hakkani-Tur, Juliana Kew, Marek Fiser, Aleksandra Faust
-
Patent number: 11941504Abstract: Implementations relate to using deep reinforcement learning to train a model that can be utilized, at each of a plurality of time steps, to determine a corresponding robotic action for completing a robotic task. Implementations additionally or alternatively relate to utilization of such a model in controlling a robot. The robotic action determined at a given time step utilizing such a model can be based on: current sensor data associated with the robot for the given time step, and free-form natural language input provided by a user. The free-form natural language input can direct the robot to accomplish a particular task, optionally with reference to one or more intermediary steps for accomplishing the particular task. For example, the free-form natural language input can direct the robot to navigate to a particular landmark, with reference to one or more intermediary landmarks to be encountered in navigating to the particular landmark.Type: GrantFiled: March 22, 2019Date of Patent: March 26, 2024Assignee: GOOGLE LLCInventors: Pararth Shah, Dilek Hakkani-Tur, Juliana Kew, Marek Fiser, Aleksandra Faust
-
Publication number: 20230394102Abstract: The present disclosure is generally directed to methods, apparatus, and computer-readable media (transitory and non-transitory) for learning to automatically navigate interactive web documents and/or websites. More particularly, various approaches are presented for training various deep Q network (DQN) agents to perform various tasks associated with reinforcement learning, including hierarchical reinforcement learning, in challenging web navigation environments with sparse rewards and large state and action spaces. These agents include a web navigation agent that can use learned value function(s) to automatically navigate through interactive web documents, as well as a training agent, referred to herein as a “meta-trainer,” that can be trained to generate synthetic training examples. Some approaches described herein may be implemented when expert demonstrations are available. Other approaches described herein may be implemented when expert demonstrations are not available.Type: ApplicationFiled: August 16, 2023Publication date: December 7, 2023Inventors: Aleksandra Faust, Dilek Hakkani-Tur, Izzeddin Gur, Ulrich Rueckert
-
Patent number: 11734375Abstract: The present disclosure is generally directed to methods, apparatus, and computer-readable media (transitory and non-transitory) for learning to automatically navigate interactive web documents and/or websites. More particularly, various approaches are presented for training various deep Q network (DQN) agents to perform various tasks associated with reinforcement learning, including hierarchical reinforcement learning, in challenging web navigation environments with sparse rewards and large state and action spaces. These agents include a web navigation agent that can use learned value function(s) to automatically navigate through interactive web documents, as well as a training agent, referred to herein as a “meta-trainer,” that can be trained to generate synthetic training examples. Some approaches described herein may be implemented when expert demonstrations are available. Other approaches described herein may be implemented when expert demonstrations are not available.Type: GrantFiled: September 27, 2019Date of Patent: August 22, 2023Assignee: GOOGLE LLCInventors: Aleksandra Faust, Dilek Hakkani-Tur, Izzeddin Gur, Ulrich Rueckert
-
Publication number: 20220391687Abstract: Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for generating and searching reinforcement learning algorithms. In some implementations, a computer-implemented system generates a sequence of candidate reinforcement learning algorithms. Each candidate reinforcement learning algorithm in the sequence is configured to receive an input environment state characterizing a state of an environment and to generate an output that specifies an action to be performed by an agent interacting with the environment. For each candidate reinforcement learning algorithm in the sequence, the system performs a performance evaluation for a set of a plurality of training environments. For each training environment, the system adjusts a set of environment-specific parameters of the candidate reinforcement learning algorithm by performing training of the candidate reinforcement learning algorithm to control a corresponding agent in the training environment.Type: ApplicationFiled: June 3, 2021Publication date: December 8, 2022Inventors: John Dalton Co-Reyes, Yingjie Miao, Daiyi Peng, Sergey Vladimir Levine, Quoc V. Le, Honglak Lee, Aleksandra Faust
-
Patent number: 11436441Abstract: A computer-implemented method is disclosed for training one or more machine-learned models. The method can include inputting a first image frame and a second image frame into a feature disentanglement model and receiving, as an output of the machine-learned feature disentanglement model, a state feature and a perspective feature. The method can include inputting the state feature and the perspective feature into a machine-learned decoder model and receiving, as an output of the machine-learned decoder model, the reconstructed image frame. The method can include comparing the reconstructed image frame with a third image frame corresponding with the location and the perspective orientation. The method can include adjusting one or more parameters of the machine-learned feature disentanglement model based on the comparison of the reconstructed image frame and the third image frame.Type: GrantFiled: December 17, 2019Date of Patent: September 6, 2022Assignee: GOOGLE LLCInventors: Jie Tan, Sehoon Ha, Tingnan Zhang, Xinlei Pan, Brian Andrew Ichter, Aleksandra Faust
-
Publication number: 20210334320Abstract: The present disclosure is generally directed to methods, apparatus, and computer-readable media (transitory and non-transitory) for learning to automatically navigate interactive web documents and/or websites. More particularly, various approaches are presented for training various deep Q network (DQN) agents to perform various tasks associated with reinforcement learning, including hierarchical reinforcement learning, in challenging web navigation environments with sparse rewards and large state and action spaces. These agents include a web navigation agent that can use learned value function(s) to automatically navigate through interactive web documents, as well as a training agent, referred to herein as a “meta-trainer,” that can be trained to generate synthetic training examples. Some approaches described herein may be implemented when expert demonstrations are available. Other approaches described herein may be implemented when expert demonstrations are not available.Type: ApplicationFiled: September 27, 2019Publication date: October 28, 2021Inventors: Aleksandra Faust, Dilek Hakkani-Tur, Izzeddin Gur, Ulrich Rueckert
-
Publication number: 20210325894Abstract: Using reinforcement learning to train a policy network that can be utilized, for example, by a robot in performing robot navigation and/or other robotic tasks. Various implementations relate to techniques for automatically learning a reward function for training of a policy network through reinforcement learning, and automatically learning a neural network architecture for the policy network.Type: ApplicationFiled: September 13, 2019Publication date: October 21, 2021Inventors: Aleksandra Faust, Hao-tien Chiang, Anthony Francis, Marek Fiser
-
Patent number: 11067988Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing an interactive autonomous vehicle agent. One of the methods includes receiving a request to generate an experience tuple for a vehicle in a particular driving context. A predicted environment observation representing a predicted environment of the autonomous vehicle after the candidate action is taken by the autonomous vehicle in an initial environment is generated, including providing an initial environment observation and the candidate action as input to a vehicle behavior model neural network trained to generate predicted environment observations. An immediate quality value is generated from a context-specific quality model that generates immediate quality values that are specific to the particular driving context.Type: GrantFiled: March 13, 2019Date of Patent: July 20, 2021Assignee: Waymo LLCInventors: Aleksandra Faust, Matthieu Devin, Yu-hsin Joyce Chen, Franklin Morley, Vadim Furman, Carlos Alberto Fuertes Pascual
-
Publication number: 20210182620Abstract: A computer-implemented method is disclosed for training one or more machine-learned models. The method can include inputting a first image frame and a second image frame into a feature disentanglement model and receiving, as an output of the machine-learned feature disentanglement model, a state feature and a perspective feature. The method can include inputting the state feature and the perspective feature into a machine-learned decoder model and receiving, as an output of the machine-learned decoder model, the reconstructed image frame. The method can include comparing the reconstructed image frame with a third image frame corresponding with the location and the perspective orientation. The method can include adjusting one or more parameters of the machine-learned feature disentanglement model based on the comparison of the reconstructed image frame and the third image frame.Type: ApplicationFiled: December 17, 2019Publication date: June 17, 2021Inventors: Jie Tan, Sehoon Ha, Tingnan Zhang, Xinlei Pan, Brian Andrew Ichter, Aleksandra Faust
-
Publication number: 20210086353Abstract: Implementations relate to using deep reinforcement learning to train a model that can be utilized, at each of a plurality of time steps, to determine a corresponding robotic action for completing a robotic task. Implementations additionally or alternatively relate to utilization of such a model in controlling a robot. The robotic action determined at a given time step utilizing such a model can be based on: current sensor data associated with the robot for the given time step, and free-form natural language input provided by a user. The free-form natural language input can direct the robot to accomplish a particular task, optionally with reference to one or more intermediary steps for accomplishing the particular task. For example, the free-form natural language input can direct the robot to navigate to a particular landmark, with reference to one or more intermediary landmarks to be encountered in navigating to the particular landmark.Type: ApplicationFiled: March 22, 2019Publication date: March 25, 2021Inventors: Pararth Shah, Dilek Hakkani-Tur, Juliana Kew, Marek Fiser, Aleksandra Faust
-
Patent number: 10837811Abstract: Systems and methods are disclosed that include tools that utilize Dynamic Detector Tuning (DDT) software that identifies near-optimal parameter settings for each sensor using a neuro-dynamic programming (reinforcement learning) paradigm. DDT adapts parameter values to the current state of the environment by leveraging cooperation within a neighborhood of sensors. The key metric that guides the dynamic tuning is consistency of each sensor with its nearest neighbors: parameters are automatically adjusted on a per station basis to be more or less sensitive to produce consistent agreement of detections in its neighborhood. The DDT algorithm adapts in near real-time to changing conditions in an attempt to automatically self-tune a signal detector to identify (detect) only signals from events of interest. The disclosed systems and methods reduce the number of missed legitimate detections and the number of false detections, resulting in improved event detection.Type: GrantFiled: November 30, 2017Date of Patent: November 17, 2020Assignee: National Technology & Engineering Solutions of Sandia, LLCInventors: Timothy J. Draelos, Aleksandra Faust, Hunter A. Knox, Matthew G. Peterson
-
Patent number: 10311658Abstract: Aspects of the disclosure relate to detecting vehicle collisions. In one example, one or more computing devices may receive acceleration data of a vehicle and the expected acceleration data of the vehicle over a period of time. The one or more computing devices may determine a change in the vehicle's acceleration over the period of time, where the change in the vehicle's acceleration over the period of time is the difference between the expected acceleration data and the acceleration data. The one or more computing devices may detect an occurrence when the change in the vehicle's acceleration is greater than a threshold value and assign the occurrence into a collision category. Based on the assigned collision category, the one or more computing devices may perform a responsive action.Type: GrantFiled: October 7, 2016Date of Patent: June 4, 2019Assignee: Waymo LLCInventors: Aleksandra Faust, Nathaniel Fairfield
-
Patent number: 10254759Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing an interactive autonomous vehicle agent. One of the methods includes receiving a request to generate an experience tuple for a vehicle in a particular driving context. A predicted environment observation representing a predicted environment of the autonomous vehicle after the candidate action is taken by the autonomous vehicle in an initial environment is generated, including providing an initial environment observation and the candidate action as input to a vehicle behavior model neural network trained to generate predicted environment observations. An immediate quality value is generated from a context-specific quality model that generates immediate quality values that are specific to the particular driving context.Type: GrantFiled: September 14, 2017Date of Patent: April 9, 2019Assignee: Waymo LLCInventors: Aleksandra Faust, Matthieu Devin, Yu-hsin Joyce Chen, Franklin Morley, Vadim Furman, Carlos Alberto Fuertes Pascual
-
Publication number: 20180102001Abstract: Aspects of the disclosure relate to detecting vehicle collisions. In one example, one or more computing devices may receive acceleration data of a vehicle and the expected acceleration data of the vehicle over a period of time. The one or more computing devices may determine a change in the vehicle's acceleration over the period of time, where the change in the vehicle's acceleration over the period of time is the difference between the expected acceleration data and the acceleration data. The one or more computing devices may detect an occurrence when the change in the vehicle's acceleration is greater than a threshold value and assign the occurrence into a collision category. Based on the assigned collision category, the one or more computing devices may perform a responsive action.Type: ApplicationFiled: October 7, 2016Publication date: April 12, 2018Inventors: Aleksandra Faust, Nathaniel Fairfield
-
Patent number: 9828107Abstract: The present invention provides a vehicle with redundant systems to increase the overall safety of the vehicle. In other aspects, the present invention provides a method for learning control of non-linear motion systems through combined learning of state value and action-value functions.Type: GrantFiled: August 25, 2015Date of Patent: November 28, 2017Assignee: STC.UNMInventors: Arnold Peter Ruymgaart, Lydia Tapia, Aleksandra Faust, Rafael Fierro