Patents by Inventor Bala Venkata Sai Ravi Krishna Kolluri
Bala Venkata Sai Ravi Krishna Kolluri has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11820014Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using simulated local demonstration data for robotic demonstration learning. One of the methods includes receiving perceptual data of a workcell of a robot to be configured to execute a task according to a skill template, wherein the skill template specifies one or more subtasks required to perform the skill, wherein at least one of the subtasks is a demonstration subtask that relies on learning visual characteristics of the workcell. A virtual model is generated of a portion of the workcell. A training system generates simulated local demonstration data from the virtual model of the portion of the workcell and tunes a base control policy for the demonstration subtask using the simulated local demonstration data generated from the virtual model of the portion of the workcell.Type: GrantFiled: May 21, 2020Date of Patent: November 21, 2023Assignee: Intrinsic Innovation LLCInventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Ralf Oliver Michael Schönherr, Benjamin M. Davis, Ning Ye
-
Publication number: 20230356393Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributing skill templates for robotic demonstration learning. One of the methods includes receiving from the user device by a skill template distribution system, a selection of an available skill template. The skill template distribution system provides a skill template, wherein the skill template comprises information representing a state machine of one or more tasks, and wherein the skill template specifies which of the one or more tasks are demonstration subtasks requiring local demonstration data. The skill template distribution system trains a machine learning model for the demonstration subtask using a local demonstration data to generate learned parameter values.Type: ApplicationFiled: June 26, 2023Publication date: November 9, 2023Inventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Benjamin M. Davis, Ralf Oliver Michael Schönherr, Ning Ye
-
Patent number: 11780086Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a demonstration device for robotic demonstration learning. One of the methods includes generating, by a demonstration device for a robot, a representation of a sequence of states input by a user of the demonstration device. The representation is provided by the demonstration device to a robot execution system. The representation of the sequence of actions is translated into a plurality of robot commands corresponding to the representation of the sequence of states input by the user on the demonstration device. The plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device are executed. Demonstration data is generated from one or more sensor streams of the robot while executing the plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device.Type: GrantFiled: October 17, 2022Date of Patent: October 10, 2023Assignee: Intrinsic Innovation LLCInventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Ralf Oliver Michael Schönherr, Ning Ye
-
Publication number: 20230286148Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computing interpolated robot control parameters. One of the methods includes receiving, by a real-time bridge from a control agent for a robot, a non-real-time command for the robot, wherein the non-real-time command specifies a trajectory to be attained by a component of the robot and a target value for a control parameter, wherein the control parameter controls how a real-time controller will cause the robot to react to one or more external stimuli encountered during a control cycle of the real-time controller. The real-time bridge provides the one or more real-time commands translated from the non-real-time command and interpolated control parameter information to the real-time controller, thereby causing the robot to effectuate the trajectory of the non-real-time command according to the interpolated control parameter information.Type: ApplicationFiled: May 17, 2023Publication date: September 14, 2023Inventors: Michael Beardsworth, Klas Jonas Alfred Kronander, Sean Alexander Cassero, Bala Venkata Sai Ravi Krishna Kolluri
-
Patent number: 11691283Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computing interpolated robot control parameters. One of the methods includes receiving, by a real-time bridge from a control agent for a robot, a non-real-time command for the robot, wherein the non-real-time command specifies a trajectory to be attained by a component of the robot and a target value for a control parameter, wherein the control parameter controls how a real-time controller will cause the robot to react to one or more external stimuli encountered during a control cycle of the real-time controller. The real-time bridge provides the one or more real-time commands translated from the non-real-time command and interpolated control parameter information to the real-time controller, thereby causing the robot to effectuate the trajectory of the non-real-time command according to the interpolated control parameter information.Type: GrantFiled: May 27, 2020Date of Patent: July 4, 2023Assignee: Intrinsic Innovation LLCInventors: Michael Beardsworth, Klas Jonas Alfred Kronander, Sean Alexander Cassero, Bala Venkata Sai Ravi Krishna Kolluri
-
Patent number: 11685047Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributing skill templates for robotic demonstration learning. One of the methods includes receiving, from the user device by a skill template distribution system, a selection of an available skill template. The skill template distribution system provides a skill template, wherein the skill template comprises information representing a state machine of one or more tasks, and wherein the skill template specifies which of the one or more tasks are demonstration subtasks requiring local demonstration data. The skill template distribution system trains a machine learning model for the demonstration subtask using a local demonstration data to generate learned parameter values.Type: GrantFiled: May 21, 2020Date of Patent: June 27, 2023Assignee: Intrinsic Innovation LLCInventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Benjamin M. Davis, Ralf Oliver Michael Schönherr, Ning Ye
-
Patent number: 11679497Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributed robotic demonstration learning. One of the methods includes receiving a skill template to be trained to cause a robot to perform a particular skill having a plurality of subtasks. One or more demonstration subtasks defined by the skill template are identified, wherein each demonstration subtask is an action to be refined using local demonstration data. On online execution system uploads sets of local demonstration data to a cloud-based training system. The cloud-based training system generates respective trained model parameters for each set of local demonstration data. The skill template is executed on the robot using the trained model parameters generated by the cloud-based training system.Type: GrantFiled: May 21, 2020Date of Patent: June 20, 2023Assignee: Intrinsic Innovation LLCInventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Benjamin M. Davis, Ralf Oliver Michael Schönherr, Ning Ye
-
Publication number: 20230114561Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a demonstration device for robotic demonstration learning. One of the methods includes generating, by a demonstration device for a robot, a representation of a sequence of states input by a user of the demonstration device. The representation is provided by the demonstration device to a robot execution system. The representation of the sequence of actions is translated into a plurality of robot commands corresponding to the representation of the sequence of states input by the user on the demonstration device. The plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device are executed. Demonstration data is generated from one or more sensor streams of the robot while executing the plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device.Type: ApplicationFiled: October 17, 2022Publication date: April 13, 2023Inventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Ralf Oliver Michael Schönherr, Ning Ye
-
Publication number: 20230046520Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using learnable robotic control plans. One of the methods comprises obtaining a learnable robotic control plan comprising data defining a state machine that includes a plurality of states and a plurality of transitions between states, wherein: one or more states are learnable states, and each learnable state comprises data defining (i) one or more learnable parameters of the learnable state and (ii) a machine learning procedure for automatically learning a respective value for each learnable parameter of the learnable state; and processing the learnable robotic control plan to generate a specific robotic control plan, comprising: obtaining data characterizing a robotic execution environment; and for each learnable state, executing, using the obtained data, the respective machine learning procedures defined by the learnable state to generate a respective value for each learnable parameter of the learnable state.Type: ApplicationFiled: August 10, 2021Publication date: February 16, 2023Inventors: Ning Ye, Maryam Bandari, Klas Jonas Alfred Kronander, Bala Venkata Sai Ravi Krishna Kolluri, Jianlan Luo, Wenzhao Lian, Chang Su
-
Patent number: 11534913Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for integrating sensor streams for robotic demonstration learning. One of the methods includes selecting, by a learning system for a robot, a base update rate for combining multiple sensor streams into a task state representation. The learning system repeatedly generates the task state representation at the base update rate, including combining, during each time period defined by the update rate, the task state representation from most recently updated sensor data processed by the plurality of neural networks. The learning system repeatedly uses the task state representations to generate commands for the robot at the base update rate.Type: GrantFiled: May 21, 2020Date of Patent: December 27, 2022Assignee: Intrinsic Innovation LLCInventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Ralf Oliver Michael Schönherr, Benjamin M. Davis, Ning Ye
-
Patent number: 11524402Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for providing user feedback for robotic demonstration learning. One of the methods includes initiating a local demonstration learning process to collect respective local demonstration data for each of one or more demonstration subtasks defined by a skill template to be executed by a robot. Local demonstration data is repeatedly collected for each of the one or more demonstration subtasks of the skill template while a user manipulates a robot to perform each of the one or more demonstration subtasks defined by the skill template. A respective progress value for each of the one or more demonstration subtasks defined by the skill template is maintained. A user interface presentation is generated that presents a suggested demonstration to be performed by the user based on a respective progress value for each demonstration subtask.Type: GrantFiled: May 21, 2020Date of Patent: December 13, 2022Assignee: Intrinsic Innovation LLCInventor: Bala Venkata Sai Ravi Krishna Kolluri
-
Patent number: 11472025Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a demonstration device for robotic demonstration learning. One of the methods includes generating, by a demonstration device for a robot, a representation of a sequence of states input by a user of the demonstration device. The representation is provided by the demonstration device to a robot execution system. The representation of the sequence of actions is translated into a plurality of robot commands corresponding to the representation of the sequence of states input by the user on the demonstration device. The plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device are executed. Demonstration data is generated from one or more sensor streams of the robot while executing the plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device.Type: GrantFiled: May 21, 2020Date of Patent: October 18, 2022Assignee: Intrinsic Innovation LLCInventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Ralf Oliver Michael Schönherr, Ning Ye
-
Patent number: 11397904Abstract: A data processing system receives location data points from computing devices. The system annotates the location data points with entities and determines a duration each of the computing devices was at corresponding entities. The system aggregates the location data points into a set of sequences based on the duration and the entities and stores the set of sequences in a data record. The system accesses the database record including a set of sequences generated from location data points received from computing devices. The system receives, from a computing device, a request for a location sequence that includes a query. The system identifies an attribute of the computing device. The system identifies a sequence based on the set of sequences using the query and the attribute. The system transmits the sequence for display on a display device.Type: GrantFiled: July 3, 2019Date of Patent: July 26, 2022Assignee: GOOGLE LLCInventors: Margaret Aycinena Lippow, Amir Jonatan Padovitz, Akshay Narendra Java, Bala Venkata Sai Ravi Krishna Kolluri
-
Patent number: 11263712Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computerized travel services. One of the methods includes identifying photographs using an index of photographs, the photographs being identified from the index as photographs geographically related to a point of interest or destination and having a creation timestamp corresponding to a time of the year; determining for each of the photographs, a relevancy score based at least in part on: selection success data of the photograph for image queries referring to the point of interest or destination, and references to the point of interest or destination in documents associated with the photograph; and selecting a selected photograph from the photographs based at least in part on a respective visual quality score and the respective relevancy scores, the visual quality score representing a degree of visual quality of the respective photographs.Type: GrantFiled: November 14, 2019Date of Patent: March 1, 2022Assignee: Google LLCInventors: Barnaby John James, Bala Venkata Sai Ravi Krishna Kolluri
-
Publication number: 20210370504Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computing interpolated robot control parameters. One of the methods includes receiving, by a real-time bridge from a control agent for a robot, a non-real-time command for the robot, wherein the non-real-time command specifies a trajectory to be attained by a component of the robot and a target value for a control parameter, wherein the control parameter controls how a real-time controller will cause the robot to react to one or more external stimuli encountered during a control cycle of the real-time controller. The real-time bridge provides the one or more real-time commands translated from the non-real-time command and interpolated control parameter information to the real-time controller, thereby causing the robot to effectuate the trajectory of the non-real-time command according to the interpolated control parameter information.Type: ApplicationFiled: May 27, 2020Publication date: December 2, 2021Inventors: Michael Beardsworth, Klas Jonas Alfred Kronander, Sean Alexander Cassero, Bala Venkata Sai Ravi Krishna Kolluri
-
Publication number: 20210362327Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributed robotic demonstration learning. One of the methods includes receiving a skill template to be trained to cause a robot to perform a particular skill having a plurality of subtasks. One or more demonstration subtasks defined by the skill template are identified, wherein each demonstration subtask is an action to be refined using local demonstration data. On online execution system uploads sets of local demonstration data to a cloud-based training system. The cloud-based training system generates respective trained model parameters for each set of local demonstration data. The skill template is executed on the robot using the trained model parameters generated by the cloud-based training system.Type: ApplicationFiled: May 21, 2020Publication date: November 25, 2021Inventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Benjamin M. Davis, Ralf Oliver Michael Schönherr, Ning Ye
-
Publication number: 20210362333Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using simulated local demonstration data for robotic demonstration learning. One of the methods includes receiving perceptual data of a workcell of a robot to be configured to execute a task according to a skill template, wherein the skill template specifies one or more subtasks required to perform the skill, wherein at least one of the subtasks is a demonstration subtask that relies on learning visual characteristics of the workcell. A virtual model is generated of a portion of the workcell. A training system generates simulated local demonstration data from the virtual model of the portion of the workcell and tunes a base control policy for the demonstration subtask using the simulated local demonstration data generated from the virtual model of the portion of the workcell.Type: ApplicationFiled: May 21, 2020Publication date: November 25, 2021Inventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Ralf Oliver Michael Schönherr, Benjamin M. Davis, Ning Ye
-
Publication number: 20210362330Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributing skill templates for robotic demonstration learning. One of the methods includes receiving, from the user device by a skill template distribution system, a selection of an available skill template. The skill template distribution system provides a skill template, wherein the skill template comprises information representing a state machine of one or more tasks, and wherein the skill template specifies which of the one or more tasks are demonstration subtasks requiring local demonstration data. The skill template distribution system trains a machine learning model for the demonstration subtask using a local demonstration data to generate learned parameter values.Type: ApplicationFiled: May 21, 2020Publication date: November 25, 2021Inventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Benjamin M. Davis, Ralf Oliver Michael Schönherr, Ning Ye
-
Publication number: 20210362329Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for integrating sensor streams for robotic demonstration learning. One of the methods includes selecting, by a learning system for a robot, a base update rate for combining multiple sensor streams into a task state representation. The learning system repeatedly generates the task state representation at the base update rate, including combining, during each time period defined by the update rate, the task state representation from most recently updated sensor data processed by the plurality of neural networks. The learning system repeatedly uses the task state representations to generate commands for the robot at the base update rate.Type: ApplicationFiled: May 21, 2020Publication date: November 25, 2021Inventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Ralf Oliver Michael Schönherr, Benjamin M. Davis, Ning Ye
-
Publication number: 20210362328Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a demonstration device for robotic demonstration learning. One of the methods includes generating, by a demonstration device for a robot, a representation of a sequence of states input by a user of the demonstration device. The representation is provided by the demonstration device to a robot execution system. The representation of the sequence of actions is translated into a plurality of robot commands corresponding to the representation of the sequence of states input by the user on the demonstration device. The plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device are executed. Demonstration data is generated from one or more sensor streams of the robot while executing the plurality of robot commands corresponding to the sequence of actions input by the user on the demonstration device.Type: ApplicationFiled: May 21, 2020Publication date: November 25, 2021Inventors: Bala Venkata Sai Ravi Krishna Kolluri, Stefan Schaal, Ralf Oliver Michael Schönherr, Ning Ye