Patents by Inventor Kurt Konolige
Kurt Konolige has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20230398683Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.Type: ApplicationFiled: June 22, 2023Publication date: December 14, 2023Inventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
-
Patent number: 11727593Abstract: Methods for annotating objects within image frames are disclosed. Information is obtained that represents a camera pose relative to a scene. The camera pose includes a position and a location of the camera relative to the scene. Data is obtained that represents multiple images, including a first image and a plurality of other images, being captured from different angles by the camera relative to the scene. A 3D pose of the object of interest is identified with respect to the camera pose in at least the first image. A 3D bounding region for the object of interest in the first image is defined, which indicates a volume that includes the object of interest. A location and orientation of the object of interest is determined in the other images based on the defined 3D bounding region of the object of interest and the camera pose in the other images.Type: GrantFiled: August 9, 2021Date of Patent: August 15, 2023Assignee: Google LLCInventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser, Paul Wohlhart
-
Patent number: 11691273Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.Type: GrantFiled: November 5, 2021Date of Patent: July 4, 2023Assignee: X DEVELOPMENT LLCInventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
-
Publication number: 20230090275Abstract: A method includes determining, based on a similarity measure, correlation scores between a target pixel and each of a candidate matching pixel, a left neighboring pixel of the candidate matching pixel, and a right neighboring pixel of the candidate matching pixel. When the correlation scores indicate that a neighboring pixel of the candidate matching pixel is better correlated to the target pixel than the candidate matching pixel, (i) a correlation score is determined between the target pixel and a second neighboring pixel of the neighboring pixel that is opposite the candidate matching pixel in the second image and (ii) an interpolation is fit to the correlation scores of the neighboring pixel, the second neighboring pixel, and the candidate matching pixel.Type: ApplicationFiled: August 17, 2022Publication date: March 23, 2023Inventor: Kurt Konolige
-
Patent number: 11565401Abstract: Methods and apparatus related to receiving a request that includes robot instructions and/or environmental parameters, operating each of a plurality of robots based on the robot instructions and/or in an environment configured based on the environmental parameters, and storing data generated by the robots during the operating. In some implementations, at least part of the stored data that is generated by the robots is provided in response to the request and/or additional data that is generated based on the stored data is provided in response to the request.Type: GrantFiled: March 22, 2021Date of Patent: January 31, 2023Assignee: X DEVELOPMENT LLCInventors: Peter Pastor Sampedro, Mrinal Kalakrishnan, Ali Yahya Valdovinos, Adrian Li, Kurt Konolige, Vincent Dureau
-
Patent number: 11425353Abstract: An example method includes receiving first and second images from a first and second imager. The method further includes determining first correlation scores between a target pixel from the first image and respective pixels from the second image based on a first similarity measure. The method also includes identifying a candidate matching pixel to the target pixel based on the first correlation scores. The method further includes determining second correlation scores between the target pixel and each of the candidate matching pixel, a left neighboring pixel, and a right neighboring pixel based on a second similarity measure. The method also includes determining whether the candidate matching pixel is better correlated to the target pixel than each of the neighboring pixels based on the second correlation scores. After determining that the candidate matching pixel is better correlated, the method further includes determining a disparity value by interpolating the second correlation scores.Type: GrantFiled: November 18, 2016Date of Patent: August 23, 2022Assignee: X Development LLCInventor: Kurt Konolige
-
Patent number: 11383380Abstract: Example embodiments may relate to methods and systems for selecting a grasp point on an object. In particular, a robotic manipulator may identify characteristics of a physical object within a physical environment. Based on the identified characteristics, the robotic manipulator may determine potential grasp points on the physical object corresponding to points at which a gripper attached to the robotic manipulator is operable to grip the physical object. Subsequently, the robotic manipulator may determine a motion path for the gripper to follow in order to move the physical object to a drop-off location for the physical object and then select a grasp point, from the potential grasp points, based on the determined motion path. After selecting the grasp point, the robotic manipulator may grip the physical object at the selected grasp point with the gripper and move the physical object through the determined motion path to the drop-off location.Type: GrantFiled: November 18, 2019Date of Patent: July 12, 2022Assignee: Intrinsic Innovation LLCInventors: Gary Bradski, Steve Croft, Kurt Konolige, Ethan Rublee, Troy Straszheim, John Zevenbergen, Stefan Hinterstoisser, Hauke Strasdat
-
Patent number: 11341364Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network that is used to control a robotic agent interacting with a real-world environment.Type: GrantFiled: September 20, 2018Date of Patent: May 24, 2022Assignee: Google LLCInventors: Konstantinos Bousmalis, Alexander Irpan, Paul Wohlhart, Yunfei Bai, Mrinal Kalakrishnan, Julian Ibarz, Sergey Vladimir Levine, Kurt Konolige, Vincent O. Vanhoucke, Matthew Laurance Kelcey
-
Publication number: 20220058419Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.Type: ApplicationFiled: November 5, 2021Publication date: February 24, 2022Inventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
-
Patent number: 11209265Abstract: Methods and systems for depth sensing are provided. A system includes a first and second optical sensor each including a first plurality of photodetectors configured to capture visible light interspersed with a second plurality of photodetectors configured to capture infrared light within a particular infrared band. The system also includes a computing device configured to (i) identify first corresponding features of the environment between a first visible light image captured by the first optical sensor and a second visible light image captured by the second optical sensor; (ii) identify second corresponding features of the environment between a first infrared light image captured by the first optical sensor and a second infrared light image captured by the second optical sensor; and (iii) determine a depth estimate for at least one surface in the environment based on the first corresponding features and the second corresponding features.Type: GrantFiled: September 26, 2019Date of Patent: December 28, 2021Assignee: X Development LLCInventor: Kurt Konolige
-
Patent number: 11195041Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.Type: GrantFiled: May 1, 2020Date of Patent: December 7, 2021Assignee: X DEVELOPMENT LLCInventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
-
Patent number: 11151744Abstract: Methods for annotating objects within image frames are disclosed. Information is obtained that represents a camera pose relative to a scene. The camera pose includes a position and a location of the camera relative to the scene. Data is obtained that represents multiple images, including a first image and a plurality of other images, being captured from different angles by the camera relative to the scene. A 3D pose of the object of interest is identified with respect to the camera pose in at least the first image. A 3D bounding region for the object of interest in the first image is defined, which indicates a volume that includes the object of interest. A location and orientation of the object of interest is determined in the other images based on the defined 3D bounding region of the object of interest and the camera pose in the other images.Type: GrantFiled: September 16, 2019Date of Patent: October 19, 2021Assignee: X Development LLCInventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser, Paul Wohlhart
-
Publication number: 20210187736Abstract: Example methods and systems for determining 3D scene geometry by projecting patterns of light onto a scene are provided. In an example method, a first projector may project a first random texture pattern having a first wavelength and a second projector may project a second random texture pattern having a second wavelength. A computing device may receive sensor data that is indicative of an environment as perceived from a first viewpoint of a first optical sensor and a second viewpoint of a second optical sensor. Based on the received sensor data, the computing device may determine corresponding features between sensor data associated with the first viewpoint and sensor data associated with the second viewpoint. And based on the determined corresponding features, the computing device may determine an output including a virtual representation of the environment that includes depth measurements indicative of distances to at least one object.Type: ApplicationFiled: March 9, 2021Publication date: June 24, 2021Inventors: Gary Bradski, Kurt Konolige, Ethan Rublee
-
Patent number: 10981270Abstract: Methods and apparatus related to receiving a request that includes robot instructions and/or environmental parameters, operating each of a plurality of robots based on the robot instructions and/or in an environment configured based on the environmental parameters, and storing data generated by the robots during the operating. In some implementations, at least part of the stored data that is generated by the robots is provided in response to the request and/or additional data that is generated based on the stored data is provided in response to the request.Type: GrantFiled: August 2, 2019Date of Patent: April 20, 2021Assignee: X DEVELOPMENT LLCInventors: Peter Pastor Sampedro, Mrinal Kalakrishnan, Ali Yahya Valdovinos, Adrian Li, Kurt Konolige, Vincent Dureau
-
Patent number: 10967506Abstract: Example methods and systems for determining 3D scene geometry by projecting patterns of light onto a scene are provided. In an example method, a first projector may project a first random texture pattern having a first wavelength and a second projector may project a second random texture pattern having a second wavelength. A computing device may receive sensor data that is indicative of an environment as perceived from a first viewpoint of a first optical sensor and a second viewpoint of a second optical sensor. Based on the received sensor data, the computing device may determine corresponding features between sensor data associated with the first viewpoint and sensor data associated with the second viewpoint. And based on the determined corresponding features, the computing device may determine an output including a virtual representation of the environment that includes depth measurements indicative of distances to at least one object.Type: GrantFiled: November 30, 2017Date of Patent: April 6, 2021Assignee: X Development LLCInventors: Gary Bradski, Kurt Konolige, Ethan Rublee
-
Publication number: 20200279134Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network that is used to control a robotic agent interacting with a real-world environment.Type: ApplicationFiled: September 20, 2018Publication date: September 3, 2020Inventors: Konstantinos Bousmalis, Alexander Irpan, Paul Wohlhart, Yunfei Bai, Mrinal Kalakrishnan, Julian Ibarz, Sergey Vladimir Levine, Kurt Konolige, Vincent O. Vanhoucke, Matthew Laurance Kelcey
-
Publication number: 20200265260Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.Type: ApplicationFiled: May 1, 2020Publication date: August 20, 2020Inventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
-
Patent number: 10671874Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.Type: GrantFiled: July 23, 2018Date of Patent: June 2, 2020Assignee: X DEVELOPMENT LLCInventors: Kurt Konolige, Nareshkumar Rajkumar, Stefan Hinterstoisser
-
Patent number: 10656646Abstract: An example method includes determining a target area of a ground plane in an environment of a mobile robotic device, where the target area of the ground plane is in front of the mobile robotic device in a direction of travel of the mobile robotic device. The method further includes receiving depth data from a depth sensor on the mobile robotic device. The method also includes identifying a portion of the depth data representative of the target area. The method additionally includes determining that the portion of the depth data lacks information representing at least one section of the target area. The method further includes providing an output signal identifying at least one zone of non-traversable space for the mobile robotic device in the environment, where the at least one zone of non-traversable space corresponds to the at least one section of the target area.Type: GrantFiled: December 14, 2017Date of Patent: May 19, 2020Assignee: X Development LLCInventors: Kevin William Watts, Kurt Konolige
-
Publication number: 20200078938Abstract: Example embodiments may relate to methods and systems for selecting a grasp point on an object. In particular, a robotic manipulator may identify characteristics of a physical object within a physical environment. Based on the identified characteristics, the robotic manipulator may determine potential grasp points on the physical object corresponding to points at which a gripper attached to the robotic manipulator is operable to grip the physical object. Subsequently, the robotic manipulator may determine a motion path for the gripper to follow in order to move the physical object to a drop-off location for the physical object and then select a grasp point, from the potential grasp points, based on the determined motion path. After selecting the grasp point, the robotic manipulator may grip the physical object at the selected grasp point with the gripper and move the physical object through the determined motion path to the drop-off location.Type: ApplicationFiled: November 18, 2019Publication date: March 12, 2020Inventors: Gary Bradski, Steve Croft, Kurt Konolige, Ethan Rublee, Troy Straszheim, John Zevenbergen, Stefan Hinterstoisser, Hauke Strasdat