Patents by Inventor Michael Montemerlo
Michael Montemerlo has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11961272Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate long range localization. One of the methods includes obtaining, for a particular location of a vehicle having a camera and a detection sensor, surfel data including a plurality of surfels. Each surfel in the surfel data has a respective location and corresponds to a different respective detected surface in an environment. Image data captured by the camera is obtained. It is determined that a region of interest for detecting objects for a vehicle planning process is outside a detectable region for the detection sensor. In response, it is determined that the image data for the region of interest matches surfel color data for the surfels corresponding to the region of interest. In response, the vehicle planning process is performed with the region of interest designated as having no unexpected objects.Type: GrantFiled: May 24, 2023Date of Patent: April 16, 2024Assignee: Waymo LLCInventors: David Yonchar Margines, Carlos Hernandez Esteban, Michael Montemerlo, Peter Pawlowski, Colin Andrew Braley
-
Patent number: 11853070Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for updating data of a map. One of the methods includes storing a map of an environment, the map comprising surfels and a road graph; receiving new surfel data for the surfels; adjusting the surfels based on the new surfel data; determining a vector field difference between the surfels of the stored map and the adjusted surfels; adjusting a portion of the road graph based on the vector field difference; generating an updated map comprising the adjusted surfels and the adjusted portion of the road graph; and storing the updated map.Type: GrantFiled: September 4, 2020Date of Patent: December 26, 2023Assignee: Waymo LLCInventors: Michael Montemerlo, Peter Michal Pawlowski, Joy Weng Zhang
-
Publication number: 20230334776Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sensor calibration with environment map. In some implementations, a three-dimensional surfel representation of a real-world environment is obtained. One or more surfels of the surfel representation having a particular classification of the different classifications are selected. Input sensor data from one or more sensors installed on an autonomous or semi-autonomous vehicle are received. The input sensor data is compared to the surfel representation to identify one or more differences between the observation and the surfel representation. At least one sensor of the one or more sensors is calibrated using the one or more differences between the observation and the surfel representation.Type: ApplicationFiled: June 21, 2023Publication date: October 19, 2023Inventors: Christian Lauterbach, Colin Andrew Braley, Carlos Hernandez Esteban, Michael Montemerlo
-
Publication number: 20230298301Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate long range localization. One of the methods includes obtaining, for a particular location of a vehicle having a camera and a detection sensor, surfel data including a plurality of surfels. Each surfel in the surfel data has a respective location and corresponds to a different respective detected surface in an environment. Image data captured by the camera is obtained. It is determined that a region of interest for detecting objects for a vehicle planning process is outside a detectable region for the detection sensor. In response, it is determined that the image data for the region of interest matches surfel color data for the surfels corresponding to the region of interest. In response, the vehicle planning process is performed with the region of interest designated as having no unexpected objects.Type: ApplicationFiled: May 24, 2023Publication date: September 21, 2023Inventors: David Yonchar Margines, Carlos Hernandez Esteban, Michael Montemerlo, Peter Pawlowski, Colin Andrew Braley
-
Patent number: 11761769Abstract: Aspects of the disclosure provide for localizing a vehicle. In one instance, a weather condition in which the vehicle is currently driving may be identified. A plurality of sensor inputs including intensity information, elevation information, and radar sensor information may be received. For each of the plurality of sensor inputs, an alignment score is determined by comparing the intensity information, elevation information, and radar sensor information to a corresponding pre-stored image for each of the intensity information, the elevation information, and the radar sensor information. A set of weights for the plurality of sensor inputs may be determined based on the identified weather condition. The alignment scores may then be combined using the set of weights in order to localize the vehicle.Type: GrantFiled: March 30, 2022Date of Patent: September 19, 2023Assignee: Waymo LLCInventors: Samuel Panzer, Michael Montemerlo, Michael James
-
Publication number: 20230281866Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate a prediction for a state of an environment. One of the methods includes obtaining surfel data comprising a plurality of surfels, wherein each surfel corresponds to a respective different location in an environment, and each surfel has associated data that comprises an uncertainty measure; obtaining sensor data for one or more locations in the environment, the sensor data having been captured by one or more sensors of a first vehicle; determining one or more particular surfels corresponding to respective locations of the obtained sensor data; and combining the surfel data and the sensor data to generate a respective object prediction for each of the one or more locations of the obtained sensor data.Type: ApplicationFiled: December 30, 2022Publication date: September 7, 2023Inventors: Carlos Hernandez Esteban, Michael Montemerlo, Peter Pawlowski, David Yonchar Margines
-
Patent number: 11734880Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sensor calibration with environment map. In some implementations, a three-dimensional surfel representation of a real-world environment is obtained. One or more surfels of the surfel representation having a particular classification of the different classifications are selected. Input sensor data from one or more sensors installed on an autonomous or semi-autonomous vehicle are received. The input sensor data is compared to the surfel representation to identify one or more differences between the observation and the surfel representation. At least one sensor of the one or more sensors is calibrated using the one or more differences between the observation and the surfel representation.Type: GrantFiled: December 31, 2020Date of Patent: August 22, 2023Assignee: Waymo LLCInventors: Christian Lauterbach, Colin Andrew Braley, Carlos Hernandez Esteban, Michael Montemerlo
-
Patent number: 11699274Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate long range localization. One of the methods includes obtaining, for a particular location of a vehicle having a camera and a detection sensor, surfel data including a plurality of surfels. Each surfel in the surfel data has a respective location and corresponds to a different respective detected surface in an environment. Image data captured by the camera is obtained. It is determined that a region of interest for detecting objects for a vehicle planning process is outside a detectable region for the detection sensor. In response, it is determined that the image data for the region of interest matches surfel color data for the surfels corresponding to the region of interest. In response, the vehicle planning process is performed with the region of interest designated as having no unexpected objects.Type: GrantFiled: October 6, 2020Date of Patent: July 11, 2023Assignee: Waymo LLCInventors: David Yonchar Margines, Carlos Hernandez Esteban, Michael Montemerlo, Peter Pawlowski, Colin Andrew Braley
-
Patent number: 11676392Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using surfels for vehicle localization. One of the methods includes obtaining surfel data comprising a plurality of surfels, wherein each surfel corresponds to a respective different location in an environment, and each surfel has associated data that comprises a stability measure, wherein the stability measure characterizes a permanence of a surface represented by the surfel; obtaining sensor data for a plurality of locations in the environment, the sensor data having been captured by one or more sensors of a first vehicle; determining a plurality of high-stability surfels from the plurality of surfels in the surfel data; and determining a location in the environment of the first vehicle using the plurality of selected high-stability surfels and the sensor data.Type: GrantFiled: June 3, 2020Date of Patent: June 13, 2023Assignee: Waymo LLCInventor: Michael Montemerlo
-
Patent number: 11675084Abstract: The present disclosure relates to methods and systems that facilitate determination of a pose of a vehicle based on various combinations of map data and sensor data received from light detection and ranging (LIDAR) devices and/or camera devices. An example method includes receiving point cloud data from a (LIDAR) device and transforming the point cloud data to provide a top-down image. The method also includes comparing the top-down image to a reference image and determining, based on the comparison, a yaw error. An alternative method includes receiving camera image data from a camera and transforming the camera image data to provide a top-down image. The method also includes comparing the top-down image to a reference image and determining, based on the comparison, a yaw error.Type: GrantFiled: April 23, 2020Date of Patent: June 13, 2023Assignee: Waymo LLCInventors: John Tisdale, Michael Montemerlo, Andrew Chatham
-
Publication number: 20230152461Abstract: The present disclosure relates to methods and systems that facilitate determination of a pose of a vehicle based on various combinations of map data and sensor data received from light detection and ranging (LIDAR) devices and/or camera devices. An example method includes receiving point cloud data from a (LIDAR) device and transforming the point cloud data to provide a top-down image. The method also includes comparing the top-down image to a reference image and determining, based on the comparison, a yaw error. An alternative method includes receiving camera image data from a camera and transforming the camera image data to provide a top-down image. The method also includes comparing the top-down image to a reference image and determining, based on the comparison, a yaw error.Type: ApplicationFiled: January 19, 2023Publication date: May 18, 2023Inventors: John Tisdale, Michael Montemerlo, Andrew Chatham
-
Patent number: 11561552Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing data characterizing a first region in an environment to generate a prediction characterizing a second region in the environment. One of the methods includes obtaining surfel data comprising a plurality of surfels; obtaining sensor data for a plurality of locations in a first region of the environment; determining, from the surfel data, a plurality of first surfels corresponding to respective locations in the first region of the environment; determining, using the first surfels and the sensor data, a difference between i) a first representation of the first region of the environment corresponding to the first surfels and ii) a second representation of the first region of the environment corresponding to the sensor data; and generating a respective object prediction for one or more locations in a second region of the environment.Type: GrantFiled: September 15, 2020Date of Patent: January 24, 2023Assignee: Waymo LLCInventors: David Harrison Silver, Carlos Hernandez Esteban, Michael Montemerlo, Peter Pawlowski, David Yonchar Margines
-
Patent number: 11541903Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate a prediction for a state of an environment. One of the methods includes obtaining surfel data comprising a plurality of surfels, wherein each surfel corresponds to a respective different location in an environment, and each surfel has associated data that comprises an uncertainty measure; obtaining sensor data for one or more locations in the environment, the sensor data having been captured by one or more sensors of a first vehicle; determining one or more particular surfels corresponding to respective locations of the obtained sensor data; and combining the surfel data and the sensor data to generate a respective object prediction for each of the one or more locations of the obtained sensor data.Type: GrantFiled: June 3, 2020Date of Patent: January 3, 2023Assignee: Waymo LLCInventors: Carlos Hernandez Esteban, Michael Montemerlo, Peter Pawlowski, David Yonchar Margines
-
Patent number: 11473931Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing an initial representation of an environment to generate an updated representation that includes representations of one or more objects there were obscured in the initial representation. One of the methods includes obtaining initial surfel data comprising a plurality of surfels; determining a plurality of non-static surfels from the plurality of surfels; obtaining guidance data that characterizes a prediction for static surfaces in one or more first regions of the environment that were obscured by the plurality of non-static surfels in the initial surfel data; and processing the guidance data and the initial surfel data to generate final surfel data that comprises, for each first region of the environment, a plurality of predicted surfels that represent static surfaces in the first region that are partially or wholly obscured by objects represented by respective non-static surfels.Type: GrantFiled: September 15, 2020Date of Patent: October 18, 2022Assignee: Waymo LLCInventors: Michael Montemerlo, Colin Andrew Braley
-
Publication number: 20220221280Abstract: Aspects of the disclosure provide for localizing a vehicle. In one instance, a weather condition in which the vehicle is currently driving may be identified. A plurality of sensor inputs including intensity information, elevation information, and radar sensor information may be received. For each of the plurality of sensor inputs, an alignment score is determined by comparing the intensity information, elevation information, and radar sensor information to a corresponding pre-stored image for each of the intensity information, the elevation information, and the radar sensor information. A set of weights for the plurality of sensor inputs may be determined based on the identified weather condition. The alignment scores may then be combined using the set of weights in order to localize the vehicle.Type: ApplicationFiled: March 30, 2022Publication date: July 14, 2022Inventors: Samuel Panzer, Michael Montemerlo, Michael James
-
Publication number: 20220204019Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sensor calibration with environment map. In some implementations, a three-dimensional surfel representation of a real-world environment is obtained. One or more surfels of the surfel representation having a particular classification of the different classifications are selected. Input sensor data from one or more sensors installed on an autonomous or semi-autonomous vehicle are received. The input sensor data is compared to the surfel representation to identify one or more differences between the observation and the surfel representation. At least one sensor of the one or more sensors is calibrated using the one or more differences between the observation and the surfel representation.Type: ApplicationFiled: December 31, 2020Publication date: June 30, 2022Inventors: Christian Lauterbach, Colin Andrew Braley, Carlos Hernandez Esteban, Michael Montemerlo
-
Patent number: 11320272Abstract: Aspects of the disclosure provide for localizing a vehicle. In one instance, a weather condition in which the vehicle is currently driving may be identified. A plurality of sensor inputs including intensity information, elevation information, and radar sensor information may be received. For each of the plurality of sensor inputs, an alignment score is determined by comparing the intensity information, elevation information, and radar sensor information to a corresponding pre-stored image for each of the intensity information, the elevation information, and the radar sensor information. A set of weights for the plurality of sensor inputs may be determined based on the identified weather condition. The alignment scores may then be combined using the set of weights in order to localize the vehicle.Type: GrantFiled: July 7, 2020Date of Patent: May 3, 2022Assignee: Waymo LLCInventors: Samuel Panzer, Michael Montemerlo, Michael James
-
Publication number: 20220108119Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate long range localization. One of the methods includes obtaining, for a particular location of a vehicle having a camera and a detection sensor, surfel data including a plurality of surfels. Each surfel in the surfel data has a respective location and corresponds to a different respective detected surface in an environment. Image data captured by the camera is obtained. It is determined that a region of interest for detecting objects for a vehicle planning process is outside a detectable region for the detection sensor. In response, it is determined that the image data for the region of interest matches surfel color data for the surfels corresponding to the region of interest. In response, the vehicle planning process is performed with the region of interest designated as having no unexpected objects.Type: ApplicationFiled: October 6, 2020Publication date: April 7, 2022Inventors: David Yonchar Margines, Carlos Hernandez Esteban, Michael Montemerlo, Peter Pawlowski, Colin Andrew Braley
-
Publication number: 20220082408Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing an initial representation of an environment to generate an updated representation that includes representations of one or more objects there were obscured in the initial representation. One of the methods includes obtaining initial surfel data comprising a plurality of surfels; determining a plurality of non-static surfels from the plurality of surfels; obtaining guidance data that characterizes a prediction for static surfaces in one or more first regions of the environment that were obscured by the plurality of non-static surfels in the initial surfel data; and processing the guidance data and the initial surfel data to generate final surfel data that comprises, for each first region of the environment, a plurality of predicted surfels that represent static surfaces in the first region that are partially or wholly obscured by objects represented by respective non-static surfels.Type: ApplicationFiled: September 15, 2020Publication date: March 17, 2022Inventors: Michael Montemerlo, Colin Andrew Braley
-
Publication number: 20220083073Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing data characterizing a first region in an environment to generate a prediction characterizing a second region in the environment. One of the methods includes obtaining surfel data comprising a plurality of surfels; obtaining sensor data for a plurality of locations in a first region of the environment; determining, from the surfel data, a plurality of first surfels corresponding to respective locations in the first region of the environment; determining, using the first surfels and the sensor data, a difference between i) a first representation of the first region of the environment corresponding to the first surfels and ii) a second representation of the first region of the environment corresponding to the sensor data; and generating a respective object prediction for one or more locations in a second region of the environment.Type: ApplicationFiled: September 15, 2020Publication date: March 17, 2022Inventors: David Harrison Silver, Carlos Hernandez Esteban, Michael Montemerlo, Peter Pawlowski, David Yonchar Margines