Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate long range localization. One of the methods includes obtaining, for a particular location of a vehicle having a camera and a detection sensor, surfel data including a plurality of surfels. Each surfel in the surfel data has a respective location and corresponds to a different respective detected surface in an environment. Image data captured by the camera is obtained. It is determined that a region of interest for detecting objects for a vehicle planning process is outside a detectable region for the detection sensor. In response, it is determined that the image data for the region of interest matches surfel color data for the surfels corresponding to the region of interest. In response, the vehicle planning process is performed with the region of interest designated as having no unexpected objects.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate long range localization. One of the methods includes obtaining, for a particular location of a vehicle having a camera and a detection sensor, surfel data including a plurality of surfels. Each surfel in the surfel data has a respective location and corresponds to a different respective detected surface in an environment. Image data captured by the camera is obtained. It is determined that a region of interest for detecting objects for a vehicle planning process is outside a detectable region for the detection sensor. In response, it is determined that the image data for the region of interest matches surfel color data for the surfels corresponding to the region of interest. In response, the vehicle planning process is performed with the region of interest designated as having no unexpected objects.

Abstract: Aspects of the disclosure provide for the generation of a service area map for autonomous vehicles. For instance, graph nodes of a road network may be iterated through in order to identify a set of reachable graph nodes based on a set of routing parameters that define driving limits for the autonomous vehicles. The road network may include the graph nodes as well as edges connecting ones of the graph nodes. A set of S2 cells may be identified based on the set of reachable graph nodes. Vertices of each S2 cell of the set of S2 cells may be determined based on whether each S2 cell of the set of S2 cells is occupied by any of the graph nodes of the set of reachable graph nodes. Contours through cells may be drawn based on the scores. The service area map may be generated using the contours.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate a prediction for a state of an environment. One of the methods includes obtaining surfel data comprising a plurality of surfels, wherein each surfel corresponds to a respective different location in an environment, and each surfel has associated data that comprises an uncertainty measure; obtaining sensor data for one or more locations in the environment, the sensor data having been captured by one or more sensors of a first vehicle; determining one or more particular surfels corresponding to respective locations of the obtained sensor data; and combining the surfel data and the sensor data to generate a respective object prediction for each of the one or more locations of the obtained sensor data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate long range localization. One of the methods includes obtaining, for a particular location of a vehicle having a camera and a detection sensor, surfel data including a plurality of surfels. Each surfel in the surfel data has a respective location and corresponds to a different respective detected surface in an environment. Image data captured by the camera is obtained. It is determined that a region of interest for detecting objects for a vehicle planning process is outside a detectable region for the detection sensor. In response, it is determined that the image data for the region of interest matches surfel color data for the surfels corresponding to the region of interest. In response, the vehicle planning process is performed with the region of interest designated as having no unexpected objects.

Abstract: Aspects of the disclosure provide for the generation of a service area map for autonomous vehicles. For instance, graph nodes of a road network may be iterated through in order to identify a set of reachable graph nodes based on a set of routing parameters that define driving limits for the autonomous vehicles. The road network may include the graph nodes as well as edges connecting ones of the graph nodes. A set of S2 cells may be identified based on the set of reachable graph nodes. Vertices of each S2 cell of the set of S2 cells may be determined based on whether each S2 cell of the set of S2 cells is occupied by any of the graph nodes of the set of reachable graph nodes. Contours through cells may be drawn based on the scores. The service area map may be generated using the contours.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing data characterizing a first region in an environment to generate a prediction characterizing a second region in the environment. One of the methods includes obtaining surfel data comprising a plurality of surfels; obtaining sensor data for a plurality of locations in a first region of the environment; determining, from the surfel data, a plurality of first surfels corresponding to respective locations in the first region of the environment; determining, using the first surfels and the sensor data, a difference between i) a first representation of the first region of the environment corresponding to the first surfels and ii) a second representation of the first region of the environment corresponding to the sensor data; and generating a respective object prediction for one or more locations in a second region of the environment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate a prediction for a state of an environment. One of the methods includes obtaining surfel data comprising a plurality of surfels, wherein each surfel corresponds to a respective different location in an environment, and each surfel has associated data that comprises an uncertainty measure; obtaining sensor data for one or more locations in the environment, the sensor data having been captured by one or more sensors of a first vehicle; determining one or more particular surfels corresponding to respective locations of the obtained sensor data; and combining the surfel data and the sensor data to generate a respective object prediction for each of the one or more locations of the obtained sensor data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using traffic light viewsheds. One of the methods includes obtaining, by an on-board computer system of vehicle, viewshed data comprising a plurality of viewsheds for a plurality of respective traffic lights. An indication of a candidate driving plan that uses input from a traffic light detection of a particular traffic light having a viewshed is received. If a current location of the vehicle is within the particular viewshed for the particular traffic light, execution of the candidate driving plan is conditioned based on whether the current location of the vehicle is within the particular viewshed for the particular traffic light.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate long range localization. One of the methods includes obtaining, for a particular location of a vehicle having a camera and a detection sensor, surfel data including a plurality of surfels. Each surfel in the surfel data has a respective location and corresponds to a different respective detected surface in an environment. Image data captured by the camera is obtained. It is determined that a region of interest for detecting objects for a vehicle planning process is outside a detectable region for the detection sensor. In response, it is determined that the image data for the region of interest matches surfel color data for the surfels corresponding to the region of interest. In response, the vehicle planning process is performed with the region of interest designated as having no unexpected objects.

Abstract: Methods, systems, and apparatus for generation and use of surfel maps to plan for occlusions. One of the methods includes receiving a previously-generated surfel map depicting an area in which a vehicle is located, the surfel map comprising a plurality of surfels, each surfel corresponding to a respective different location in the area in which a vehicle is located; receiving, from one or more sensors, sensor data representing the area in which the vehicle is located; determining, based on the sensor data, that the area in which a vehicle is located includes a dynamic object having a changed shape relative to its representation in the surfel map; and generating an updated path for the vehicle to travel that avoids an occlusion by the changed shape of the dynamic object of a line of sight of one or more sensors to an area of interest.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing data characterizing a first region in an environment to generate a prediction characterizing a second region in the environment. One of the methods includes obtaining surfel data comprising a plurality of surfels; obtaining sensor data for a plurality of locations in a first region of the environment; determining, from the surfel data, a plurality of first surfels corresponding to respective locations in the first region of the environment; determining, using the first surfels and the sensor data, a difference between i) a first representation of the first region of the environment corresponding to the first surfels and ii) a second representation of the first region of the environment corresponding to the sensor data; and generating a respective object prediction for one or more locations in a second region of the environment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for autonomous driving with surfel maps. In some implementations, a three-dimensional representation of a real-world environment is obtained. Each of the surfels can correspond to a respective point of plurality of points in a three-dimensional space of the real-world environment. Input sensor data is received from multiple sensors installed on the autonomous vehicle. A pedestrian is detected from the input sensor data. A determination is made that the pedestrian is located behind a barrier. A driving plan is updated based on determining that the pedestrian is located behind a barrier.

Abstract: Aspects of the disclosure provide for the generation of a service area map for autonomous vehicles. For instance, graph nodes of a road network may be iterated through in order to identify a set of reachable graph nodes based on a set of routing parameters that define driving limits for the autonomous vehicles. The road network may include the graph nodes as well as edges connecting ones of the graph nodes. A set of S2 cells may be identified based on the set of reachable graph nodes. Vertices of each S2 cell of the set of S2 cells may be determined based on whether each S2 cell of the set of S2 cells is occupied by any of the graph nodes of the set of reachable graph nodes. Contours through cells may be drawn based on the scores. The service area map may be generated using the contours.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate a prediction for a state of an environment. One of the methods includes obtaining surfel data comprising a plurality of surfels, wherein each surfel corresponds to a respective different location in an environment, and each surfel has associated data that comprises an uncertainty measure; obtaining sensor data for one or more locations in the environment, the sensor data having been captured by one or more sensors of a first vehicle; determining one or more particular surfels corresponding to respective locations of the obtained sensor data; and combining the surfel data and the sensor data to generate a respective object prediction for each of the one or more locations of the obtained sensor data.