Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting a pull-over location using machine learning. One of the methods includes obtaining data specifying a target pull-over location for an autonomous vehicle travelling on a roadway. A plurality of candidate pull-over locations in a vicinity of the target pull-over location are identified. For each candidate pull-over location, an input that includes features of the candidate pull-over location is processed using a machine learning model to generate a respective likelihood score representing a predicted likelihood that the candidate pull-over location is an optimal location. The features of the candidate pull-over location include one or more features that compare the candidate pull-over location to the target pull-over location. Using the respective likelihood scores, one of the candidate pull-over locations is selected as an actual pull-over location for the autonomous vehicle.

Abstract: Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for performing three-dimensional auto-labeling on sensor data. The system obtains a sensor data segment that includes a temporal sequence of three-dimensional point clouds generated from sensor readings of an environment by one or more sensors. The system identifies, from the sensor data segment, (i) a plurality of object tracks that each corresponds to a different object in the environment and (ii) for each object track, respective initial three-dimensional regions in each of one or more of the point clouds in which the corresponding object appears. The system generates, for each object track, extracted object track data that includes at least the points in the respective initial three-dimensional regions for the object track.

Abstract: An example method involves identifying one or more potential route segments that collectively connect at least two geographical points, receiving spatiotemporal weather information that predicts future weather conditions along each of the potential segments, and, for each potential segment, evaluating a partial cost function that comprises a summation of a set of segment-weighted cost factors, where at least one segment-weighted cost factor comprises an adverse weather risk factor based on the future weather conditions along the potential segment. The method also involves selecting, based on a minimization of a total cost function, a set of selected segments and corresponding segment target speeds for the vehicle to utilize while traversing between the at least two geographical points so as to avoid adverse weather conditions, the total cost function being the sum of partial cost functions associated with a set of segments that collectively connect the at least two geographical points.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining a task schedule for generating prediction data for different agents. In one aspect, a method comprises receiving data that characterizes an environment in a vicinity of a vehicle at a current time step, the environment comprising a plurality of agents; receiving data that identifies high-priority agents for which respective data characterizing the agents must be generated at the current time step; identifying available computing resources at the current time step; processing the data that characterizes the environment using a complexity scoring model to determine a respective complexity score for each of the high-priority agents; and determining a schedule for the current time step that allocates the generation of the data characterizing the high-priority agents across the available computing resources based on the complexity scores.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for intervention behavior prediction. One of the methods includes receiving data characterizing a scene that includes a first agent and a second agent in an environment. A confounder prediction input generated from the data is processed using a confounder prediction model. A plurality of predicted conditional probability distributions is generated, wherein each predicted conditional probability distribution is conditioned on: (i) a planned intervention by the second agent, and (ii) the confounder variable belonging to a corresponding confounder class.

Abstract: Methods, systems, and apparatus for generation and use of surfel maps to plan for occlusions. One of the methods includes receiving a previously-generated surfel map depicting an area in which a vehicle is located, the surfel map comprising a plurality of surfels, each surfel corresponding to a respective different location in the area in which a vehicle is located; receiving, from one or more sensors, sensor data representing the area in which the vehicle is located; determining, based on the sensor data, that the area in which a vehicle is located includes a dynamic object having a changed shape relative to its representation in the surfel map; and generating an updated path for the vehicle to travel that avoids an occlusion by the changed shape of the dynamic object of a line of sight of one or more sensors to an area of interest.

Abstract: Aspects of the disclosure relate to enabling roadside assistance to a vehicle that requires assistance having an autonomous driving mode. For instance, a technician may be assigned to the vehicle that requires assistance. A signal corresponding to user input at a remote computing device requesting a change to a state of the vehicle may be received. The signal may be based on details of the assigned technician. Based on the validation, an instruction may be sent to the vehicle to change the state of the vehicle.

Abstract: Example embodiments relate to light detection and ranging (lidar) devices having vertical-cavity surface-emitting laser (VCSEL) emitters. An example lidar device includes an array of individually addressable VCSELs configured to emit light pulses into an environment surrounding the lidar device. The lidar device also includes a firing circuit configured to selectively fire the individually addressable VCSELs in the array. In addition, the lidar device includes a controller configured to control the firing circuit using a control signal. Further, the lidar device includes a plurality of detectors. Each detector in the plurality of detectors is configured to detect reflections of light pulses that are emitted by one or more individually addressable VCSELs in the array and reflected by one or more objects in the environment surrounding the lidar device.

Abstract: An example method includes receiving point cloud information about a field of view of a lidar system. The point cloud information includes spatiotemporal and amplitude information about return light received. The method also includes determining, based on the point cloud information, a set of bright light returns from at least one highly reflective object. The bright light returns include return light having an amplitude above a photon threshold and a corresponding bright light return range. The method yet further includes determining, based on the point cloud information, a set of crosstalk returns. The crosstalk returns include return light having a corresponding crosstalk return range. The method includes adjusting, based on a normalized number of crosstalk returns, at least one of: a cleaning system, an operating mode of a lidar system, or an operating mode of a vehicle.

Abstract: Example embodiments relate to calibration and localization of a light detection and ranging (lidar) device using a previously calibrated and localized lidar device. An example embodiment includes a method. The method includes receiving, by a computing device associated with a second vehicle, a first point cloud captured by a first lidar device of a first vehicle. The first point cloud includes points representing the second vehicle. The method also includes receiving, by the computing device, pose information indicative of a pose of the first vehicle. In addition, the method includes capturing, using a second lidar device of the second vehicle, a second point cloud. Further, the method includes receiving, by the computing device, a third point cloud representing the first vehicle. Yet further, the method includes calibrating and localizing, by the computing device, the second lidar device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining estimated ground truth object keypoint labels for sensor readings of objects. In one aspect, a method comprises obtaining a plurality of sets of label data for a sensor reading of an object; obtaining respective quality control data corresponding to each of the plurality of sets of label data, the respective quality control data comprising: data indicating whether the labeled location of the first object keypoint in the corresponding set of label data is accurate; and determining an estimated ground truth location for the first object keypoint in the sensor data keypoint from (i) the labeled locations that were indicated as accurate by the corresponding quality control data and (ii) not from the labeled locations that were indicated as not accurate by the corresponding quality control data.

Abstract: The technology relates to determining general weather conditions affecting the roadway around a vehicle, and how such conditions may impact driving and route planning for the vehicle when operating in an autonomous mode. For instance, the on-board sensor system may detect whether the road is generally icy as opposed to a small ice patch on a specific portion of the road surface. The system may also evaluate specific driving actions taken by the vehicle and/or other nearby vehicles. Based on such information, the vehicle's control system is able to use the resultant information to select an appropriate braking level or braking strategy. As a result, the system can detect and respond to different levels of adverse weather conditions. The on-board computer system may share road condition information with nearby vehicles and with remote assistance, so that it may be employed with broader fleet planning operations.

Abstract: Example embodiments relate to techniques for three dimensional (3D) object detection and localization. A computing system may cause a radar unit to transmit radar signals and receive radar reflections relative to an environment of a vehicle. Based on the radar reflections, the computing system may determine a heading and a range for a nearby object. The computing system may also receive an image depicting a portion of the environment that includes the object from a vehicle camera and remove peripheral areas of the image to generate an image patch that focuses upon the object based on the heading and the range for the object. The image patch and the heading and the range for the object can be provided as inputs into a neural network that provides output parameters corresponding to the object, which can be used to control the vehicle.

Abstract: Example embodiments relate to techniques for detecting adverse road conditions using radar. A computing device may generate a first radar representation that represents a field of view for a radar unit coupled to a vehicle and during clear weather conditions and store the first radar representation in memory. The computing device may receive radar data from the radar unit during navigation of the vehicle on a road and determine a second radar representation based on the radar data. The computing device may also perform a comparison between the first radar representation and the second radar representation and determine a road condition for the road based on the comparison. The road condition may represent a quantity of precipitation located on the road and provide control instructions to the vehicle based on the road condition for the road.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for searching an autonomous vehicle sensor data repository. One of the methods includes maintaining a collection of sensor samples and one or more embeddings of each sensor sample. Each sensor sample is generated from sensor data at multiple time steps and characterizes an environment at each of the multiple time steps. Each embedding corresponds to a respective portion of the sensor sample and has been generated by an embedding neural network. A query specifying a query portion of a query sensor sample is received. A query embedding corresponding to the query portion of the query sensor sample is generated through the embedding neural network. A plurality of relevant sensor samples that have embeddings that are closest to the query embedding are identified as characterizing similar scenarios to the query portion of the query sensor sample.

Abstract: The present disclosure relates to optical systems, vehicles, and methods that are configured to illuminate and image a wide field of view of an environment. An example optical system includes a camera having an optical axis and an outer lens element disposed along the optical axis. The optical system also includes a plurality of illumination modules, each of which includes at least one light-emitter device configured to emit light along a respective emission axis and a secondary optical element optically coupled to the at least one light-emitter device. The secondary optical element is configured to provide a light emission pattern having an azimuthal angle extent of at least 170 degrees so as to illuminate a portion of an environment of the optical system.

Abstract: Examples described relate to systems, methods, and apparatus for transmitting image data. The apparatus may comprise a memory buffer configured to store data elements of an image frame and generate a signal indicating that each of the data elements of the image frame have been written to the memory buffer, a processor configured to initiate a flush operation for reading out at least one data element of the image frame from the memory buffer and to output the at least one data element at a first rate, a rate adjustment unit configured to receive the at least one data element from the processor at the first rate and to output the at least one data element at a second rate, and a multiplexer configured to receive the at least one data element from the processor at the first rate and configured to receive the at least one data element from the rate adjustment unit at the second rate.