Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for intervention behavior prediction. A method includes receiving data characterizing a scene that includes a first agent and a second agent and receiving intervention data specifying a planned intervention to be performed by the second agent. A conditional behavior prediction output that assigns, to each of a plurality of possible future behaviors, (i) a respective conditional likelihood that the first agent performs the possible future behavior given that the second agent performs the planned intervention and (ii) a predicted value of a confounder variable for the possible future behavior is-generated using a conditional behavior prediction model.

Abstract: A method includes obtaining, by a processing device, input data derived from a set of sensors of an autonomous vehicle (AV), generating, by the processing device using a set of lane detection classifier heads, at least one heatmap based on a fused bird's eye view (BEV) feature generated from the input data, obtaining, by the processing device, a set of polylines using the at least one heatmap, wherein each polyline of the set of polylines corresponds to a respective track of a first set of tracks for a first frame, and generating, by the processing device, a second set of tracks for a second frame after the first frame by using a statistical filter based on a set of extrapolated tracks for the second frame and a set of track measurements for the second frame, wherein each track measurement of the set of track measurements corresponds to a respective updated polyline obtained for the second frame.

Abstract: The technology relates to maneuvering a vehicle prior to making a turn from a current lane of the vehicle. As an example, a route that includes making the turn from the lane is identified. An area of the lane prior to the turn having a lane width of at least a predetermined size is also identified. Sensor data identifying an object within the lane is received from a perception system of a vehicle. Characteristics of the object, including a location of the object relative to the vehicle, are identified from the sensor data. The vehicle is then maneuvered through the area prior to making the turn using the identified characteristics.

Abstract: Aspects of the disclosure relate to methods for controlling a vehicle having an autonomous driving mode. For instance, sensor data may be received from one or more sensors of the perception system of the vehicle, the sensor data identifying characteristics of an object perceived by the perception system. When it is determined that the object is no longer being perceived by the one or more sensors of the perception system, predicted characteristics for the object may be generated based on one or more of the identified characteristics. The predicted characteristics of the object may be used to control the vehicle in the autonomous driving mode such that the vehicle is able to respond to the object when it is determined that the object is no longer being perceived by the one or more sensors of the perception system.

Abstract: An example method involves detecting a sensor-testing trigger. Detecting the sensor-testing trigger may comprise determining that a vehicle is within a threshold distance to a target in an environment of the vehicle. The method also involves obtaining sensor data collected by a sensor of the vehicle after the detection of the sensor-testing trigger. The sensor data is indicative of a scan of a region of the environment that includes the target. The method also involves comparing the sensor data with previously-collected sensor data indicating detection of the target by one or more sensors during one or more previous scans of the environment. The method also involves generating performance metrics related to the sensor of the vehicle based on the comparison.

Abstract: Described examples relate to an apparatus comprising a first sensor configured to scan an area of interest during a first time period and a second sensor configured to capture a plurality of images of a field of view. The apparatus may include at least one controller configured to receive the plurality of images captured by the second sensor, compare the timestamp information associated with at least one image of the plurality of images to at least one time period of the first time period, and select a base image from the plurality of images based on the comparison.

Abstract: Example implementations may relate to determining a strategy for a drop process associated with a light detection and ranging (LIDAR) device. In particular, the LIDAR device could emit light pulses and detect return light pulses, and could generate a set of data points representative of the detected return light pulses. The drop process could involve a computing system discarding data point(s) of the set and/or preventing emission of light pulse(s) by the LIDAR device. Accordingly, the computing system could detect a trigger to engage in the drop process, and may responsively (i) use information associated with the environment around the vehicle, operation of the vehicle, and/or operation of the LIDAR device as a basis to determine the strategy for the drop process, and (ii) engage in the drop process in accordance with the determined strategy.

Abstract: One example method involves generating a calibration control signal that causes an actuator to rotate a first platform at least one complete rotation about an axis. The method also involves receiving encoder output signals. The encoder output signals are indicative of angular positions of the first platform about the axis. The method also involves receiving sensor output signals from an orientation sensor mounted on the first platform. The sensor output signals are indicative of a rate of change to an orientation of the orientation sensor. The method also involves determining calibration data based on given sensor output signals received from the orientation sensor during the at least one complete rotation. The calibration data is for mapping the encoder output signals to calibrated measurements of the angular positions of the first platform about the axis.

Abstract: The technology involves evaluating components of an autonomous vehicle that can be meaningfully evaluated over a single operational iteration. One or more snapshots of single iteration scenarios can be tested quickly and efficiently, either on vehicle or via a back-end system. Each snapshot corresponds to a particular point in time when a given component runs. Each snapshot contains a serialized set of inputs necessary to evaluate the particular component. These inputs comprise the minimal amount of information needed to accurately and faithfully recreate what the component did or does. Each snapshot is triggered at the particular point in time based on one or more criteria associated with either a driving scenario or a component of the vehicle during autonomous driving. A serialized snapshot may be retrieved from storage and deserialized, so that the system may evaluate the state of the component at the particular point in time.

Abstract: A method is provided that includes receiving user input identifying a travel destination for a first vehicle, determining, by a processor, a first route for the first vehicle to follow, and configuring the first vehicle to follow the first route. The method further includes obtaining a model for a second vehicle that shares a road with the first vehicle and comparing model to a pre-determined template for a vehicle that is known to be a special purpose vehicle in order to determine whether the first template and the second template match. The method further includes determining, by the processor, a second route that leads to the travel destination, when a match is found to exist, and switching the first vehicle from following the first route to following the second route.

Abstract: The disclosed systems and techniques facilitate efficient detection and classification of traffic signs in driving environments. The disclosed techniques include, obtaining, using a sensing system of a vehicle a first set of perspective camera images of an environment and a second set of radar images of the environment. The techniques further include generating, using a first neural network, one or more camera features characterizing the first set of images, generating, using a second neural network, one or more radar features characterizing the second set of images, and processing the one or more camera features and the one or more radar features to obtain an identification of one or more traffic signs in the environment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing sensor data, e.g., laser sensor data, using neural networks. One of the methods includes obtaining a temporal sequence of multiple three-dimensional point clouds generated from sensor readings of an environment collected by one or more sensors within a given time period, each three-dimensional point cloud comprising a respective plurality of points in a first coordinate system; processing, using a feature extraction neural network, an input that comprises data derived from the temporal sequence of multiple three-dimensional point clouds to generate a feature embedding; receiving a query that specifies one time point within the given time period; and generating, from the feature embedding and conditioned on the query, one or more outputs that characterize one or more objects in the environment at the time point specified in the received query.

Abstract: The technology provides enhanced localization approaches using vehicle-obtained information in place of or to enhance global positioning information to localize a user's position. Such information can be shared with the user's client device in real-time prior to pickup or meeting at a selected location. This can supplement or replace inaccurate localization information available at the client device, and can be done as needed when the user is within a threshold range of one or more autonomous vehicles. A client device of a user can compute its position using the vehicle positioning information. The vehicle positioning approach may be performed when the user is within a certain range of one or more vehicles. Vehicle localization information may also be used to correct the client device's localization information. Here, using hyper-accurate vehicle positioning, one or more vehicles can compute pseudorange errors for each “visible” satellite in a global positioning service.

Abstract: The technology employs spatial audio information to enhance wayfinding for pickup, drop-off and in-vehicle situations. The spatial information has a directional component, and a sense of distance can also be incorporated into the audio information. Audio cues or other spatial information is provided via headphones worn by a user. The spatial audio gives the user direction information, which can help locate the vehicle. In addition, this approach can be used when the rider is in the vehicle prior to exiting. For instance, spatial audio can be provided to the rider to give them contextual information about the environment outside the vehicle prior to exiting, such as whether a bicyclist is approaching on the side they will be exiting. This contextual information can alert the rider to wait or otherwise be more situationally aware when departing the vehicle.

Abstract: The technology involves pullover maneuvers for vehicles operating in an autonomous driving mode. A vehicle operating in an autonomous driving mode is able to identify parking locations that may be occluded or otherwise not readily visible. A best case estimation for any potential parking locations, including partially or fully occluded areas, is compared against predictions for identified visible parking locations. This can include calculating a cost for each potential parking location and comparing that cost to a baseline cost. Upon determining that an occluded location would provide a more viable option than any visible parking locations, the vehicle is able to initiate a driving adjustment (e.g., slow down) prior to arriving at the occluded location. This enables the vehicle to minimize passenger discomfort while also providing notice to other road users of an intent to pull over.

Abstract: Aspects of the disclosure provide a method of controlling an autonomous vehicle. The method may include receiving sensor data identifying a road user in a lane adjacent to the autonomous vehicle. A minimum magnitude of road user behavior for the road user that is likely to result in a collision with the autonomous vehicle. A collision cost may be determined based on the determined minimum magnitude of road user behavior and a probability density function of previously observed magnitudes of road user behavior. The autonomous vehicle may be controlled based on the collision cost.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating trajectory predictions for one or more target agents, e.g., a vehicle, a cyclist, or a pedestrian, in an environment. In one aspect, one of the methods include: obtaining scene context data characterizing a scene at a current time point in an environment that includes multiple target agents; generating, from the scene context data, an encoded representation of the scene in the environment; and generating, by a diffusion model based on the encoded representation, a respective trajectory prediction output that predicts a respective future trajectory for each of the multiple target agents after the current time point.

Abstract: Aspects of the technology employs snapshots, which are serialized sets of inputs for evaluating a components of a vehicle's autonomous driving system. Snapshots can be performed very quickly with minimal compile latency and runtime, allowing rapid evaluation of individual autonomous vehicle elements in isolation. Parameter variation and changes to initial conditions associated with a given component may be applied via a robust user interface (UI), enabling a user to perform permutations to a component or scenario and receive immediate feedback, e.g., graphically presented in a three-dimensional (3D) scene to understand how the vehicle is operating. The UI may include various control elements to enable real-time adjustment to the parameters or conditions of a snapshot. This can include one or more mutations of a snapshot to evaluate different situations or conditions, which can further aid in understanding how the vehicle would function in an autonomous driving mode.