Abstract: This application relates to techniques for dynamically determining whether to engage an autonomous controller of a vehicle. A computing system may receive a request to engage the autonomous controller (e.g., autonomous mode) of the vehicle. In some examples, the request may be received from a simulation computing system configured to test an updated autonomous controller in a simulation. Based on a determination that conditions associated with engaging autonomy are satisfied, the computing system engages the autonomous controller. Based on a determination that conditions associated with engaging autonomy are not satisfied, the computing system disables the engagement of the autonomous controller such that the vehicle is controlled according to an initial operational mode (e.g., manual mode, semi-autonomous mode, previous version of the autonomous controller, etc.).

Abstract: Techniques for performing multiple simultaneous pose generation for an autonomous vehicle. For instance, a system that navigates the autonomous vehicle can include at least a first component that determines first poses for the autonomous vehicle using at least a first portion of sensor data captured by one or more sensors and a second component that determines second poses for the autonomous vehicle using at least a second portion of the sensor data. The first component may have more computational resources than the second component and determine poses at a different frequency than the second component. The system may generate trajectories for the autonomous vehicle using the first poses when the first component is operating correctly. Additionally, the system may generate trajectories for the autonomous vehicle using the second poses when the first component is not operating correctly.

Abstract: A machine-learned (ML) model for detecting that depth data (e.g., lidar data, radar data) comprises a false positive attributable to particulate matter, such as dust, steam, smoke, rain, etc. The ML model may be trained based at least in part on simulated depth data generated by a fluid dynamics model and/or by collecting depth data during operation of a device (e.g., an autonomous vehicle. In some examples, an autonomous vehicle may identify depth data that may be associated with particulate matter based at least in part on an outlier region in a thermal image. For example, the outlier region may be associated with steam.

Abstract: A sensor simulation system may generate sensor data for use in simulations by rendering two-dimensional views of a three-dimensional simulated environment. In various examples, the sensor simulation system uses sensor dependency data to determine specific views to be re-rendered at different times during the simulation. The sensor simulation system also may generate unified views with multi-sensor data at each region (e.g., pixel) of the two-dimensional view for consumption by different sensor types. A hybrid technique may be used in some implementations in which rasterization is used to generate a view, after which ray tracing is used to align the view with a particular sensor. Spatial and temporal upsampling techniques also may be used, including depth-aware and velocity-aware analyses for simulated objects, to improve view resolution and reduce the frequency of re-rendering views.

Abstract: Techniques are discussed for controlling a vehicle, such as an autonomous vehicle, based on occluded areas in an environment. An occluded area can represent areas where sensors of the vehicle are unable to sense portions of the environment due to obstruction by another object or sensor limitation. An occluded region for an object is determined, along with one or more visible regions proximate the occluded region. Entry and/or exit regions may be determined based on known directions of traffic and/or drivable surface boundaries. Based on a threshold speed, the vehicle can designate portions of the occluded region as pseudo-visible. Additionally, if a dynamic object traverses through the occluded region from the entry region, a portion of the occluded region may be considered pseudo-visible. Pseudo-visibility may also be determined based on movement of an occluded area. The vehicle can be controlled to traverse the environment based on the pseudo-visibility.

Abstract: A LIDAR sensor assembly includes a laser light source to emit laser light, and a light sensor to produce a light signal in response to sensing reflections of the laser light emitted by the laser light source from a reference surface that is fixed in relation to the LIDAR sensor assembly. A controller of the LIDAR sensor assembly can process a plurality of samples of reflected light signals, process the samples to remove erroneous readings, and then provide accurate distance measurement. The system can use low-pass filters, or other components, to filter the plurality of samples to enable the “actual,” or primary, reflected light signal (i.e., light signal reflected off of a surface in an environment external to the sensor assembly, as opposed to extraneous, internal reflections off of lenses or other components or noise) to be identified and an accurate time of flight to be calculated.

Abstract: Techniques for determining predictions on a top-down representation of an environment based on object movement are discussed herein. Sensors of a first vehicle (such as an autonomous vehicle) may capture sensor data of an environment, which may include object(s) separate from the first vehicle (e.g., a vehicle, a pedestrian, a bicycle). A multi-channel image representing a top-down view of the object(s) and the environment may be generated based in part on the sensor data. Environmental data (object extents, velocities, lane positions, crosswalks, etc.) may also be encoded in the image. Multiple images may be generated representing the environment over time and input into a prediction system configured to output a trajectory template (e.g., general intent for future movement) and a predicted trajectory (e.g., more accurate predicted movement) associated with each object. The prediction system may include a machine learned model configured to output the trajectory template(s) and the predicted trajector(ies).

Abstract: Aspects of validating information sent to and/or received by a computing device, such as a vehicle computing device, are discussed herein. A vehicle may receive information associated with a computing device, the information indicating that the computing device is authorized to provide guidance information to an autonomous vehicle. Further, the vehicle can receive a message comprising the guidance information associated with the autonomous vehicle. The vehicle can determine that the message is associated with the computing device and can evaluate the message based at least in part on configuration data associated with maneuvering the autonomous vehicle or state information associated with the autonomous vehicle. The vehicle can be controlled based at least in part on evaluating the message.

Abstract: A vehicle computing system may implement techniques to emit warning sounds from a vehicle to notify other entities in an environment with the vehicle of the vehicle operation. A vehicle computing system may be configured to emit warning sounds based on a speed associated with the vehicle. The vehicle computing system may determine a planned vehicle trajectory and that a speed corresponding to the planned vehicle trajectory is associated with the warning sound emission. The vehicle computing system may determine a time that the vehicle will be at the speed and may emit the warning sound substantially at the time. In some examples, the vehicle computing system may emit a different sound prior to the time and may transition to the warning sound associated with the speed at the time. By warning other entities of the vehicle operation, the warning sound emission system may improve the safety of vehicle operation.

Abstract: Methods and system for disinfecting an autonomous vehicle includes one or more LEDs coupled to the autonomous vehicle to emit UVC light into a passenger compartment of the autonomous vehicle. The autonomous vehicle can include sensors to confirm that the passenger compartment is empty and can be configured to prevent ingress of passengers into the passenger compartment while the UVC light is being emitted. Moreover, visible light indicators in the vehicle can be controlled to provide different visual indicators during the disinfecting cycle and when the vehicle is available to a passenger. In examples, the disinfecting cycle can be performed while the autonomous vehicle is in use, e.g., while the autonomous vehicle is traversing to a location to retrieve a passenger. The methods and systems can provide improved cleaning and disinfection of autonomous vehicles without the need to take the vehicles out of operation.

Abstract: Techniques for determining an error model based on vehicle data and ground truth data are discussed herein. To determine whether a complex system (which may be not capable of being inspected) is able to operate safely, various operating regimes (scenarios) can be identified based on operating data. To provide safe operation of such a system, an error model can be determined that can provide a probability associated with perception data and a vehicle can determine a trajectory based on the probability of an error associated with the perception data.

Abstract: An object model for a real-world object is constructed using a parameter-based object generator, where the parameters for the object model are determined using sensor data such as photographic images, radar images, or LIDAR images obtained from the real-world object. Conversion of the sensor data to appropriate object parameters is accomplished using a machine learning system. The machine learning system is calibrated by generating a plurality of object parameters, generating a corresponding plurality of learning objects from the plurality of object parameters, generating a plurality of simulated sensor return data for the learning objects, and then providing the simulated sensor return data and object parameters to the machine learning system.

Abstract: A vehicle may include a primary system and a secondary system to validate operation of the primary system and to control the vehicle to avoid collisions. For example, the secondary system may receive multiple trajectories from the primary system, such as a primary trajectory and a secondary, contingent, trajectory associated with a deceleration or other maneuver. The secondary system may determine if a trajectory is associated with a potential collision, if the trajectory is consistent with a current or previous pose, if the trajectory is compatible with a capability of the vehicle, etc. The secondary system may select the primary trajectory if valid, the secondary trajectory if the primary trajectory is invalid, or another trajectory generated by the secondary system if the primary trajectory and the secondary trajectory are invalid. If no valid trajectory is determined, the vehicle may decelerate at a maximum rate.

Abstract: A vehicle may include an authority tracker to grant and/or limit authority to provide guidance to the vehicle. The authority tracker may store a state in a memory identifying an actor that currently has the authority. An actor requesting to have authority to provide guidance to the vehicle may transmit a control message to the authority tracker which may, in turn, authorize or deny the request based on a policy. The authority tracker may periodically, and/or in response to a request, broadcast a state message identifying the actor that currently has authority, as recorded in the memory.

Abstract: A collision avoidance system may validate, reject, or replace a trajectory generated to control a vehicle. The collision avoidance system may comprise a secondary perception component that may comprise one or more machine learned models, each of which may be trained to output one or more occupancy maps based at least in part on sensor data of different types. The occupancy maps may include a prediction of whether at least a portion of an environment is occupied at a future time by any one of multiple object types. Occupancy maps associated with a same time may be aggregated into a data structure that may be used to validate, reject, or replace the trajectory.

Abstract: Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can return pixels subject to over-exposure or saturation, which may be from stray light. In some examples, multiple exposures captured at different exposure times can be used to determine a saturation value for sensor data. The saturation value may be used to determine a threshold intensity against which intensity values of a primary exposure are compared. A filtered data set can be obtained based on the comparison.

Abstract: Systems, methods, and apparatuses described herein are directed to performing segmentation on voxels representing three-dimensional data to identify static and dynamic objects. LIDAR data may be captured by a perception system for an autonomous vehicle and represented in a voxel space. Operations may include determining a drivable surface by parsing individual voxels to determine an orientation of a surface normal of a planar approximation of the voxelized data relative to a reference direction. Clustering techniques can be used to grow a ground plane including a plurality of locally flat voxels. Ground plane data can be set aside from the voxel space, and the remaining voxels can be clustered to determine objects. Voxel data can be analyzed over time to determine dynamic objects. Segmentation information associated with ground voxels, static object, and dynamic objects can be provided to a tracker and/or planner in conjunction with operating the autonomous vehicle.