Abstract: Techniques are discussed for evaluating trajectories based on risk associated with the trajectories with respect to predicted locations of objects in an environment. A vehicle can capture sensor data of an environment, which may include object(s) separate from the vehicle, such as another vehicle or a pedestrian. A prediction system can output a discretized probability distribution comprising prediction probabilities associated with possible locations of the object in the future. Heat maps, as an example discretized probability distribution, can represent one or more objects. Trajectories can be generated for the vehicle to follow in the environment. An overlap between a region of the vehicle along a trajectory and the heat map can be determined, and a probability associated with the overlap can represent a risk associated with a trajectory navigating through the environment. The vehicle can be controlled based on risks associated with the various trajectories.

Abstract: A three-dimensional bounding box is determined from a two-dimensional image and a point cloud. A feature vector associated with the image and a feature vector associated with the point cloud may be passed through a neural network to determine parameters of the three-dimensional bounding box. Feature vectors associated with each of the points in the point cloud may also be determined and considered to produce estimates of the three-dimensional bounding box on a per-point basis.

Abstract: Techniques and methods for contacting a teleoperator. For instance, while navigating to a location, progress of an autonomous vehicle may stop. This may be caused by the autonomous vehicle yielding to another vehicle, such as at an intersection or when the other vehicle is located along a route of the autonomous vehicle. While yielding to the other vehicle, the autonomous vehicle may determine that the progress has stopped for a threshold amount of time. Additionally, in some circumstances, the autonomous vehicle may determine that the progress not been stopped due to traffic or a traffic light. Based at least in part on the determinations, the autonomous vehicle may send sensor data to the teleoperator, where the sensor data represents an environment for which the autonomous vehicle is located. Additionally, the autonomous vehicle may send an indication of the other vehicle to the teleoperator.

Abstract: Techniques for generating trajectories and drivable areas for navigating a vehicle in an environment are discussed herein. The techniques can include receiving a reference trajectory representing an initial trajectory for a vehicle, such as an autonomous vehicle, to traverse the environment. Portions of the reference trajectory can be identified as corresponding to actions to navigate around a double-parked vehicle or to change lanes, for example. In some cases, a portion of the reference trajectory can be identified based on a proximity to an object in the environment. A weight can be associated with the portions of the reference trajectory, and the techniques can include evaluating a reference cost function at points of the reference trajectory based on the associated weights to generate a target trajectory. Further, the techniques can include controlling the autonomous vehicle to traverse the environment based at least in part on the target trajectory.

Abstract: Techniques are disclosed for tracking objects in sensor data, such as multiple images or multiple LIDAR clouds. The techniques may include comparing segmentations of sensor data such as by, for example, determining a similarity of a first segmentation of first sensor data and a second segmentation of second sensor data. Comparing the similarity may comprise determining a first embedding associated with the first segmentation and a second embedding associated with the second segmentation and determining a distance between the first embedding and the second embedding. The techniques may improve the accuracy and/or safety of systems integrating the techniques discussed herein.

Abstract: A vehicle can include various sensors to detect objects in an environment. Sensor data can be captured by a perception system in a vehicle and represented in a voxel space. Operations may include analyzing the data from a top-down perspective. From this perspective, techniques can associate and generate masks that represent objects in the voxel space. Through manipulation of the regions of the masks, the sensor data and/or voxels associated with the masks can be clustered or otherwise grouped to segment data associated with the objects.

Abstract: Techniques are discussed for determining prediction probabilities of an object based on a top-down representation of an environment. Data representing objects in an environment can be captured. Aspects of the environment can be represented as map data. A multi-channel image representing a top-down view of object(s) in the environment can be generated based on the data representing the objects and map data. The multi-channel image can be used to train a machine learned model by minimizing an error between predictions from the machine learned model and a captured trajectory associated with the object. Once trained, the machine learned model can be used to generate prediction probabilities of objects in an environment, and the vehicle can be controlled based on such prediction probabilities.

Abstract: Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can include noise. In some examples, the sensor data is filtered by comparing a measured intensity value to a threshold intensity value. The threshold intensity value can be determined on a per-pixel basis using depth information and/or intensity information for a corresponding image frame.

Abstract: Techniques are discussed for controlling a vehicle, such as an autonomous vehicle, based on predicted occluded areas in an environment. An occluded area can represent areas where sensors of the vehicle are unable to sense portions of the environment due to obstruction by another object. A first occluded region for an object is determined at a first time based on a location of the object. A predicted location for the object can be used to determine a predicted occluded region caused by the object at a second time after the first time. Predicted occluded regions can be determined for multiple trajectories for the vehicle to follow and/or at multiple points along such trajectories, and a trajectory can be selected based on associated occlusion scores and/or trajectory scores associated therewith. The vehicle can be controlled to traverse the environment based on the selected trajectory.

Abstract: Techniques for decimating portions of a map of an environment are discussed herein. The environment can be represented by a three-dimensional (3D) map including a plurality of polygons and semantic information associated with the polygons. In some cases, decimation operations may be based on semantic information associated with the environment. Differing decimation operations and/or levels may be applied to polygons of different semantic classifications or differing contribution levels. Boundaries between regions having different semantic information can be preserved. Meshes can be decimated using different decimation operators or decimation levels and an accuracy of localizing can be compared using the various decimated meshes. An optimal mesh can be selected and sent to vehicles for localizing the vehicles in the environment.

Abstract: The described techniques relate to coordinating and managing faults of systems of a vehicle, such as an autonomous vehicle, to enable the vehicle to respond safely and appropriately to the faults. In examples, a centralized fault monitor system receives faults from different vehicle systems, maps the received faults to associated fault constraints, prioritizes different and shared parameters between the fault constraints, and communicates the constraint parameters to appropriate vehicle systems accordingly.

Abstract: Autonomous vehicles use accurate and detailed maps for navigation. Expanding functionality of an autonomous vehicle to a new, e.g., unmapped, region can include determining drivable surface segments of the new region and comparing the segments to segments or classes of segments from an already-mapped region. Segments of the new region that are similar to segments from the mapped region can be identified as potentially navigable. An autonomous vehicle can travel through the new region via those segments indicated as navigable. In addition, during travel through the new region, data may be collected using sensors on the autonomous vehicle to map additional portions of the region and/or confirm a driving ability in the new region. Functionality of an autonomous vehicle may be limited based on how similar the segments are to one another.

Abstract: A method and system of using excess computational resources on autonomous vehicles. Such excess computational resources may be available during periods of low demand, or other periods of idleness (e.g., parking). Where portions of computing resources are available amongst a fleet of vehicles, such excess computing resources may be pooled as a single resource. The excess computational resources may be used, for example, to train and/or test machine-learning models. Performance metrics of such models may be determined using hardware and software on the autonomous vehicle, for example sensors. Models having performance metrics outperforming current models may be considered as validated models. Validated models may be transmitted to a remote computing system for dissemination to a fleet of vehicles.

Abstract: Perception sensors of a vehicle can be used for various operating functions of the vehicle. A computing device may receive sensor data from the perception sensors, and may calibrate the perception sensors using the sensor data, to enable effective operation of the vehicle. To calibrate the sensors, the computing device may project the sensor data into a voxel space, and determine a voxel score including an occupancy score and a residual value for each voxel. The computing device may then adjust an estimated position and/or orientation of the sensors, and associated sensor data, from at least one perception sensor to minimize the voxel score. The computing device may calibrate the sensor using the adjustments corresponding to the minimized voxel score. Additionally, the computing device may be configured to calculate an error in a position associated with the vehicle by calibrating data corresponding to a same point captured at different times.

Abstract: Junction queueing for vehicles is described, in which vehicles may be queued based on arrival time at a junction, position relative to a stopping location at the junction, and/or an amount of time waiting for other vehicles to proceed through the junction (timeout). In some examples, the queue may be first generated based on the arrival times of any other vehicle relative to a particular vehicle generating the queue. The queue may be updated based on arrival times of other vehicles (e.g., after the particular vehicle), whether another vehicle has proceeded out-of-turn (e.g., based on a position at the junction), and/or a timeout for vehicles that wait for others to yield at the junction. In some examples, hysteresis and alterations of the score for safety reasons may alter queue order. The queue may be used to control the particular vehicle to traverse the junction.

Abstract: A lighting system of a vehicle may include light units and controllers for operating the light units. Each light unit is multifunctional in that the light unit may operate in various modes, which may be invoked at different times under varying circumstances. A light unit on the first end of a vehicle may operate as a headlight if the first end is the front end of the vehicle. On the other hand, the light unit may operate as a tail light if the first end is the rear end of the vehicle. Furthermore, the light unit may operate as a turn signal in either direction or brake light while the first end is the rear end of the vehicle. The light unit includes a lens array positioned to receive light from various light sources. The lighting system may operate in a fashion that allows for lighting redundancy on each end of the vehicle in the event of a lighting controller failure.

Abstract: A system for detecting and/or mitigating the effects of an obstruction on a surface of a sensor may include a surface configured to receive the sensor, and comprising a sensor window configured to provide a path through which the sensor senses the environment. The system may also include a vibratory actuator configured to facilitate vibration of the sensor window and a heating element configured to heat the sensor window. The system may further include an obstruction detection system configured to detect an obstruction, such as moisture, and an obstruction mitigation controller configured to initiate activation of the vibratory actuator and/or the heating element to mitigate the obstruction. The obstruction detection system may be configured to receive a signal from one or more sensors and detect the obstruction based at least in part on the signal.

Abstract: Techniques are discussed for determining predicted trajectories based on a top-down representation of an environment. Sensors of a first vehicle can capture sensor data of an environment, which may include agent(s) separate from the first vehicle, such as a second vehicle or a pedestrian. A multi-channel image representing a top-down view of the agent(s) and the environment and comprising semantic information can be generated based on the sensor data. Semantic information may include a bounding box and velocity information associated with the agent, map data, and other semantic information. Multiple images can be generated representing the environment over time. The image(s) can be input into a prediction system configured to output a heat map comprising prediction probabilities associated with possible locations of the agent in the future. A predicted trajectory can be generated based on the prediction probabilities and output to control an operation of the first vehicle.

Abstract: An impact energy absorbing system for a vehicle includes a first impact energy absorbing structure to absorb energy from an impact with an external object, and a second energy absorbing structure to absorb energy from an impact with an internal displaceable object internal. The second energy absorbing structure may be disposed between the displaceable object and a protected object internal to the vehicle. The second impact energy absorbing structure may be disposed within the vehicle and configured to protect the protected object from impact by the displaceable object being displaced toward the protected object during a collision. Additionally, the second impact energy absorbing structure may cause the displaceable object to avoid impacting and/or damaging the protected object.