Abstract: Methods and systems are provided for detecting objects by utilizing uncertainties. In some aspects, a process can include steps for receiving, by an autonomous vehicle system, a frame of a scene with a detected object; estimating, by the autonomous vehicle system, an overall probability of the detected object in the frame; estimating, by the autonomous vehicle system, covariances for each state of at least one bounding box; and balancing, by the autonomous vehicle system, confidence values of the at least one bounding box based on the overall probability of the detected object and the covariances of each state of the at least one bounding box.

Abstract: The disclosed technology provides solutions for applying a flare detection model to an image for detecting road flares in an environment by an autonomous vehicle (AV). A method comprises accessing a flare detection model that is trained to recognize a bounded outer region and a bounded inner region; and detect a flare object within the bounded inner region and refrain from detecting an object in the bounded outer region; applying the flare detection model to an image of the environment to recognize an outer region corresponding to a glow of a flare and an inner region that corresponds to the flare in the environment; and detecting the flare in the environment within the inner region based on the inner region being recognized by the flare detection model and contained within the outer region that is also recognized by the flare detection model. Systems and machine-readable media are also provided.

Abstract: Systems and techniques are provided for processing sensor data using a machine learning model. An example method can include receiving, by a machine learning model configured to perform object detection, sensor data corresponding to one or more sensors on an autonomous vehicle; determining, based on the sensor data, a first object prediction by a first detection head of the machine learning model, wherein the first detection head is configured to generate the first object prediction based on a first input from a first sensor from the one or more sensors; and determining, based on the sensor data, a second object prediction by a second detection head of the machine learning model, wherein the second detection head is configured to generate the second object prediction based on a fusion of the first input from the first sensor and a second input from a second sensor from the one or more sensors.

Abstract: Various technologies described herein pertain to detecting data in a lidar point cloud representative of vapor and controlling an autonomous vehicle based on such detection. The lidar point cloud is outputted by a lidar sensor system of the autonomous vehicle. The techniques set forth herein utilize dual return lidar data outputted by the lidar sensor system. The dual return lidar data includes two lidar returns received responsive to a light beam emitted (e.g., at a particular azimuthal angle) by the lidar sensor system into a driving environment of the autonomous vehicle. The data in the lidar point cloud detected as being caused by vapor can be removed from downstream processing; accordingly, the autonomous vehicle can be controlled such that the autonomous vehicle need not stop for or maneuver around vapor in the driving environment.

Abstract: Methods and systems are provided for detecting objects by utilizing uncertainties. In some aspects, a process can include steps for receiving, by an autonomous vehicle system, a frame of a scene with a detected object; estimating, by the autonomous vehicle system, an overall probability of the detected object in the frame; estimating, by the autonomous vehicle system, covariances for each state of at least one bounding box; and balancing, by the autonomous vehicle system, confidence values of the at least one bounding box based on the overall probability of the detected object and the covariances of each state of the at least one bounding box.

Abstract: The disclosed technology provides solutions for object detection and classification with both high recall and high precision, by using a first stage with high recall, and a second stage to provide high precision. The dimensional state of a pointcloud is reduced from 3 to 2, and proposed bounding boxes are generated. The original pointcloud data is filtered according to the bounding boxes, and fused with learned features, with the fused data processed to generate the high recall and high precision output.

Abstract: An AV is described herein. The AV includes a lidar sensor system. The AV additionally includes a computing system that executes a road surface analysis component to determine, based upon lidar sensor data, whether a road surface feature is present on or in a roadway in a travel path of the AV. The AV can be configured to initiate a mitigation maneuver responsive to determining that the road surface feature is present. Performing the mitigation maneuver causes the AV to avoid the road surface feature or decelerate prior to reaching the road surface feature, thereby improving the apparent quality or comfort of the ride to a passenger of the AV.

Abstract: The present disclosure provides a perception system for a vehicle. The perception system includes a perception filter for determining a region of interest (“ROI”) for the vehicle based on an intent of the vehicle and a current state of the vehicle; and a perception module for perceiving an environment of the vehicle based on the ROI; wherein the vehicle is caused to take appropriate action based on the perceived environment, the current state of the vehicle, and the intent of the vehicle.

Abstract: An AV is described herein. The AV includes a lidar sensor system. The AV additionally includes a computing system that executes a road surface analysis component to determine, based upon lidar sensor data, whether a road surface feature is present on or in a roadway in a travel path of the AV. The AV can be configured to initiate a mitigation maneuver responsive to determining that the road surface feature is present. Performing the mitigation maneuver causes the AV to avoid the road surface feature or decelerate prior to reaching the road surface feature, thereby improving the apparent quality or comfort of the ride to a passenger of the AV.

Abstract: Various technologies described herein pertain to detecting sensor data to be annotated for autonomous vehicle perception algorithm training. A label identifying a type of an object is assigned to an object at a particular location in an environment based on sensor data generated by a sensor system of an autonomous vehicle for a given time. The label is assigned based on a confidence score assigned to the type of the object by a computer-implemented perception algorithm. The computer-implemented perception algorithm assigns the confidence score to the type of the object based on the sensor data corresponding to the particular location in the environment for the given time. An output of a heuristic is generated based on the label and/or the confidence score, and the output of the heuristic is used to control whether to cause the sensor data for the given time to be annotated.

Abstract: Various technologies described herein pertain to detecting sensor data to be annotated for autonomous vehicle perception algorithm training. A label identifying a type of an object is assigned to an object at a particular location in an environment based on sensor data generated by a sensor system of an autonomous vehicle for a given time. The label is assigned based on a confidence score assigned to the type of the object by a computer-implemented perception algorithm. The computer-implemented perception algorithm assigns the confidence score to the type of the object based on the sensor data corresponding to the particular location in the environment for the given time. An output of a heuristic is generated based on the label and/or the confidence score, and the output of the heuristic is used to control whether to cause the sensor data for the given time to be annotated.

Abstract: Various technologies described herein pertain to detecting sensor data to be annotated for autonomous vehicle perception algorithm training. A label identifying a type of an object is assigned to an object at a particular location in an environment based on sensor data generated by a sensor system of an autonomous vehicle for a given time. The label is assigned based on a confidence score assigned to the type of the object by a computer-implemented perception algorithm. The computer-implemented perception algorithm assigns the confidence score to the type of the object based on the sensor data corresponding to the particular location in the environment for the given time. An output of a heuristic is generated based on the label and/or the confidence score, and the output of the heuristic is used to control whether to cause the sensor data for the given time to be annotated.

Abstract: Various technologies described herein pertain to detecting data in a lidar point cloud representative of vapor and controlling an autonomous vehicle based on such detection. The lidar point cloud is outputted by a lidar sensor system of the autonomous vehicle. The techniques set forth herein utilize dual return lidar data outputted by the lidar sensor system. The dual return lidar data includes two lidar returns received responsive to a light beam emitted (e.g., at a particular azimuthal angle) by the lidar sensor system into a driving environment of the autonomous vehicle. The data in the lidar point cloud detected as being caused by vapor can be removed from downstream processing; accordingly, the autonomous vehicle can be controlled such that the autonomous vehicle need not stop for or maneuver around vapor in the driving environment.

Abstract: The present disclosure provides a perception system for a vehicle. The perception system includes a perception filter for determining a region of interest (“ROI”) for the vehicle based on an intent of the vehicle and a current state of the vehicle; and a perception module for perceiving an environment of the vehicle based on the ROI; wherein the vehicle is caused to take appropriate action based on the perceived environment, the current state of the vehicle, and the intent of the vehicle.

Abstract: The present disclosure provides perception system for a vehicle that includes a plurality of imaging devices for producing images of an environment of the vehicle; a perception filter for receiving the images produced by the imaging devices, wherein the perception filter determines compute resource priority instructions based on an intent of the vehicle and a current state of the vehicle; and a compute module for receiving the compute resource priority instructions from the perception filter and allocating compute resources among the imaging devices in accordance with the compute resource priority instructions.

Abstract: The present disclosure provides a perception system for a vehicle. The perception system includes a number of imaging devices associated with the vehicle and a perception filter for determining a region of interest (“ROI”) for the vehicle based on an intent of the vehicle and a current state of the vehicle. The ROI is used to select images of an environment of the vehicle produced by the imaging devices and the perception filter receives and processes the images produced by the imaging device. The perception system further includes a perception module for receiving the processed images from the perception filter and perceiving the environment of the vehicle based at least in part on the received images.

Abstract: The present disclosure provides perception system for a vehicle that includes a plurality of imaging devices for producing images of an environment of the vehicle; a perception filter for receiving the images produced by the imaging devices, wherein the perception filter crops and filters the received images based on an intent of the vehicle and a current state of the vehicle; and a perception module for receiving at least one of the cropped and filtered images from the perception filter and perceiving the environment of the vehicle based on the received at least one of the cropped and filtered images.

Abstract: An AV is described herein. The AV includes a lidar sensor system. The AV additionally includes a computing system that executes a road surface analysis component to determine, based upon lidar sensor data, whether a road surface feature is present on or in a roadway in a travel path of the AV. The AV can be configured to initiate a mitigation maneuver responsive to determining that the road surface feature is present. Performing the mitigation maneuver causes the AV to avoid the road surface feature or decelerate prior to reaching the road surface feature, thereby improving the apparent quality or comfort of the ride to a passenger of the AV.

Abstract: The present disclosure provides perception system for a vehicle that includes a plurality of imaging devices for producing images of an environment of the vehicle; a perception filter for receiving the images produced by the imaging devices, wherein the perception filter determines compute resource priority instructions based on an intent of the vehicle and a current state of the vehicle; and a compute module for receiving the compute resource priority instructions from the perception filter and allocating compute resources among the imaging devices in accordance with the compute resource priority instructions.

Abstract: The present disclosure provides a perception system for a vehicle. The perception system includes a number of imaging devices associated with the vehicle and a perception filter for determining a region of interest (“ROI”) for the vehicle based on an intent of the vehicle and a current state of the vehicle. The ROI is used to select images of an environment of the vehicle produced by the imaging devices and the perception filter receives and processes the images produced by the imaging device. The perception system further includes a perception module for receiving the processed images from the perception filter and perceiving the environment of the vehicle based at least in part on the received images.