Abstract: A road sign interpretation system includes a front-facing camera mounted on or in a vehicle collecting image data of multiple road signs. A first convolutional neural network (CNN) receives the image data from the front-facing camera and yields a set of sign predictions including one or more sign text instances. A second CNN defining a text extractor receives the image data from the front-facing camera and extracts text candidates including the multiple sign text instances. Sign and sign data localization is provided in the second CNN to compute a text order from the multiple sign text instances. A sign text synthesizer module receives individual sign text instances from the first CNN and individual ones of the sign text instances in digitized forms from an optical character recognizer (OCR). A semantic encoding and interpretation module receives the sign text instances and identifies semantics of the multiple road signs.

Abstract: According to several aspects, a method for path planning for a vehicle includes determining a predicted trajectory of a remote vehicle. The predicted trajectory of the remote vehicle includes a plurality of predicted trajectory nodes. The method further includes determining a plurality of possible trajectories for the vehicle. The plurality of possible trajectories includes a plurality of possible trajectory nodes. The method further includes determining one or more evaluation metrics of each of the plurality of possible trajectory nodes based at least in part on a weather condition in an environment surrounding the vehicle. The method further includes selecting an optimal trajectory for the vehicle from the plurality of possible trajectories based at least in part on the one or more evaluation metrics of each of the plurality of possible trajectory nodes. The method further includes performing a first action based at least in part on the optimal trajectory.

Abstract: A system for updating a road map for a vehicle includes a plurality of vehicle sensors and a vehicle controller in electrical communication with the plurality of vehicle sensors. The vehicle controller is programmed to gather input data about an environment surrounding the vehicle using the plurality of vehicle sensors. The input data includes at least one abnormal traffic pattern indication. The vehicle controller is further programmed to generate an input label map based at least in part on the input data. The vehicle controller is further programmed to generate a vehicle output label map based at least in part on the input label map. The vehicle output label map is generated using a machine learning algorithm. The vehicle controller is further programmed to perform a first action based at least in part on the vehicle output label map.

Abstract: A hybrid probabilistic driving behavior modeling system conditioned on weather for a vehicle includes one or more controllers executing instructions to determine, by a longitudinal driving model stored by the one or more controllers, a probabilistic longitudinal velocity of the vehicle with respect to a current weather condition based on a car-following model, a semantic rule system, and a speed and visibility model. The current weather condition indicates an adverse weather condition impacting driving conditions for the vehicle. The one or more controllers determine, by a probabilistic lateral driving model by the one or more controllers, one or more lane choices for the vehicle with respect to the current weather condition based on a route plan of the vehicle and perception data indicative of an environment surrounding the vehicle and selects a final lane choice from the one or more lane choices.

Abstract: An end-to-end perception perturbation modeling system for a vehicle includes one or more controllers storing a detection model in memory. The detection model includes a plurality of detection model plots that each indicate a probability value that an object in an environment surrounding the vehicle is detected based on a current weather condition and a distance measured between the vehicle and the object detected in the environment surrounding the vehicle. The one or more controllers execute instructions to receive an input state of the vehicle that is observed during non-inclement weather conditions and indicates one or more vehicle states, the current weather condition, and the distance. The controllers determine a perturbed state of the vehicle observed during the current weather condition.

Abstract: Described is a system for adapting to perception errors in object detection and recognition. The system receives, with a perception module, perception data from an environment proximate a mobile platform that reflects objects in the environment. Perception probes representing perception characteristics of object detections are generated from the perception data. Using the perception probes, spatial logic-based constraints and temporal logic-based constraints are generated. Spatial perception parameters are determined by solving an optimization problem using a set of the spatial logic-based constraints. Temporal perception parameters are determined by solving an optimization problem using a set of temporal logic-based constraints. The spatial perception parameters and the temporal perception parameters are combined to estimate a final perception parameter. The perception module is adjusted based on the final perception parameter.

Abstract: A method for performing object detection during autonomous driving includes: performing 3D object detection in a 3D object detection segment; uploading an output of multiple sensors in communication with the 3D object detection segment to multiple point clouds; transferring point cloud data from the multiple point clouds to a Region Proposal Network (RPN); independently performing 2D object detection in a 2D object detector in parallel with the 3D object detection in the 3D object detection segment; and taking a given input image and simultaneously learning box coordinates and class label probabilities in a 2D object detection network operating to treat object detection as a regression problem.

Abstract: A method and apparatus for verifying object classification includes comparing detected object-components with trained component-based descriptors and similarity measures to generate object classification verification data. The object classification verification data represents object classification confidence or misclassification errors to weight uncertainty for perception-based decision making. A sequence of similarity measures associated with a current frame and prior frames within a reference observation time constraint are compared to a reference temporal similarity measure boundary to generate the object classification verification data.

Abstract: Described is a system for action recognition error detection and correction using probabilistic signal temporal logic. The system is initiated by training an action recognition system to generate true positive (TP)/false positive (FP) axioms. Thereafter, the system ca be used to classify one or more actions in a video sequence as true action classifications by using the TP/FP axioms to remove false action classifications. With the remaining true classifications, a device can be controlled given the situation and relevant true classification.

Abstract: A radar anti-spoofing system for an autonomous vehicle includes a plurality of radar sensors that generate a plurality of input detection points representing radio frequency (RF) signals reflected from objects and a controller in electronic communication with the plurality of radar sensors. The one or more controllers execute instructions to determine a signal to noise ratio (SNR) distance ratio for the input detection points generated by the plurality of radar sensors, where a value of the SNR distance ratio is indicative of an object being a ghost vehicle. The one or more controllers also determine an effective particle number indicating a degree of particle degradation for the importance sampling for each variable that is part of the state variable. In response to determining the effective particle number is equal to or less than a predetermined threshold, the one or more controllers estimate a ghost position for the ghost vehicle.

Abstract: A road sign interpretation system includes a front-facing camera mounted on or in a vehicle collecting image data of multiple road signs. A first convolutional neural network (CNN) receives the image data from the front-facing camera and yields a set of sign predictions including one or more sign text instances. A second CNN defining a text extractor receives the image data from the front-facing camera and extracts text candidates including the multiple sign text instances. Sign and sign data localization is provided in the second CNN to compute a text order from the multiple sign text instances. A sign text synthesizer module receives individual sign text instances from the first CNN and individual ones of the sign text instances in digitized forms from an optical character recognizer (OCR). A semantic encoding and interpretation module receives the sign text instances and identifies semantics of the multiple road signs.

Abstract: A method of using perception-inspired event generation for situation awareness for a vehicle, including receiving perception input data from a sensor of the vehicle and processing the perception input data to classify and generate parameters related to an external entity in a vicinity of the vehicle. The method includes generating a hierarchical event structure that classifies and prioritizes the perception input data by classifying the external entity into an attention zone and prioritizing the external entity within the attention zone according to a risk level value for the external entity. A higher risk level value indicates a higher priority within the attention zone. The method further includes developing a behavior plan for the vehicle based on the hierarchical event structure.

Abstract: Described is a system for evaluating and correcting perception errors in object detection and recognition. The system receives perception data from an environment proximate a mobile platform. Perception probes are generated from the perception data which describe perception characteristics of object detections in the perception data. For each perception probe, probabilistic distributions for true positive and false positive values are determined, resulting in true positive and false negative perception probes. Statistical characteristics of true positive perception probes and false positive perception probes are then determined. Based on the statistical characteristics, true positive perception probes are clustered. An axiom is generated to determine statistical constraints for perception validity for each perception probe cluster. The axiom is evaluated to classify the perception probes as valid or erroneous. Optimal perception parameters are generated by solving an optimization problem based on the axiom.

Abstract: A method for updating a localization of an autonomous vehicle includes receiving perception input data from a sensor of the autonomous vehicle and receiving map data including road lane information in a vicinity of the autonomous vehicle. The method includes processing the perception input data to extract perceived road lane information including a perceived x position, a perceived y position, a perceived z position, a perceived lane type, a perceived lane color, and a perceived lane curvature and processing the map data to extract map road lane information including a map x position, a map y position, a map z position, a map lane type, a map lane color, and a map lane curvature. The method includes calculating a transformation matrix from the perceived road lane information and the map road lane information and updating the map data and a localization of the vehicle based on the transformation matrix.

Abstract: A method using unsupervised velocity prediction and correction for urban driving from sequences of noisy position estimates includes: performing a vehicle velocity prediction for one or more other vehicles in a vicinity of a host automobile vehicle; calculating a first heuristic based on a uniformity test; calculating a second heuristic based on a vehicle speed of the one or more other vehicles; combining the first heuristic and the second heuristic using a weighted sum; determining an uncertainty mask applying the combined first heuristic and the second heuristic and a heuristic threshold; and applying the uncertainty mask to identify a velocity correction for use by the host automobile vehicle.

Abstract: Described is a system for detecting and correcting perception errors in a perception system. In operation, the system generates a list of detected objects from perception data of a scene, which allows for the generation of a list of background classes from backgrounds in the perception data associated with the list of detected objects. For each detected object in the list of detected objects, a closest background class is identified from the list of background classes. Vectors can then be used to determine a semantic feature, which is used to identify axioms. An optimal perception parameter is then generated, which is used to adjust perception parameters in the perception system to minimize perception errors.

Abstract: A radar anti-spoofing system for an autonomous vehicle includes a plurality of radar sensors that generate a plurality of input detection points representing radio frequency (RF) signals reflected from objects and a controller in electronic communication with the plurality of radar sensors. The one or more controllers execute instructions to determine a signal to noise ratio (SNR) distance ratio for the input detection points generated by the plurality of radar sensors, where a value of the SNR distance ratio is indicative of an object being a ghost vehicle. The one or more controllers also determine an effective particle number indicating a degree of particle degradation for the importance sampling for each variable that is part of the state variable. In response to determining the effective particle number is equal to or less than a predetermined threshold, the one or more controllers estimate a ghost position for the ghost vehicle.

Abstract: A method for performing object detection during autonomous driving includes: performing 3D object detection in a 3D object detection segment; uploading an output of multiple sensors in communication with the 3D object detection segment to multiple point clouds; transferring point cloud data from the multiple point clouds to a Region Proposal Network (RPN); independently performing 2D object detection in a 2D object detector in parallel with the 3D object detection in the 3D object detection segment; and taking a given input image and simultaneously learning box coordinates and class label probabilities in a 2D object detection network operating to treat object detection as a regression problem.

Abstract: A method of path planning for a host vehicle includes: receiving host vehicle, environmental and obstacle information; calculating one or more projected host vehicle locations; computing a projected obstacle location for each obstacle; and determining a collision potential between each projected host vehicle location and each projected obstacle location. Until a maximum number of steps is reached, and while at least one projected host vehicle location has an associated collision potential below a collision threshold, the method further includes repeating the calculating, computing and determining steps.

Abstract: A method using unsupervised velocity prediction and correction for urban driving from sequences of noisy position estimates includes: performing a vehicle velocity prediction for one or more other vehicles in a vicinity of a host automobile vehicle; calculating a first heuristic based on a uniformity test; calculating a second heuristic based on a vehicle speed of the one or more other vehicles; combining the first heuristic and the second heuristic using a weighted sum; determining an uncertainty mask applying the combined first heuristic and the second heuristic and a heuristic threshold; and applying the uncertainty mask to identify a velocity correction for use by the host automobile vehicle.