Abstract: Described is a system for adapting to perception errors in object detection and recognition. The system receives, with a perception module, perception data from an environment proximate a mobile platform that reflects objects in the environment. Perception probes representing perception characteristics of object detections are generated from the perception data. Using the perception probes, spatial logic-based constraints and temporal logic-based constraints are generated. Spatial perception parameters are determined by solving an optimization problem using a set of the spatial logic-based constraints. Temporal perception parameters are determined by solving an optimization problem using a set of temporal logic-based constraints. The spatial perception parameters and the temporal perception parameters are combined to estimate a final perception parameter. The perception module is adjusted based on the final perception parameter.

Abstract: Described is a system for action recognition error detection and correction using probabilistic signal temporal logic. The system is initiated by training an action recognition system to generate true positive (TP)/false positive (FP) axioms. Thereafter, the system ca be used to classify one or more actions in a video sequence as true action classifications by using the TP/FP axioms to remove false action classifications. With the remaining true classifications, a device can be controlled given the situation and relevant true classification.

Abstract: Described is a system for evaluating and correcting perception errors in object detection and recognition. The system receives perception data from an environment proximate a mobile platform. Perception probes are generated from the perception data which describe perception characteristics of object detections in the perception data. For each perception probe, probabilistic distributions for true positive and false positive values are determined, resulting in true positive and false negative perception probes. Statistical characteristics of true positive perception probes and false positive perception probes are then determined. Based on the statistical characteristics, true positive perception probes are clustered. An axiom is generated to determine statistical constraints for perception validity for each perception probe cluster. The axiom is evaluated to classify the perception probes as valid or erroneous. Optimal perception parameters are generated by solving an optimization problem based on the axiom.

Abstract: Described is a system for detecting and correcting perception errors in a perception system. In operation, the system generates a list of detected objects from perception data of a scene, which allows for the generation of a list of background classes from backgrounds in the perception data associated with the list of detected objects. For each detected object in the list of detected objects, a closest background class is identified from the list of background classes. Vectors can then be used to determine a semantic feature, which is used to identify axioms. An optimal perception parameter is then generated, which is used to adjust perception parameters in the perception system to minimize perception errors.

Abstract: Embodiments include methods, systems, and computer readable storage medium for a method for providing path-planning guidance by resolving multiple behavioral predictions associated with operating a vehicle is disclosed. The method includes installing a vehicle system into a vehicle, wherein the vehicle system provides path planning guidance based on training data using and fused hypotheses and/or decisions associated with the training data. The method further includes determining, by a processor, a location of the vehicle on a map containing a road network, and determining, by the processor, whether one or more agents exist within a predetermined range of the vehicle. The method further includes selecting, by the processor, an output trajectory to traverse the road network based on the location of the vehicle on the map and the existence of one or more agents. The method further includes controlling, by the processor, operation of the vehicle using the output trajectory.

Abstract: A method, autonomous vehicle and system for operating an autonomous vehicle. A sensor obtains data of an agent. A processor determines a measure of complexity of the environment in which the autonomous vehicle is operating from the sensor data, selects a control scheme for operating the autonomous vehicle based on the determined complexity, and operates the autonomous vehicle using the selected control scheme.

Abstract: An autonomous vehicle and a system and method of operating the autonomous vehicle. A maneuver classifier is trained at an offline processor to identify a driving maneuver for a driving context. An online processor is configured to receive the driving context, operate the maneuver classifier to identify the driving maneuver based on the driving context, perform the driving maneuver at the autonomous vehicle, grade the driving maneuver as it is being performed at the autonomous vehicle, and adjust a performance of the driving maneuver at the autonomous vehicle based on the grade.

Abstract: Described is a system for contrast and entropy-based perception adaption to optimize perception. The system is operable for receiving an input image of a scene with a camera system and detecting one or more objects (having perception data) in the input image. The perception data of the one or more objects is converted into probes, which are then converted into axioms using probabilistic signal temporal logic. The axioms are evaluated based on probe bounds. If the axioms are within the probe bounds, then results are provided; however, if the axioms are outside of the probe bounds, the system estimates optimal contrast bounds and entropy bounds as perception parameters. The contrast and entropy in the camera system are then adjusted based on the perception parameters.

Abstract: Described is a system to evaluate and reduce perception error in object detection and recognition. The system includes a perception module that receives perception data (of an object(s)) from an environment proximate a mobile platform. Perception probes are generated that describe one or more characteristics of the objects. The perception probes are converted into probabilistic signal temporal logic (PSTL)-based constraints that provide axioms having statistical analysis of the perception probes. The axioms are evaluated to classify the perception probes as valid or erroneous. Optimal perception parameters are generated by solving an optimization problem based on the axioms, which allows the system to adjust the perception module based on the optimal perception parameters.

Abstract: Described is a system for learning actions for image-based action recognition in an autonomous vehicle. The system separates a set of labeled action image data from a source domain into components. The components are mapped onto a set of action patterns, thereby creating a dictionary of action patterns. For each action in the set of labeled action data, a mapping is learned from the action pattern representing the action onto a class label for the action. The system then maps a set of new unlabeled target action image data onto a shared embedding feature space in which action patterns can be discriminated. For each target action in the set of new unlabeled target action image data, a class label for the target action is identified. Based on the identified class label, the autonomous vehicle is caused to perform a vehicle maneuver corresponding to the identified class label.

Abstract: Embodiments include methods, systems and computer readable storage medium for a method for collision avoidance by a vehicle is disclosed. The method includes installing a vehicle system into a vehicle, wherein the vehicle system provides collision avoidance guidance based on training data using movement information from one or more agents and behaviors associated with one or more individuals associated with the one or more agents or the vehicle. The method further includes detecting, by a processor, a collision course between the vehicle and the one or more mobile agents and/or one or more stationary agents. The method further includes calculating, by the processor, one or more decisions that avoid a collision in response to detecting a collision course. The method further includes selecting, by the processor, a decision from the one or more decisions and controlling, by the processor, operation of the vehicle based on the selected decision.

Abstract: Described is a system for action recognition through application of deep embedded clustering. For each image frame of an input video, the system computes skeletal joint-based pose features representing an action of a human in the image frame. Non-linear mapping of the pose features into an embedded action space is performed. Temporal classification of the action is performed and a set of categorical gesture-based labels is obtained. The set of categorical gesture-based labels is used to control movement of a machine.

Abstract: Described is a system for contrast and entropy-based perception adaption to optimize perception. The system is operable for receiving an input image of a scene with a camera system and detecting one or more objects (having perception data) in the input image. The perception data of the one or more objects is converted into probes, which are then converted into axioms using probabilistic signal temporal logic. The axioms are evaluated based on probe bounds. If the axioms are within the probe bounds, then results are provided; however, if the axioms are outside of the probe bounds, the system estimates optimal contrast bounds and entropy bounds as perception parameters. The contrast and entropy in the camera system are then adjusted based on the perception parameters.

Abstract: Described is a system for implicitly predicting movement of an object. In an aspect, the system includes one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations of providing an image of a first trajectory to a predictive autoencoder, and using the predictive autoencoder, generating a predicted tactical response that comprises a second trajectory based on images of previous tactical responses that were used to train the predictive autoencoder, and controlling a device based on the predicted tactical response.

Abstract: Described is a system for detecting and correcting perception errors in a perception system. In operation, the system generates a list of detected objects from perception data of a scene, which allows for the generation of a list of background classes from backgrounds in the perception data associated with the list of detected objects. For each detected object in the list of detected objects, a closest background class is identified from the list of background classes. Vectors can then be used to determine a semantic feature, which is used to identify axioms. An optimal perception parameter is then generated, which is used to adjust perception parameters in the perception system to minimize perception errors.

Abstract: Described is a system to evaluate and reduce perception error in object detection and recognition. The system includes a perception module that receives perception data (of an object(s)) from an environment proximate a mobile platform. Perception probes are generated that describe one or more characteristics of the objects. The perception probes are converted into probabilistic signal temporal logic (PSTL)-based constraints that provide axioms having statistical analysis of the perception probes. The axioms are evaluated to classify the perception probes as valid or erroneous. Optimal perception parameters are generated by solving an optimization problem based on the axioms, which allows the system to adjust the perception module based on the optimal perception parameters.

Abstract: Described is a system for learning actions for image-based action recognition in an autonomous vehicle. The system separates a set of labeled action image data from a source domain into components. The components are mapped onto a set of action patterns, thereby creating a dictionary of action patterns. For each action in the set of labeled action data, a mapping is learned from the action pattern representing the action onto a class label for the action. The system then maps a set of new unlabeled target action image data onto a shared embedding feature space in which action patterns can be discriminated. For each target action in the set of new unlabeled target action image data, a class label for the target action is identified. Based on the identified class label, the autonomous vehicle is caused to perform a vehicle maneuver corresponding to the identified class label.

Abstract: Embodiments include methods, systems and computer readable storage medium for a method for collision avoidance by a vehicle is disclosed. The method includes installing a vehicle system into a vehicle, wherein the vehicle system provides collision avoidance guidance based on training data using movement information from one or more agents and behaviors associated with one or more individuals associated with the one or more agents or the vehicle. The method further includes detecting, by a processor, a collision course between the vehicle and the one or more mobile agents and/or one or more stationary agents. The method further includes calculating, by the processor, one or more decisions that avoid a collision in response to detecting a collision course. The method further includes selecting, by the processor, a decision from the one or more decisions and controlling, by the processor, operation of the vehicle based on the selected decision.

Abstract: Embodiments include methods, systems, and computer readable storage medium for a method for providing path-planning guidance by resolving multiple behavioral predictions associated with operating a vehicle is disclosed. The method includes installing a vehicle system into a vehicle, wherein the vehicle system provides path planning guidance based on training data using and fused hypotheses and/or decisions associated with the training data. The method further includes determining, by a processor, a location of the vehicle on a map containing a road network, and determining, by the processor, whether one or more agents exist within a predetermined range of the vehicle. The method further includes selecting, by the processor, an output trajectory to traverse the road network based on the location of the vehicle on the map and the existence of one or more agents. The method further includes controlling, by the processor, operation of the vehicle using the output trajectory.

Abstract: An autonomous vehicle, system and method of operating the autonomous vehicle. The system includes a sensor, a reasoning engine and a navigation system. The sensor receives token data. The reasoning engine performs an abductive inference on a fact determined from the token data to estimate a backward condition, and a deductive inference to the estimated backward condition in to order to predict a forward condition. The navigation system operates the autonomous vehicle based on the predicted forward condition.