Abstract: Systems and methods are provided for implementing safety-aware artificial intelligence (AI) that can be used for autonomously controlling systems, such as an autonomous vehicle, in a manner that is proven to satisfy given safety constraints. Additionally, a safety-aware training technique can be applied to learned AI-based models, such as neural networks. The safety-aware training techniques can apply automated reasoning tools (ART) while the AI model is trained, in order to produce a model that is provable safe with respect to the safety constraints. The ART can integrate verification into the training process, and thereby dynamically re-train the model based on the safety verification in a feedback loop approach. The ART can be configured to either verify that the AI model is provably safety, or to provide updates to the training parameters used during to re-train the AI model in instances when the safety verification has failed.

Abstract: Systems and methods are provided for monitoring time-series data relative to a temporal logic specification regarding expected behavior of a system, such as a vehicle. The time-series data and a threshold value(s) specified in the temporal logic specification may be encrypted and analyzed without decrypting the time-series data to maintain the privacy of a user(s) of the vehicle. Encryption of the time-series data and the threshold value(s) may be accomplished using an order preserving encryption scheme. Analysis of the time-series data may be accomplished utilizing a batch processing-type architecture or a continuous processing-type architecture. When utilizing the continuous processing-type architecture, historical time-series data may be stored and utilized to determine whether currently-monitored time-series data satisfies the temporal logic specification.

Abstract: Systems and methods are provided for implementing safety-aware artificial intelligence (AI) that can be used for autonomously controlling systems, such as an autonomous vehicle, in a manner that is proven to satisfy given safety constraints. Additionally, a safety-aware training technique can be applied to learned AI-based models, such as neural networks. The safety-aware training techniques can apply automated reasoning tools (ART) while the AI model is trained, in order to produce a model that is provable safe with respect to the safety constraints. The ART can integrate verification into the training process, and thereby dynamically re-train the model based on the safety verification in a feedback loop approach. The ART can be configured to either verify that the AI model is provably safety, or to provide updates to the training parameters used during to re-train the AI model in instances when the safety verification has failed.

Abstract: Systems and methods are provided for monitoring time-series data relative to a temporal logic specification regarding expected behavior of a system, such as a vehicle. The time-series data and a threshold value(s) specified in the temporal logic specification may be encrypted and analyzed without decrypting the time-series data to maintain the privacy of a user(s) of the vehicle. Encryption of the time-series data and the threshold value(s) may be accomplished using an order preserving encryption scheme. Analysis of the time-series data may be accomplished utilizing a batch processing-type architecture or a continuous processing-type architecture. When utilizing the continuous processing-type architecture, historical time-series data may be stored and utilized to determine whether currently-monitored time-series data satisfies the temporal logic specification.

Abstract: Systems and methods are provided for monitoring time-series data relative to a temporal logic specification regarding expected behavior of a system, such as a vehicle. The time-series data and a threshold value(s) specified in the temporal logic specification may be encrypted and analyzed without decrypting the time-series data to maintain the privacy of a user(s) of the vehicle. Encryption of the time-series data and the threshold value(s) may be accomplished using an order preserving encryption scheme. Analysis of the time-series data may be accomplished utilizing a batch processing-type architecture or a continuous processing-type architecture. When utilizing the continuous processing-type architecture, historical time-series data may be stored and utilized to determine whether currently-monitored time-series data satisfies the temporal logic specification.

Abstract: A system for determining desired control paths for controlling operation of a fuel cell circuit includes a memory to store a model of the fuel cell circuit and an input device to receive system requirements. The system also includes a model processor designed to select sets of time-series actuator states corresponding to time-series control of an actuator of the fuel cell circuit and to perform simulations of the model using the multiple sets of time-series actuator states as controls for the actuator. The model processor is also performs an analysis of results of the simulations to determine whether the results for each of the multiple sets of time-series actuator states satisfy the system requirements and how far the results are from the system requirements, and selects a final set of time-series actuator states that satisfy the system requirements based on the analysis.

Abstract: Systems and methods for evaluating closed-loop control systems are disclosed. In one embodiment, a method of evaluating a control system includes determining, using a processing device, one or more convergence classifier functions from a closed-loop model, wherein the one or more convergence classifier functions convey convergent behavior of the closed-loop model over a pre-determined period of time. The method further includes generating, using the processing device, a plurality of test cases of an input space of the closed-loop model under evaluation, and determining, using the processing device, whether one or more individual test cases of the plurality of test cases do not satisfy the one or more convergence classifier functions.

Abstract: A system for determining desired control paths for controlling operation of a fuel cell circuit includes a memory to store a model of the fuel cell circuit and an input device to receive system requirements. The system also includes a model processor designed to select sets of time-series actuator states corresponding to time-series control of an actuator of the fuel cell circuit and to perform simulations of the model using the multiple sets of time-series actuator states as controls for the actuator. The model processor is also performs an analysis of results of the simulations to determine whether the results for each of the multiple sets of time-series actuator states satisfy the system requirements and how far the results are from the system requirements, and selects a final set of time-series actuator states that satisfy the system requirements based on the analysis.

Abstract: Systems and methods are provided for monitoring time-series data relative to a temporal logic specification regarding expected behavior of a system, such as a vehicle. The time-series data and a threshold value(s) specified in the temporal logic specification may be encrypted and analyzed without decrypting the time-series data to maintain the privacy of a user(s) of the vehicle. Encryption of the time-series data and the threshold value(s) may be accomplished using an order preserving encryption scheme. Analysis of the time-series data may be accomplished utilizing a batch processing-type architecture or a continuous processing-type architecture. When utilizing the continuous processing-type architecture, historical time-series data may be stored and utilized to determine whether currently-monitored time-series data satisfies the temporal logic specification.

Abstract: A computer-implemented method for automatically identifying a faulty behavior of a control system. The method includes receiving, at a test processor, a description of the faulty behavior. The method also includes selecting, using the test processor, a goal state based on a heuristic decision. The method also includes selecting, using the test processor, a selected system state. The method also includes selecting, using the test processor, a selected variable to the control system based on the goal state. The method also includes loading, from a memory, a control model of the control system. The method also includes performing, using the test processor, a simulation of the control model using the selected variable and the selected system state as parameters of the simulation. The method also includes determining, using the test processor, whether the faulty behavior was observed based on the simulation.

Abstract: Systems and methods for evaluating closed-loop control systems are disclosed. In one embodiment, a method of evaluating a control system includes determining, using a processing device, one or more convergence classifier functions from a closed-loop model, wherein the one or more convergence classifier functions convey convergent behavior of the closed-loop model over a pre-determined period of time. The method further includes generating, using the processing device, a plurality of test cases of an input space of the closed-loop model under evaluation, and determining, using the processing device, whether one or more individual test cases of the plurality of test cases do not satisfy the one or more convergence classifier functions.

Abstract: A computer-implemented method for automatically identifying a faulty behavior of a control system. The method includes receiving, at a test processor, a description of the faulty behavior. The method also includes selecting, using the test processor, a goal state based on a heuristic decision. The method also includes selecting, using the test processor, a selected system state. The method also includes selecting, using the test processor, a selected variable to the control system based on the goal state. The method also includes loading, from a memory, a control model of the control system. The method also includes performing, using the test processor, a simulation of the control model using the selected variable and the selected system state as parameters of the simulation. The method also includes determining, using the test processor, whether the faulty behavior was observed based on the simulation.

Abstract: In one embodiment, a method of evaluating stability of software code for a control system includes receiving a set of initial trajectories by a semidefinite programming solver module, and determining one or more candidate Lyapunov functions based on the set of initial trajectories. The method further includes performing a plurality of simulations using a model of the control system to create a set of discovered trajectories, and evaluating the set of discovered trajectories to determine one or more counterexample trajectories that violate one or more Lyapunov conditions. If one or more counterexample trajectories are discovered, then the method includes inputting the set of discovered trajectories including the one or more counterexample trajectories into the semidefinite programming solver module, and determining, by the semidefinite programming solver module, one or more additional candidate Lyapunov functions from the set of initial trajectories and the set of discovered trajectories.

Abstract: Systems and methods for mining a temporal requirement from a block diagram model of a closed loop control system are disclosed. One embodiment of a method includes simulating the closed loop control system of a vehicle to obtain simulation traces and determining a candidate requirement by instantiating a template requirement with values of the simulation traces to locate parameter values that suggest that the template requirement is fulfilled. Some embodiments of the method include determining whether a counterexample to the candidate requirement exists; and in response to determining that the counterexample to the candidate requirement exists, obtaining the counterexample to the candidate requirement and adding the counterexample to the simulation traces for inspection.