Abstract: K-nearest multi-agent reinforcement learning for collaborative tasks with variable numbers of agents. Centralized reinforcement learning is challenged by variable numbers of agents, whereas decentralized reinforcement learning is challenged by dependencies among agents' actions. An algorithm is disclosed that can address both of these challenges, among others, by grouping agents with their k-nearest agents during training and operation of a policy network. The observations of all k+1 agents in each group are used as the input to the policy network to determine the next action tor each of the k+1 agents in the group. When an agent belongs to more than one group, such that multiple actions are determined for the agent, an aggregation strategy can be used to determine the final action for that agent.

Abstract: Example implementations involve fault detection and isolation in industrial networks through defining a component as a combination of measurements and parameters and define an industrial network as a set of components connected with different degrees of connections (weights). Faults in industrial network are defined as unpermitted changes in component parameters. Further, the fault detection and isolation in industrial networks are formulated as a node classification problem in graph theory. Example implementations detect and isolate faults in industrial networks through 1) uploading/learning network structure, 2) detecting component communities in the network, 3) extracting features for each community, 4) using the extracted features for each community to detect and isolate faults, 5) at each time step, based on the faulty components provide maintenance recommendation for the network.

Abstract: Example implementations described herein involve an approach to address an imperfect simulator challenge using off-line data plus reward modification. The proposed solution is robust to simulator error, and therefore, it requires less maintenance in keeping the simulators updated. Even when the simulators are accurate, it is costly to keep them accurate over time. Moreover, compared to other robust reinforcement learning algorithms, the proposed approach does not assume the distribution of uncertainties in the simulator are known. Less complexity leads to fewer potential errors as well as lower computational cost during the training. Finally, the proposed approach has better performance compared to the state-of-the-art methods (higher overall cumulative rewards).

Abstract: Example implementations described herein involve a new data-driven analytical redundancy relationship (ARR) generation for fault detection and isolation. The proposed solution uses historical data during normal operation to extract the data-driven ARRs among sensor measurements, and then uses them for fault detection and isolation. The proposed solution thereby does not need to rely on the system model, can detect and isolate more faults than traditional data-driven methods, can work when the system is not fully observable, and does not rely on a vast amount of historical fault data, which can save on memory storage or database storage. The proposed solution can thereby be practical in many real cases where there are data limitations.

Abstract: In example implementations described herein, there are systems and methods for processing sensor data from an equipment over a period of time to generate sensor time series data; processing the sensor time series data in a kernel weight layer configured to generate weights to weigh the sensor time series data; providing the weighted sensor time series data to fully connected layers configured to conduct a correlation on the weighted sensor time series data with predictive maintenance labels to generate an intermediate predictive maintenance label; and providing the intermediate predictive maintenance label to an inversed kernel weight layer configured to inverse the weights generated by the kernel weight layer, to generate a predictive maintenance label for the equipment.

Abstract: Example implementations described herein involve a new data-driven analytical redundancy relationship (ARR) generation for fault detection and isolation. The proposed solution uses historical data during normal operation to extract the data-driven ARRs among sensor measurements, and then uses them for fault detection and isolation. The proposed solution thereby does not need to rely on the system model, can detect and isolate more faults than traditional data-driven methods, can work when the system is not fully observable, and does not rely on a vast amount of historical fault data, which can save on memory storage or database storage. The proposed solution can thereby be practical in many real cases where there are data limitations.

Abstract: Example implementations described herein involve a new reinforcement learning algorithm to address short-term goals. In the training step, the proposed solution learns the system dynamic model (short-term prediction) in a linear format in terms of actions. It also learns the expected rewards (long-term prediction) in a linear format in terms of actions. In the application step, the proposed solution uses the learned models plus simple optimization algorithms to find actions that satisfy both short-term goals and long-term goals. Through the example implementations, there is no need to design sensitive reward functions for achieving short-term and long-term goals concurrently. Further, there is better performance in achieving short-term and long-term goals compared to the traditional reward modification methods, and it is possible to modify the short-term goals without time-consuming retraining.

Abstract: In example implementations described herein, there are systems and methods for processing sensor data from an equipment over a period of time to generate sensor time series data; processing the sensor time series data in a kernel weight layer configured to generate weights to weigh the sensor time series data; providing the weighted sensor time series data to fully connected layers configured to conduct a correlation on the weighted sensor time series data with predictive maintenance labels to generate an intermediate predictive maintenance label; and providing the intermediate predictive maintenance label to an inversed kernel weight layer configured to inverse the weights generated by the kernel weight layer, to generate a predictive maintenance label for the equipment.

Abstract: Example implementations involve fault detection and isolation in industrial networks through defining a component as a combination of measurements and parameters and define an industrial network as a set of components connected with different degrees of connections (weights). Faults in industrial network are defined as unpermitted changes in component parameters. Further, the fault detection and isolation in industrial networks are formulated as a node classification problem in graph theory. Example implementations detect and isolate faults in industrial networks through 1) uploading/learning network structure, 2) detecting component communities in the network, 3) extracting features for each community, 4) using the extracted features for each community to detect and isolate faults, 5) at each time step, based on the faulty components provide maintenance recommendation for the network.