Abstract: A system with deep reinforcement learning based control determines optimal actions for major components in a commercial building to minimize operation costs while maximizing comprehensive comfort levels of occupants. An unsupervised deep Q-network method is introduced to handle the energy management problem by evaluating the influence of operation costs on comfort levels considering the environment factors at each time slot. An optimum control decision can be derived that targets both immediate and long-term goals, where exploration and exploitation are considered simultaneously.

Abstract: Systems and methods are disclosed for control voltage profiles, line flows and transmission losses of a power grid by forming an autonomous multi-objective control model with one or more neural networks as a Deep Reinforcement Learning (DRL) agent; training the DRL agent to provide data-driven, real-time and autonomous grid control strategies; and coordinating and optimizing power controllers to regulate voltage profiles, line flows and transmission losses in the power grid with a Markov decision process (MDP) operating with reinforcement learning to control problems in dynamic and stochastic environments.

Abstract: State estimation in an electric power system includes acquiring electrical measurements from the electric power system at a reporting rate of the electrical measurements, processing the electrical measurements into sequence data including positive sequence data, processing positive sequence data by an observable state estimator to generate a plurality of estimated states including a plurality of estimated observable states, parameters of the observable state estimator being updated by a first training module in a first time thread, processing the plurality of estimated states by an unobservable state estimator to generate a plurality of estimated unobservable states, parameters of the unobservable state estimator being updated by a second training module in a second time thread independent of the first time thread, and outputting a plurality of final estimated states generated by concatenating the plurality of estimated observable states and the plurality of estimated unobservable states.

Abstract: A system with deep reinforcement learning based control determines optimal actions for major components in a commercial building to minimize operation costs while maximizing comprehensive comfort levels of occupants. An unsupervised deep Q-network method is introduced to handle the energy management problem by evaluating the influence of operation costs on comfort levels considering the environment factors at each time slot. An optimum control decision can be derived that targets both immediate and long-term goals, where exploration and exploitation are considered simultaneously.

Abstract: Systems and methods are disclosed for control voltage profiles, line flows and transmission losses of a power grid by forming an autonomous multi-objective control model with one or more neural networks as a Deep Reinforcement Learning (DRL) agent; training the DRL agent to provide data-driven, real-time and autonomous grid control strategies; and coordinating and optimizing power controllers to regulate voltage profiles, line flows and transmission losses in the power grid with a Markov decision process (MDP) operating with reinforcement learning to control problems in dynamic and stochastic environments.

Abstract: Techniques and apparatuses are described that enable transformative Remedial Action Scheme (RAS) analyses and methodologies for a bulk electric power system, including methods of designing, reviewing, revising, testing, implementing, verifying, or validating a RAS. An improved RAS improves operation of the power system, including performance, reliability, control, and asset utilization. The example methodologies discussed—also referred to as a transformative Remedial Action Scheme tool (TRAST)—provide an end-to-end solution for adaptively setting RAS parameters based on realistic and near real-time operation conditions to improve power grid reliability and grid asset utilization, by leveraging utility data analysis and employing dynamic simulations and machine learning to significantly simplify and shorten the entire RAS process.

Abstract: Systems and methods for autonomous line flow control in an electric power system are disclosed which includes acquiring state information at a line in the electric power system at a first time step, obtaining a flow data of the line at a next time step based on the acquired state information, generating an early warning signal when the obtained flow data is higher than a predetermined threshold, activating a deep reinforcement learning (DRL) agent to generate an action using a DRL algorithm based on the state information, and executing the action to adjust a topology of the electric power system.

Abstract: Systems and methods for autonomous voltage control in an electric power system are disclosed which include acquiring state information at buses of the electric power system, detecting a state violation from the state information, generating a first action setting based on the state violation using a deep reinforcement learning (DRL) algorithm by a first artificial intelligent (AI) agent assigned to a first region of the electric power system where the state violation occurs, and maintaining a second action setting by a second AI agent assigned to a second region of the electric power system where no substantial state violation is detected.

Abstract: State estimation in an electric power system includes acquiring electrical measurements from the electric power system at a reporting rate of the electrical measurements, processing the electrical measurements into sequence data including positive sequence data, processing positive sequence data by an observable state estimator to generate a plurality of estimated states including a plurality of estimated observable states, parameters of the observable state estimator being updated by a first training module in a first time thread, processing the plurality of estimated states by an unobservable state estimator to generate a plurality of estimated unobservable states, parameters of the unobservable state estimator being updated by a second training module in a second time thread independent of the first time thread, and outputting a plurality of final estimated states generated by concatenating the plurality of estimated observable states and the plurality of estimated unobservable states.

Abstract: Autonomous parameter calibration for a model of an electric power system includes inputting electric measurements, simulating the model with a set of parameters to generate a first simulated response, identifying a first and a second parameter in the set of parameters, the first parameter being responsible for a deviation of the first simulated response from the electric measurements, while the second parameter being not responsible to the deviation, generating an action corresponding to the first parameter by a DRL agent based on the deviation, modifying the first parameter by the generated action while leaving the second parameter unmodified, simulating the model again with the set of parameters including the modified first parameter and the unmodified second parameter to generate a second simulated response, evaluating a fitting error between the second simulated response and the electric measurements, and terminating the parameter calibration when the fitting error falls below a predetermined threshold.

Abstract: Techniques and apparatuses are described that enable transformative Remedial Action Scheme (RAS) analyses and methodologies for a bulk electric power system, including methods of designing, reviewing, revising, testing, implementing, verifying, or validating a RAS. An improved RAS improves operation of the power system, including performance, reliability, control, and asset utilization. The example methodologies discussed—also referred to as a transformative Remedial Action Scheme tool (TRAST)—provide an end-to-end solution for adaptively setting RAS parameters based on realistic and near real-time operation conditions to improve power grid reliability and grid asset utilization, by leveraging utility data analysis and employing dynamic simulations and machine learning to significantly simplify and shorten the entire RAS process.

Abstract: Systems and methods are disclosed for controlling a power system by formulating a voltage control problem using a deep reinforcement learning (DRL) method with a control objective of training a DRL-agent to regulate the bus voltages of a power grid within a predefined zone before and after a disturbance; performing offline training with historical data to train the DRL agent; performing online retraining of the DRL agent using live PMU data; and providing autonomous control of the power system below a sub-second after training.

Abstract: Systems and methods are disclosed to control a power inverter by extracting photovoltaic (PV) maximum power under a plurality of operating conditions as one or more reference variables; predicting the future behaviors of a PV output voltage as one or more controlled variables based on a current operational condition; generating one or more control signals for the H5 inverter that minimizes an error between the reference variable and the controlled variables; and controlling the power inverter with the one or more control signals.

Abstract: Systems and methods are disclosed to control voltage profiles of a power grid by forming an autonomous voltage control model with one or more neural networks as Deep Reinforcement Learning (DRL) agents; training the DRL agents to provide data-driven, real-time and autonomous grid control strategies; and coordinating and optimizing reactive power controllers to regulate voltage profiles in the power grid with a Markov decision process (MDP) operating with reinforcement learning to control problems in dynamic and stochastic environments.

Abstract: Systems and methods are disclosed to control a power inverter by extracting photovoltaic (PV) maximum power under a plurality of operating conditions as one or more reference variables; predicting the future behaviors of a PV output voltage as one or more controlled variables based on a current operational condition; generating one or more control signals for the H5 inverter that minimizes an error between the reference variable and the controlled variables; and controlling the power inverter with the one or more control signals.

Abstract: A time-stacking method is disclosed. The time-stacking method simulates dynamics so as to obtain time-domain trajectories which correspond to a disturbance or event. The method includes providing a model for the system. The model includes differential equations and algebraic equations. The method also includes solving the differential equations and the algebraic equations over a predetermined number of time steps simultaneously using an implicit integration scheme.