Abstract: Systems and methods for training a reinforcement learning (RL) model for HVAC control are disclosed herein. Simulated experience data for the HVAC system is generated or received. The simulated experience data is used to initially train the RL model for HVAC control. The HVAC system operates within a building using the RL model and generates real experience data. A determination may be made to retrain the RL model. The real experience data is used to retrain the RL model. In some embodiments, both the simulated and real experience data are used to retrain the RL model. Experience data may be sampled according to various sampling functions. The RL model may be retrained multiple times over time. The RL model may be retrained less frequently over time as more real experience data is used to train the RL model.

Abstract: A building system including one or more memory devices storing instructions that, when executed by one or more processors cause the one or more processors to store a digital twin for a piece of building equipment, the digital twin comprising a virtual representation of the piece of building equipment, wherein the digital twin communicates with the piece of building equipment to operate the piece of building equipment and determine one or more diagnostic messages based on the virtual representation of the piece of building equipment and communicate the one or more diagnostic messages, by the digital twin, to the piece of building equipment causing the piece of building equipment to perform one or more operations. The instructions cause the one or more processors to receive one or more diagnostic message and generate a diagnostics report for the piece of building equipment based on the one or more diagnostic message responses.

Abstract: A building system operates to store a digital twin comprising a building graph, the building graph comprising a plurality of nodes representing a plurality of entities of a building and a plurality of edges between the plurality of nodes representing relationships between the plurality of entities. The instructions cause the one or more processors to determine a value for a functionality indicator for a piece of building equipment based on data received from the piece of building equipment, identify a first node of the plurality of nodes representing the functionality indicator by identifying an edge of the plurality of edges relating a second node of the plurality of nodes representing the piece of building equipment to the first node, and cause the first node to store the value for the functionality indicator, or a link to the value for the functionality indicator.

Abstract: A building system including one or more memory devices storing instructions that, when executed by one or more processors, cause the one or more processors to store a plurality of digital twins, the plurality of digital twins comprising a virtual representation of a building, determine, based on the virtual representation of the building, that an operation of the first piece of building equipment is detectable by the second piece of building equipment. The instructions cause the one or more processors to execute a diagnostics routine comprising causing, by the first digital twin, the first piece of building equipment to perform the operation and receiving, by the second digital twin, one or more detections of the operation by the second piece of building equipment and generate a diagnostics report for the first piece of building equipment and the second piece of building equipment based on a result of the diagnostics routine.

Abstract: A building system can operate to receive an indication from a user device of a user to query a digital twin of one or more buildings, the digital twin including entities including buildings, equipment, spaces, or data of the one or more buildings, the equipment, or the spaces, the digital twin including relationships between the entities. The building system can operate to receive building data from the digital twin by querying the digital twin based on the indication received from the user device. The building system can operate to generate an analytics model based on the building data, wherein the analytics model is trained based on the building data and deploy the analytics model to operate based on data of the one or more buildings and generate one or more analytic results based on the data of the one or more buildings.

Abstract: A building system of a building operates to select an instance of one or more entities of one or more particular entity types from a digital twin of the building for creating a policy function, the digital twin including representations of entities of the building and relationships between the entities of the building. The building system operates to perform an optimization that selects one or more inputs of inputs associated with the one or more entities for input to the policy function, selects one or more actions of actions associated with the one or more entities that are outputs of the policy function, and identifies one or more parameters for the policy function. The building system operates to deploy the policy function for the one or more entities by causing the digital twin to include the policy function.

Abstract: Systems and methods for training a reinforcement learning (RL) model for HVAC control are disclosed herein. A calibrated simulation model is used to train a surrogate model of the HVAC system operating within a building. The surrogate model is used to generate simulated experience data for the HVAC system. The simulated experience data can be used to train a reinforcement learning (RL) model of the HVAC system. The RL model is used to control the HVAC system based on the current state of the system and the best predicted action to perform in the current state. The HVAC system generates real experience data based on the actual operation of the HVAC system within the building. The real experience data is used to retrain the surrogate model, and additional simulated experience data is generated using the surrogate model. The RL model can be retrained using the additional simulated experience data.

Abstract: Systems and methods for training a reinforcement learning (RL) model for HVAC control are disclosed herein. Simulated experience data for the HVAC system is generated or received. The simulated experience data is used to initially train the RL model for HVAC control. The HVAC system operates within a building using the RL model and generates real experience data. A determination may be made to retrain the RL model. The real experience data is used to retrain the RL model. In some embodiments, both the simulated and real experience data are used to retrain the RL model. Experience data may be sampled according to various sampling functions. The RL model may be retrained multiple times over time. The RL model may be retrained less frequently over time as more real experience data is used to train the RL model.

Abstract: Systems and methods for training a reinforcement learning (RL) model for HVAC control are disclosed herein. Simulated experience data for the HVAC system is generated or received. The simulated experience data is used to initially train the RL model for HVAC control. The HVAC system operates within a building using the RL model and generates real experience data. A determination may be made to retrain the RL model. The real experience data is used to retrain the RL model. In some embodiments, both the simulated and real experience data are used to retrain the RL model. Experience data may be sampled according to various sampling functions. The RL model may be retrained multiple times over time. The RL model may be retrained less frequently over time as more real experience data is used to train the RL model.

Abstract: A method for controlling temperature in a building zone to increase comfort and energy efficiency is shown. The method includes receiving historical data, the historical data indicative of the temperature and occupancy of the building zone during one or more historical states. The method includes training a system model to represent a dynamic response of the building zone based on the historical data. The method includes determining a control law by optimizing a policy function implemented as a neural network configured to process the trained system model. The method includes performing online control of the building zone using the control law.

Abstract: Systems and methods for training a reinforcement learning (RL) model for HVAC control are disclosed herein. Simulated experience data for the HVAC system is generated or received. The simulated experience data is used to initially train the RL model for HVAC control. The HVAC system operates within a building using the RL model and generates real experience data. A determination may be made to retrain the RL model. The real experience data is used to retrain the RL model. In some embodiments, both the simulated and real experience data are used to retrain the RL model. Experience data may be sampled according to various sampling functions. The RL model may be retrained multiple times over time. The RL model may be retrained less frequently over time as more real experience data is used to train the RL model.

Abstract: Systems and methods for training a reinforcement learning (RL) model for HVAC control are disclosed herein. A calibrated simulation model is used to train a surrogate model of the HVAC system operating within a building. The surrogate model is used to generate simulated experience data for the HVAC system. The simulated experience data can be used to train a reinforcement learning (RL) model of the HVAC system. The RL model is used to control the HVAC system based on the current state of the system and the best predicted action to perform in the current state. The HVAC system generates real experience data based on the actual operation of the HVAC system within the building. The real experience data is used to retrain the surrogate model, and additional simulated experience data is generated using the surrogate model. The RL model can be retrained using the additional simulated experience data.

Abstract: A method for selectively removing an aluminide coating from at least one surface of a metal-based substrate by: (a) contacting the surface of the substrate with at least one stripping composition comprising nitric acid at a temperature less than about 20° C. to degrade the coating without damaging the substrate; and (b) removing the degraded coating without damaging the substrate. Also disclosed is a method for replacing a worn or damaged aluminide coating on at least one surface of a metal-based substrate by selectively removing the coating using the above steps, and then applying a new aluminide coating to the surface of the substrate. Turbine engine parts, such as high-pressure turbine blades, treated using the above methods are also disclosed.

Abstract: A method for selectively removing an aluminide coating from at least one surface of a metal-based substrate by: (a) contacting the surface of the substrate with at least one stripping composition comprising nitric acid at a temperature less than about 20° C. to degrade the coating without damaging the substrate; and (b) removing the degraded coating without damaging the substrate. Also disclosed is a method for replacing a worn or damaged aluminide coating on at least one surface of a metal-based substrate by selectively removing the coating using the above steps, and then applying a new aluminide coating to the surface of the substrate. Turbine engine parts, such as high-pressure turbine blades, treated using the above methods are also disclosed.