Abstract: A computer-implemented method using a reinforcement learning trained reduced order estimator (RL-trained ROE) and a closure model is provided for controlling a heating, ventilation, and air conditioning (HVAC) system including actuators. The method uses a processor coupled with a memory storing instructions implementing the method, wherein the instructions, when executed by the processor, carry out at steps of the method, includes acquiring setpoints of the HVAC system from a user input and measurement data from sensors arranged in the HVAC system, computing a high-dimensional state estimate using the measurement data and an estimate of reduced-order state from the RL-trained ROE, determining a controller with respect to the setpoints by using the RL-trained ROE, generating control commands based on the controller, and transmitting the control commands to the actuators of HVAC system via an output interface.