Evaluating Performance of Reinforcement Learning Agents to Control Buildings Efficiently

Judah Goldfeder, Gabriel Guerra Trigo, Philippe Martin Wyder, Neil Kachappilly, Hod Lipson; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 2903-2909

The recently introduced Smart Buildings Control Suite provides an open-source, physics-informed simulator for developing building HVAC control agents, yet its initial presentation lacked comprehensive empirical performance results. This paper addresses this gap by presenting a quantitative benchmark evaluation of standard reinforcement learning (RL) algorithms within this specific simulation environment. Our primary objective is to establish performance characteristics and demonstrate the suite's capability for evaluating modern control strategies. We train and evaluate Soft Actor-Critic (SAC) and Deep Deterministic Policy Gradient (DDPG) agents using the suite's simulator configured for Building SB1. The evaluation focuses on learning efficiency, final control performance compared to a baseline schedule, and, critically, generalization across diverse seasonal conditions not seen during training. Additionally, with the goal of encouraging the adoption of control policies by real buildings, we analyze the performance implications of a policy extraction technique which aggregates agents' policies into predictable, static schedules which can be easily implemented in buildings. Our results provide quantitative evidence that both SAC and DDPG agents can be effectively trained within the suite, achieving significantly better performance than the baseline policy, and successfully generalizing across different seasons. Furthermore, the analysis shows policy extraction incurs minimal performance loss for up to 8-hour aggregation bins, demonstrating that there is opportunity for simpler, more interpretable policies, which would still benefit performance. This work establishes initial performance benchmarks for the Smart Buildings Control Suite, validating its use for RL research, and motivating further research.