Deep Q Network (DQN) is an efficient model-free optimization method, and has the potential to be used in building cooling water systems. However, due to the high dimension of actions, this method requires a complex neural network. Therefore, both the required number of training samples and the length of convergence period are barriers for real application. Furthermore, penalty function based exploration may lead to unsafe actions, causing the application of this optimization method even more difficult. To solve these problems, an approach to limit the action space within a safe area is proposed in this paper. First of all, the action space for cooling towers and pumps are separated into two sub-regions. Secondly, for each type of equipment, the action space is further divided into safe and unsafe regions. As a result, the convergence speed is significantly improved. Compared with the traditional DQN method in a simulation environment validated by real data, the proposed method is able to save the convergence time by 1 episode (one cooling season). The results in this paper suggest that, the proposed DQN method can achieve a much quicker learning speed without any undesired consequences, and therefore is more suitable to be used in projects without pre-learning stage.

Air conditioning water systems account for a large proportion of building energy consumption. In a pressure-controlled water system, one of the key measures to save energy is to adjust the differential pressure setpoints during operation. Typically, such adjustments are based either on certain rules, which rely on operator experience, or on complicated models that are not easy to calibrate. In this paper, a data-driven control method based on reinforcement learning is proposed. The main idea is to construct an agent model that adapts to the researched problem. Instead of directly being told how to react, the agent must rely on its own experiences to learn. Compared with traditional control strategies, reinforcement learning control (RLC) exhibits more accurate and steady performances while maintaining indoor air temperature within a limited range. A case study shows that the RLC strategy is able to save substantial amounts of energy.

Operation strategies influence the building energy efficiency. In order to enhance the building energy efficiency, it’s necessary to adopt proper operation strategies on building equipment. Thus, the identification of existing operation strategies is necessary for the improvement of operation strategies. A data mining (DM) based framework is proposed in this paper to automatically identify the building operation strategies. The framework includes classification and regression tree (CART), and weighted association rule mining (WARM) method, targeting at three types of rule based control strategies: on/off control, sequencing control (for equipment of the same type), and coordinated control (for equipment of different types). The performance of this framework is validated with power metering system data and manual identification results based on on-site survey of three buildings in Shanghai. The validation results suggest that the proposed framework is capable of identifying building operation strategies accurately and automatically. Implemented on the original software named BOSA (Building Operation Strategy Analysis), this framework is promising to be used in engineering field to enhance the efficiency of building operation strategy identification work.