AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Building Simulation Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article

Differential pressure reset strategy based on reinforcement learning for chilled water systems

Xinfang Zhang1, Zhenhai Li1, Zhengwei Li1,2( ), Shunian Qiu1, Hai Wang1
School of Mechanical Engineering, Tongji University, Shanghai, China
Key Laboratory of Performance Evolution and Control for Engineering Structures of Ministry of Education, Tongji University, Shanghai, China
Show Author Information

Abstract

Air conditioning water systems account for a large proportion of building energy consumption. In a pressure-controlled water system, one of the key measures to save energy is to adjust the differential pressure setpoints during operation. Typically, such adjustments are based either on certain rules, which rely on operator experience, or on complicated models that are not easy to calibrate. In this paper, a data-driven control method based on reinforcement learning is proposed. The main idea is to construct an agent model that adapts to the researched problem. Instead of directly being told how to react, the agent must rely on its own experiences to learn. Compared with traditional control strategies, reinforcement learning control (RLC) exhibits more accurate and steady performances while maintaining indoor air temperature within a limited range. A case study shows that the RLC strategy is able to save substantial amounts of energy.

Keywords

energy savingdifferential pressure resetwater systemreinforcement learning control

Electronic Supplementary Material

Download File(s)
bs-15-2-233_ESM.zip (3.2 MB)

References

 
Barrett E, Linder S (2015). Autonomous HVAC control, a reinforcement learning approach. In: Bifet A, et al. (eds), Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2015. Lecture Notes in Computer Science, vol 9286. Cham, Switzerland. https://doi.org/10.1007/978-3-319-23461-8_1
Crossref
 

Chen Y, Norford LK, Samuelson HW, et al. (2018). Optimal control of HVAC and window systems for natural ventilation through reinforcement learning. Energy and Buildings, 169: 195–205.
Crossref Google Scholar
 

Dayan P, Niv Y (2008). Reinforcement learning: The good, the bad and the ugly. Current Opinion in Neurobiology, 18: 185–196.
Crossref Google Scholar
 

Gao D, Wang S, Shan K (2016). In-situ implementation and evaluation of an online robust pump speed control strategy for avoiding low delta-T syndrome in complex chilled water systems of high-rise buildings. Applied Energy, 171: 541–554.
Crossref Google Scholar
 

Han M, May R, Zhang X, et al. (2019). A review of reinforcement learning methodologies for controlling occupant comfort in buildings. Sustainable Cities and Society, 51: 101748.
Crossref Google Scholar
 

Hou J, Xu P, Lu X, et al. (2018). Implementation of expansion planning in existing district energy system: A case study in China. Applied Energy, 211: 269–281.
Crossref Google Scholar
 
Ji Y, Peng S, Geng L, et al. (2009). Pressure loop control of pump and valve combined EHA based on FFIM. In: Proceedings of the 9th International Conference on Electronic Measurement and Instruments, Beijing, China. https://doi.org/10.1109/ICEMI.2009.5274250
Crossref
 

Jin X, Du Z, Xiao X (2007). Energy evaluation of optimal control strategies for central VWV chiller systems. Applied Thermal Engineering, 27: 934–941.
Crossref Google Scholar
 

Lewis FL, Vrabie D, Vamvoudakis KG (2012). Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers. IEEE Control Systems Magazine, 32(6): 76–105.
Crossref Google Scholar
 

Li W, Xu P, Lu X, et al. (2016). Electricity demand response in China: Status, feasible market schemes and pilots. Energy, 114: 981–994.
Crossref Google Scholar
 

Liu S, Henze GP (2006a). Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory: Part 1. Theoretical foundation. Energy and Buildings, 38: 142–147.
Crossref Google Scholar
 

Liu S, Henze GP (2006b). Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory: Part 2: Results and analysis. Energy and Buildings, 38: 148–161.
Crossref Google Scholar
 

Liu X, Liu J, Lu J, et al. (2012). Research on operating characteristics of direct-return chilled water system controlled by variable temperature difference. Energy, 40: 236–249.
Crossref Google Scholar
 

Ma Z, Wang S (2009). Energy efficient control of variable speed pumps in complex building central air-conditioning systems. Energy and Buildings, 41: 197–205.
Crossref Google Scholar
 

Mehlhase A (2012). A python package for simulating variable-structure models with dymola. IFAC Proceedings Volumes, 45: 1081–1086.
Crossref Google Scholar
 

Pérez-Lombard L, Ortiz J, Pout C (2008). A review on buildings energy consumption information. Energy and Buildings, 40: 394–398.
Crossref Google Scholar
 

Recht B (2019). A tour of reinforcement learning: The view from continuous control. Annual Review of Control, Robotics, and Autonomous Systems, 2: 253–279.
Crossref Google Scholar
 

Sutton R, Barto A (2018). Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT Press.
 

Wang Z, Hong T (2020). Reinforcement learning for building controls: The opportunities and challenges. Applied Energy, 269: 115036.
Crossref Google Scholar
 
Wei T, Wang Y, Zhu Q (2017). Deep Reinforcement Learning for Building HVAC Control. In: Proceedings of the 54th Annual Design Automation Conference, Austin, TX, USA. https://doi.org/10.1145/3061639.3062224
Crossref
 

Yang L, Nagy Z, Goffin P, Schlueter A (2015). Reinforcement learning for optimal control of low exergy buildings. Applied Energy, 156: 577–586.
Crossref Google Scholar
 

Zhao T, Ma L, Zhang J (2016). An optimal differential pressure reset strategy based on the most unfavorable thermodynamic loop on-line identification for a variable water flow air conditioning system. Energy and Buildings, 110: 257–268.
Crossref Google Scholar
 

Zhou Y, Chen J, Yu ZJ, et al. (2020). A novel model based on multi-grained cascade forests with wavelet denoising for indoor occupancy estimation. Building and Environment, 167: 106461.
Crossref Google Scholar
Building Simulation
Volume 15 Issue 2,
February 2022
Pages 233-248
DOI: 10.1007/s12273-021-0808-5
Cite this article:
Zhang X, Li Z, Li Z, et al. Differential pressure reset strategy based on reinforcement learning for chilled water systems. Building Simulation, 2022, 15(2): 233-248. https://doi.org/10.1007/s12273-021-0808-5

656

Views

8

Crossref

8

Web of Science

11

Scopus

0

CSCD

Altmetrics
Received: 12 December 2020
Revised: 03 April 2021
Accepted: 13 April 2021
Published: 19 August 2021
© Tsinghua University Press and Springer-Verlag GmbH Germany, part of Springer Nature 2021
Return