AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home CSEE Journal of Power and Energy Systems Article
PDF (1.3 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Data-driven Optimal Dynamic Dispatch for Hydro-PV-PHS Integrated Power Systems Using Deep Reinforcement Learning Approach

Jingxian Yang1, Jichun Liu1, Yue Xiang1 ( ), Shuai Zhang2, Junyong Liu1
College of Electrical Engineering, Sichuan University, Chengdu 610065, China
State Grid Sichuan Electric Power Company, Chengdu 610065, China
Show Author Information

Abstract

To utilize electricity in a clean and integrated manner, a zero-carbon hydro-photovoltaic (PV)-pumped hydro storage (PHS) integrated power system is studied, considering the uncertainties of PV and load demand. It is a challenge for operators to develop a dynamic dispatch mechanism for such a system, and traditional dispatch methods are difficult to adapt to random changes in the actual environment. Therefore, this study proposes a real-time dynamic dispatch strategy considering economic operation and complementary regulatory ability. First, the dynamic dispatch of a hydro-PV-PHS integrated power system is presented as a multi-objective optimization problem and the weight factor between different goals is effectively calculated using information entropy. Afterwards, the dispatch model is converted into the Markov decision process, where the dynamic dispatch decision is formulated as a reinforcement learning framework. Then, a deep deterministic policy gradient (DDPG) is deployed towards the online decision for dispatch in continuous action spaces. Finally, a case study is applied to evaluate the performance of the proposed method based on a real hydro-PV-PHS integrated power system in China. Simulations show that the system agent reduces the power volatility of supply by 26.7% after hydropower regulating and further relieves power fluctuation at the point of common coupling (PCC) to the upper-level grid by 3.28% after PHS participation. The comparison results verify the effectiveness of the proposed method.

Keywords

information entropyuncertaintiesDDPGdynamic economic dispatchhydro-PV-PHS integrated power system

References

[1]

T. M. Alabi, L. Lu, and Z. Y. Yang, “Stochastic optimal planning scheme of a zero-carbon multi-energy system (ZC-MES) considering the uncertainties of individual energy demand and renewable resources: an integrated chance-constrained and decomposition algorithm (CC-DA) approach,” Energy, vol. 232, pp. 121000, May 2021.
Crossref Google Scholar
[2]

T. M. Alabi, L. Lu, and Z. Y. Yang, “Improved hybrid inexact optimal scheduling of virtual powerplant (VPP) for zero-carbon multi-energy system (ZCMES) incorporating Electric Vehicle (EV) multi-flexible approach,” Journal of Cleaner Production, vol. 326, pp. 129294, Oct. 2021.
Crossref Google Scholar
[3]

D. Wu, M. Javadi, and J. N. Jiang, “A preliminary study of impact of reduced system inertia in a low-carbon power system,” Journal of Modern Power Systems and Clean Energy, vol. 3, no. 1, pp. 82–92, Mar. 2015.
Crossref Google Scholar
[4]

L. X. Tian, Y. S. Huang, S. Liu, S. Z. Sun, J. J. Deng, and H. F. Zhao, “Application of photovoltaic power generation in rail transit power supply system under the background of energy low carbon transformation,” Alexandria Engineering Journal, vol. 60, no. 6, pp. 5167–5174, Jun. 2021.
Crossref Google Scholar
[5]

N. Li and K. W. Hedman, “Economic assessment of energy storage in systems with high levels of renewable resources,” IEEE Transactions on Sustainable Energy, vol. 6, no. 3, pp. 1103–1111, Jul. 2015.
Crossref Google Scholar
[6]

N. Li, C. Uçkun, E. M. Constantinescu, J. R. Birge, K. W. Hedman, and A. Botterud, “Flexible operation of batteries in power system scheduling with renewable energy,” IEEE Transactions on Sustainable Energy, vol. 7, no. 2, pp. 685–696, Apr. 2016.
Crossref Google Scholar
[7]

M. L. Rahman, S. Oka, and Y. Shirai, “Hybrid power generation system using offshore-wind turbine and tidal turbine for power fluctuation compensation (HOT-PC),” IEEE Transactions on Sustainable Energy, vol. 1, no. 2, pp. 92–98, Jul. 2010.
Crossref Google Scholar
[8]

Z. Z. Zhang, Y. Zhang, Q. Huang, and W. J. Lee, “Market-oriented optimal dispatching strategy for a wind farm with a multiple stage hybrid energy storage system,” CSEE Journal of Power and Energy Systems, vol. 4, no. 4, pp. 417–424, Dec. 2018.
Crossref Google Scholar
[9]

J. N. Yu, Y. M. Tang, K. Y. Chau, R. Nazar, S. Ali, and W. Iqbal, “Role of solar-based renewable energy in mitigating CO2 emissions: evidence from quantile-on-quantile estimation,” Renewable Energy, vol. 182, pp. 216–226, Jan. 2022.
Crossref Google Scholar
[10]

J. X. Yang, S. Zhang, Y. Xiang, J. C. Liu, J. Y. Liu, X. Y. Han, and F. Teng, “LSTM auto-encoder based representative scenario generation method for hybrid hydro-PV power system,” IET Generation, Transmission & Distribution, vol. 14, no. 24, pp. 5935–5943, Dec. 2020.
Crossref Google Scholar
[11]

S. Makhdoomi and A. Askarzadeh, “Daily performance optimization of a grid-connected hybrid system composed of photovoltaic and pumped hydro storage (PV/PHS),” Renewable Energy, vol. 159, pp. 272–285, Oct. 2020.
Crossref Google Scholar
[12]

T. Ma, H. X. Yang, L. Lu, and J. Q. Peng, “Pumped storage-based standalone photovoltaic power generation system: modeling and techno-economic optimization,” Applied Energy, vol. 137, pp. 649–659, Jan. 2015.
Crossref Google Scholar
[13]

S. Ali, R. A. Stewart, and O. Sahin, “Drivers and barriers to the deployment of pumped hydro energy storage applications: systematic literature review,” Cleaner Engineering and Technology, vol. 5, pp. 100281, Dec. 2021.
Crossref Google Scholar
[14]

C. Li, D. Q. Zhou, H. Wang, H. B. Cheng, and D. D. Li, “Feasibility assessment of a hybrid PV/diesel/battery power system for a housing estate in the severe cold zone-A case study of Harbin, China,” Energy, vol. 185, pp. 671–681, Oct. 2019.
Crossref Google Scholar
[15]

F. Bilgili, D. B. Lorente, S. Kuşkaya, F. Ünlü, P. Gençoğlu, and P. Rosha, “The role of hydropower energy in the level of CO2 emissions: an application of continuous wavelet transform,” Renewable Energy, vol. 178, pp. 283–294, Nov. 2021.
Crossref Google Scholar
[16]

Y. S. Zhang, C. Ma, Y. Yang, X. L. Pang, L. Liu, and J. J. Lian, “Study on short-term optimal operation of cascade hydro-photovoltaic hybrid systems,” Applied Energy, vol. 291, pp. 116828, Jun. 2021.
Crossref Google Scholar
[17]

F. L. Zhu, P. A. Zhou, Y. M. Sun, B. Xu, Y. F. Ma, W. F. Liu, D. C. Zhang, and J. M. Dawa, “A coordinated optimization framework for long-term complementary operation of a large-scale hydro-photovoltaic hybrid system: nonlinear modeling, multi-objective optimization and robust decision-making,” Energy Conversion and Management, vol. 226, pp. 113543, Dec. 2020.
Crossref Google Scholar
[18]

W. Fang, Q. Huang, S. Z. Huang, J. Yang, E. H. Meng, and Y. Y. Li, “Optimal sizing of utility-scale photovoltaic power generation complementarily operating with hydropower: a case study of the world’s largest hydro-photovoltaic plant,” Energy Conversion and Management, vol. 136, pp. 161–172, Mar. 2017.
Crossref Google Scholar
[19]

K. P. Wong and Y. W. Wong, “Hybrid genetic/simulated annealing approach to short-term multiple-fuel-constrained generation scheduling,” IEEE Transactions on Power Systems, vol. 12, no. 2, pp. 776–784, May 1997.
Crossref Google Scholar
[20]

M. Petrollese, L. Valverde, D. Cocco, G. Cau, and J. Guerra, “Real-time integration of optimal generation scheduling with MPC for the energy management of a renewable hydrogen-based microgrid,” Applied Energy, vol. 166, pp. 96–106, Mar. 2016.
Crossref Google Scholar
[21]

Z. W. Zhang, C. F. Wang, H. C. Lv, F. Q. Liu, H. Z. Sheng, and M. Yang, “Day-ahead optimal dispatch for integrated energy system considering power-to-gas and dynamic pipeline networks,” IEEE Transactions on Industry Applications, vol. 57, no. 4, pp. 3317–3328, Jul./Aug. 2021.
Crossref Google Scholar
[22]

W. Hu, H. X. Zhang, Y. Dong, Y. T. Wang, L. Dong, and M. Xiao, “Short-term optimal operation of hydro-wind-solar hybrid system with improved generative adversarial networks,” Applied Energy, vol. 250, pp. 389–403, Sep. 2019.
Crossref Google Scholar
[23]

B. Ming, P. Liu, S. L. Guo, L. Cheng, Y. L. Zhou, S. D. Gao, and H. Li, “Robust hydroelectric unit commitment considering integration of large-scale photovoltaic power: a case study in China,” Applied Energy, vol. 228, pp. 1341–1352, Oct. 2018.
Crossref Google Scholar
[24]

A. Cerejo, S. J. P. S. Mariano, P. M. S. Carvalho, and M. R. A. Calado, “Hydro-wind optimal operation for joint bidding in day-ahead market: storage efficiency and impact of wind forecasting uncertainty,” Journal of Modern Power Systems and Clean Energy, vol. 8, no. 1, pp. 142–149, Jan. 2020.
Crossref Google Scholar
[25]

X. B. Wang, J. X. Chang, X. J. Meng, and Y. M. Wang, “Short-term hydro-thermal-wind-photovoltaic complementary operation of interconnected power systems,” Applied Energy, vol. 229, pp. 945–962, Nov. 2018.
Crossref Google Scholar
[26]
B. Xiao, J. Zheng, G. G. Yan, Y. G. Shi, M. X. Jiao, Y. Wang, L. Dong, M. C. Wang, and H. Z. Yang, “Short-term optimized operation of multi-energy power system based on complementary characteristics of power sources,” in Proceedings of 2019 IEEE Innovative Smart Grid Technologies - Asia, 2019, pp. 3770–3775.
Crossref
[27]
J. X. Yang, J. C. Liu, and S. Zhang, “Optimization for short-term operation of hybrid hydro-PV power system based on NSGA-II,” in Proceedings of 2020 IEEE 4th Conference on Energy Internet and Energy System Integration, 2020, pp. 2267–2271.
Crossref
[28]

S. Makhdoomi and A. Askarzadeh, “Optimizing operation of a photovoltaic/diesel generator hybrid energy system with pumped hydro storage by a modified crow search algorithm,” Journal of Energy Storage, vol. 27, pp. 101040, Feb. 2020.
Crossref Google Scholar
[29]

L. Lu, W. L. Yuan, C. G. Su, P. L. Wang, C. T. Cheng, D. H. Yan, and Z. N. Wu, “Optimization model for the short-term joint operation of a grid-connected wind-photovoltaic-hydro hybrid energy system with cascade hydropower plants,” Energy Conversion and Management, vol. 236, pp. 114055, May 2021.
Crossref Google Scholar
[30]

C. A. Hans, P. Sopasakis, J. Raisch, C. Reincke-Collon, and P. Patrinos, “Risk-averse model predictive operation control of islanded microgrids,” IEEE Transactions on Control Systems Technology, vol. 28, no. 6, pp. 2136–2151, Nov. 2020.
Crossref Google Scholar
[31]

J. Sachs and O. Sawodny, “A two-stage model predictive control strategy for economic diesel-PV-battery island microgrid operation in rural areas,” IEEE Transactions on Sustainable Energy, vol. 7, no. 3, pp. 903–913, Jul. 2016.
Crossref Google Scholar
[32]

W. L. Yuan, X. Q. Wang, C. G. Su, C. T. Cheng, Z. Liu, and Z. N. Wu, “Stochastic optimization model for the short-term joint operation of photovoltaic power and hydropower plants based on chance-constrained programming,” Energy, vol. 222, pp. 119996, May 2021.
Crossref Google Scholar
[33]

Q. Li, Y. B. Qiu, H. Q. Yang, Y. Xu, W. R. Chen, and P. Wang, “Stability-constrained two-stage robust optimization for integrated hydrogen hybrid energy system,” CSEE Journal of Power and Energy Systems, vol. 7, no. 1, pp. 162–171, Jan. 2021.
Crossref Google Scholar
[34]

L. Zhou, A. Swain, and A. Ukil, “Reinforcement learning controllers for enhancement of low voltage ride through capability in hybrid power systems,” IEEE Transactions on Industrial Informatics, vol. 16, no. 8, pp. 5023–5031, Aug. 2020.
Crossref Google Scholar
[35]

Z. M. Yan and Y. Xu, “Real-time optimal power flow: a Lagrangian based deep reinforcement learning approach,” IEEE Transactions on Power Systems, vol. 35, no. 4, pp. 3270–3273, Jul. 2020.
Crossref Google Scholar
[36]

Y. C. Liang, C. L. Guo, Z. H. Ding, and H. C. Hua, “Agent-based modeling in electricity market using deep deterministic policy gradient algorithm,” IEEE Transactions on Power Systems, vol. 35, no. 6, pp. 4180–4192, Nov. 2020.
Crossref Google Scholar
[37]

L. Lin, X. Guan, Y. Peng, N. Wang, S. Maharjan, and T. Ohtsuki, “Deep reinforcement learning for economic dispatch of virtual power plant in internet of energy,” IEEE Internet of Things Journal, vol. 7, no. 7, pp. 6288–6301, Jul. 2020.
Crossref Google Scholar
[38]

X. Zhang, Y. B. Liu, J. J. Duan, G. Qiu, T. J. Liu, and J. Y. Liu, “DDPG-based multi-agent framework for SVC tuning in urban power grid with renewable energy resources,” IEEE Transactions on Power Systems, vol. 36, no. 6, pp. 5465–5475, Nov. 2021.
Crossref Google Scholar
[39]

L. Yu, W. W. Xie, D. Xie, Y. L. Zou, D. Y. Zhang, Z. X. Sun, L. H. Zhang, Y. Zhang, and T. Jiang, “Deep reinforcement learning for smart home energy management,” IEEE Internet of Things Journal, vol. 7, no. 4, pp. 2751–2762, Apr. 2020.
Crossref Google Scholar
[40]

Y. K. Liu, D. X. Zhang, and H. B. Gooi, “Optimization strategy based on deep reinforcement learning for home energy management,” CSEE Journal of Power and Energy Systems, vol. 6, no. 3, pp. 572–582, Sep. 2020.
Google Scholar
[41]

S. Lee and D. H. Choi, “Federated reinforcement learning for energy management of multiple smart homes with distributed energy resources,” IEEE Transactions on Industrial Informatics, vol. 18, no. 1, pp. 488–497, Jan. 2022.
Crossref Google Scholar
[42]

G. Z. Zhang, W. H. Hu, D. Cao, W. Liu, R. Huang, Q. Huang, Z. Chen, and F. Blaabjerg, “Data-driven optimal energy management for a wind-solar-diesel-battery-reverse osmosis hybrid energy system using a deep reinforcement learning approach,” Energy Conversion and Management, vol. 227, pp. 113608, Jan. 2021.
Crossref Google Scholar
[43]

X. D. Wang, Y. B. Liu, J. B. Zhao, C. Liu, J. Y. Liu, and J. Y. Yan, “Surrogate model enabled deep reinforcement learning for hybrid energy community operation,” Applied Energy, vol. 289, pp. 116722, May 2021.
Crossref Google Scholar
CSEE Journal of Power and Energy Systems
Volume 9 Issue 3,
May 2023
Pages 846-858
DOI: 10.17775/CSEEJPES.2021.07210
Cite this article:
Yang J, Liu J, Xiang Y, et al. Data-driven Optimal Dynamic Dispatch for Hydro-PV-PHS Integrated Power Systems Using Deep Reinforcement Learning Approach. CSEE Journal of Power and Energy Systems, 2023, 9(3): 846-858. https://doi.org/10.17775/CSEEJPES.2021.07210

384

Views

5

Downloads

0

Crossref

4

Web of Science

10

Scopus

1

CSCD

Altmetrics
Received: 25 September 2021
Revised: 24 November 2021
Accepted: 30 December 2021
Published: 18 August 2022
© 2021 CSEE.

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Return