Home CSEE Journal of Power and Energy Systems Article
PDF (1.9 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Open Access

Autonomous Charging of Electric Vehicle Fleets to Enhance Renewable Generation Dispatchability

Reza Bayani, Saeed D. Manshadi (), Guangyi Liu, Yawei Wang, Renchang Dai
San Diego State University, San Diego, CA, 92182 USA
GEIRI North America, San Jose, CA, 95134 USA
Show Author Information

Abstract

A total of 19% of generation capacity in California is offered by PV units and over some months, more than 10% of this energy is curtailed. In this research, a novel approach to reducing renewable generation curtailment and increasing system flexibility by means of electric vehicles’ charging coordination is presented. The presented problem is a sequential decision making process, and is solved by a fitted Q-iteration algorithm which unlike other reinforcement learning methods, needs fewer episodes of learning. Three case studies are presented to validate the effectiveness of the proposed approach. These cases include aggregator load following, ramp service and utilization of non-deterministic PV generation. The results suggest that through this framework, EVs successfully learn how to adjust their charging schedule in stochastic scenarios where their trip times, as well as solar power generation are unknown beforehand.

Keywords

Reinforcement learningelectric vehiclecurtailment reductiondispatchabilityscheduling

References

[1]
E. I. Administration. (2020, May). Annual energy outlook 2020 with projections to 2050. [Online]. Available: https://www.eia.gov/outlooks/aeo/.
[2]
H. Kikusato, Y. Fujimoto, S. I. Hanada, D. Isogawa, S. Yoshizawa, H. Ohashi, and Y. Hayashi, “Electric vehicle charging management using auction mechanism for reducing PV curtailment in distribution systems,” IEEE Transactions on Sustainable Energy, vol. 11, no. 3, pp. 1394–1403, Jul. 2020.
Crossref Google Scholar
[3]
R. Golden and B. Paulos, “Curtailment of renewable energy in California and beyond,” The Electricity Journal, vol. 28, no. 6, pp. 36–50, Jul. 2015.
Crossref Google Scholar
[4]
CAISO. (2020, May). California independent system operator website. [Online]. Available: https://www.caiso.com/.
[5]
J. Cochran, P. Denholm, B. Speer, and M. Miller, “Grid integration and the carrying capacity of the U.S. grid to incorporate variable renewable energy,” National Renewable Energy Lab. (NREL), Golden, CO (United States), Tech. Rep. NREL/TP-6A20-62607, Apr. 2015.
Crossref
[6]
J. Cochran, L. Bird, J. Heeter, and D. J. Arent, “Integrating variable renewable energy in electric power markets. Best practices from international experience,” National Renewable Energy Lab. (NREL), Golden, CO (United States), Tech. Rep. NREL/TP-6A00-53732, Apr. 2012.
Crossref
[7]
L. Bird, J. Cochran, and X. Wang, “Wind and solar energy curtailment: Experience and practices in the United States,” National Renewable Energy Lab. (NREL), Golden, CO (United States), Tech. Rep. NREL/TP-6A20–60983, Mar. 2014.
Crossref
[8]
C. B. Li, H. Q. Shi, Y. J. Cao, J. H. Wang, Y. H. Kuang, Y. Tan, and J. Wei, “Comprehensive review of renewable energy curtailment and avoidance: a specific example in China,” Renewable and Sustainable Energy Reviews, vol. 41, pp. 1067–1079, Jan. 2015.
Crossref Google Scholar
[9]
A. Henriot, “Economic curtailment of intermittent renewable energy sources,” Energy Economics, vol. 49, pp. 370–379, May 2015.
Crossref Google Scholar
[10]
E. Ela, “Using economics to determine the efficient curtailment of wind energy,” National Renewable Energy Lab. (NREL), Golden, CO (United States), Tech. Rep. NREL/TP-550-45071, Feb. 2009.
Crossref
[11]
J. Zou, S. Rahman, and X. Lai, “Mitigation of wind output curtailment by coordinating with pumped storage and increasing transmission capacity,” in 2015 IEEE Power & Energy Society General Meeting, Denver, 2015, pp. 1–5.
Crossref
[12]
B. Cleary, A. Duffy, A. OConnor, M. Conlon, and V. Fthenakis, “Assessing the economic benefits of compressed air energy storage for mitigating wind curtailment,” IEEE Transactions on Sustainable Energy, vol. 6, no. 3, pp. 1021–1028, Jul. 2015.
Crossref Google Scholar
[13]
M. Moradzadeh, B. Zwaenepoel, J. Van de Vyver, and L. Vandevelde, “Congestion-induced wind curtailment mitigation using energy storage,” in 2014 IEEE International Energy Conference (ENERGYCON), Cavtat, 2014, pp. 572–576.
Crossref
[14]
P. Denholm and T. Mai, “Timescales of energy storage needed for reducing renewable energy curtailment,” Renewable Energy, vol. 130, pp. 388–399, Jan. 2019.
Crossref Google Scholar
[15]
M. A. Hozouri, A. Abbaspour, M. Fotuhi-Firuzabad, and M. Moeini-Aghtaie, “On the use of pumped storage for wind energy maximization in transmission-constrained power systems,” IEEE Transactions on Power Systems, vol. 30, no. 2, pp. 1017–1025, Mar. 2015.
Crossref Google Scholar
[16]
D. X. Zhang, X. Q. Han, and C. Y. Deng, “Review on the research and practice of deep learning and reinforcement learning in smart grids,” CSEE Journal of Power and Energy Systems, vol. 4, no. 3, pp. 362–370, Sep. 2018.
Crossref Google Scholar
[17]
E. Mocanu, D. C. Mocanu, P. H. Nguyen, A. Liotta, M. E. Webber, M. Gibescu, and J. G. Slootweg, “On-line building energy optimization using deep reinforcement learning,” IEEE Transactions on Smart Grid, vol. 10, no. 4, pp. 3698–3708, Jul. 2019.
Crossref Google Scholar
[18]
J. K. Szinai, C. J. R. Sheppard, N. Abhyankar, and A. R. Gopal, “Reduced grid operating costs and renewable energy curtailment with electric vehicle charge management,” Energy Policy, vol. 136, pp. 111051, Jan. 2020.
Crossref Google Scholar
[19]
A. Chiş, J. Lundén, and V. Koivunen, “Reinforcement learning-based plug-in electric vehicle charging with forecasted price,” IEEE Transactions on Vehicular Technology, vol. 66, no. 5, pp. 3674–3684, May 2017.
Google Scholar
[20]
T. Chen and W. C. Su, “Indirect customer-to-customer energy trading with reinforcement learning,” IEEE Transactions on Smart Grid, vol. 10, no. 4, pp. 4338–4348, Jul. 2019.
Crossref Google Scholar
[21]
B. J. Claessens, P. Vrancx, and F. Ruelens, “Convolutional neural networks for automatic state-time feature extraction in reinforcement learning applied to residential load control,” IEEE Transactions on Smart Grid, vol. 9, no. 4, pp. 3259–3269, Jul. 2018.
Crossref Google Scholar
[22]
C. J. C. H. Watkins and P. Dayan, “Q-learning,” Machine Learning, vol. 8, no. 3–4, pp. 279–292, May 1992.
Crossref Google Scholar
[23]
D. Ernst, P. Geurts, and L. Wehenkel, “Tree-based batch mode reinforcement learning,” The Journal of Machine Learning Research, vol. 6, pp. 503–556, Dec. 2005.
Google Scholar
[24]
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, Feb. 2015.
Crossref Google Scholar
[25]
Z. D. Zhang, D. X. Zhang, and R. C. Qiu, “Deep reinforcement learning for power system applications: An overview,” CSEE Journal of Power and Energy Systems, vol. 6, no. 1, pp. 213–225, Mar. 2020.
Google Scholar
[26]
F. L. Da Silva, C. E. H. Nishida, D. M. Roijers, and A. H. R. Costa, “Coordination of electric vehicle charging through multiagent reinforcement learning,” IEEE Transactions on Smart Grid, vol. 11, no. 3, pp. 2347–2356, May 2020.
Crossref Google Scholar
[27]
T. Qian, C. C. Shao, X. L. Wang, and M. Shahidehpour, “Deep reinforcement learning for EV charging navigation by coordinating smart grid and intelligent transportation system,” IEEE Transactions on Smart Grid, vol. 11, no. 2, pp. 1714–1723, Mar. 2020.
Crossref Google Scholar
[28]
H. Ko, S. Pack, and V. C. M. Leung, “Mobility-aware vehicle-to-grid control algorithm in microgrids,” IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 7, pp. 2165–2174, Jul. 2018.
Crossref Google Scholar
[29]
H. P. Li, Z. Q. Wan, and H. B. He, “Constrained EV charging scheduling based on safe deep reinforcement learning,” IEEE Transactions on Smart Grid, vol. 11, no. 3, pp. 2427–2439, May 2020.
Crossref Google Scholar
[30]
Z. Q. Wan, H. P. Li, H. B. He, and D. Prokhorov, “Model-free real-time EV charging scheduling based on deep reinforcement learning,” IEEE Transactions on Smart Grid, vol. 10, no. 5, pp. 5246–5257, Sep. 2019.
Crossref Google Scholar
[31]
M. Shin, D. H. Choi, and J. Kim, “Cooperative management for PV/ESS-enabled electric vehicle charging stations: A multiagent deep reinforcement learning approach,” IEEE Transactions on Industrial Informatics, vol. 16, no. 5, pp. 3493–3503, May 2020.
Crossref Google Scholar
[32]
Z. Wei, Y. Li, and L. Cai, “Electric vehicle charging scheme for a park-and-charge system considering battery degradation costs,” IEEE Transactions on Intelligent Vehicles, vol. 3, no. 3, pp. 361–373, Sep. 2018.
Crossref Google Scholar
[33]
S. Vandael, B. Claessens, D. Ernst, T. Holvoet, and G. Deconinck, “Reinforcement learning of heuristic EV fleet charging in a day-ahead electricity market,” IEEE Transactions on Smart Grid, vol. 6, no. 4, pp. 1795–1805, Jul. 2015.
Crossref Google Scholar
[34]
N. Sadeghianpourhamami, J. Deleu, and C. Develder, “Definition and evaluation of model-free coordination of electrical vehicle charging with reinforcement learning,” IEEE Transactions on Smart Grid, vol. 11, no. 1, pp. 203–214, Jan. 2020.
Crossref Google Scholar
[35]
B. V. Mbuwir, F. Ruelens, F. Spiessens, and G. Deconinck, “Battery energy management in a microgrid using batch reinforcement learning,” Energies, vol. 10, no. 11, pp. 1846, Nov. 2017.
Crossref Google Scholar
[36]
S. D. Manshadi, M. E. Khodayar, K. Abdelghany, and H. Üster, “Wireless charging of electric vehicles in electricity and transportation networks,” IEEE Transactions on Smart Grid, vol. 9, no. 5, pp. 4503–4512, Sep. 2018.
Crossref Google Scholar
[37]
S. D. Manshadi and M. E. Khodayar, “Strategic behavior of in-motion wireless charging aggregators in the electricity and transportation networks,” IEEE Transactions on Vehicular Technology, vol. 69, no. 12, pp. 14780–14792, Dec. 2020.
Crossref Google Scholar
[38]
X. N. Wang, J. H. Wang, and J. Z. Liu, “Vehicle to grid frequency regulation capacity optimal scheduling for battery swapping station using deep Q-network,” IEEE Transactions on Industrial Informatics, vol. 17, no. 2, pp. 1341–1351, Feb. 2021.
Crossref Google Scholar
[39]
E. M. P. Walraven and M. T. J. Spaan, “Planning under uncertainty for aggregated electric vehicle charging with renewable energy supply,” in Proceedings of the Twenty-second European Conference on Artificial Intelligence, 2016, pp. 904–912.
CSEE Journal of Power and Energy Systems
Volume 8 Issue 3,
May 2022
Pages 669-681
DOI: 10.17775/CSEEJPES.2020.04000
Cite this article:
Bayani R, Manshadi SD, Liu G, et al. Autonomous Charging of Electric Vehicle Fleets to Enhance Renewable Generation Dispatchability. CSEE Journal of Power and Energy Systems, 2022, 8(3): 669-681. https://doi.org/10.17775/CSEEJPES.2020.04000
Metrics & Citations  
Article History
Copyright
Return