Deep Reinforcement Learning for Cascaded Hydropower Reservoirs Considering Inflow Forecasts

Wei Xu (), Xiaoli Zhang, Anbang Peng and Yue Liang
Additional contact information
Wei Xu: Chongqing Jiaotong University
Xiaoli Zhang: North China University of Water Resources and Electric Power
Anbang Peng: Nanjing Hydraulic Research Institute
Yue Liang: Chongqing Jiaotong University

Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), 2020, vol. 34, issue 9, No 22, 3003-3018

Abstract: Abstract This paper develops a deep reinforcement learning (DRL) framework for intelligence operation of cascaded hydropower reservoirs considering inflow forecasts, in which two key problems of large discrete action spaces and uncertainty of inflow forecasts are addressed. In this study, a DRL framework is first developed based on a newly defined knowledge sample form and a deep Q-network (DQN). Then, an aggregation-disaggregation model is used to reduce the multi-dimension spaces of state and action for cascaded reservoirs. Following, three DRL models are developed respectively to evaluate the performance of the newly defined decision value functions and modified decision action selection approach. In this paper, the DRL methodologies are tested on China’s Hun River cascade hydropower reservoirs system. The results show that the aggregation-disaggregation model can effectively reduce the dimensions of state and action, which also makes the model structure simpler and has higher learning efficiency. The Bayesian theory in the decision action selection approach is useful to address the uncertainty of inflow forecasts, which can improve the performance to reduce spillages during the wet season. The proposed DRL models outperform the comparison models (i.e., stochastic dynamic programming) in terms of annual hydropower generation and system reliability. This study suggests that the DRL has the potential to be implemented in practice to derive optimal operation strategies.

Keywords: Aggregation-disaggregation model; Bayesian theory; Cascaded hydropower reservoirs; Deep reinforcement learning; Large discrete action space (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
http://link.springer.com/10.1007/s11269-020-02600-w Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:waterr:v:34:y:2020:i:9:d:10.1007_s11269-020-02600-w

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11269

DOI: 10.1007/s11269-020-02600-w

Access Statistics for this article

Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA) is currently edited by G. Tsakiris

More articles in Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA) from Springer, European Water Resources Association (EWRA)
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().