Multidimensional Parallel Dynamic Programming Algorithm Based on Spark for Large-Scale Hydropower Systems

Yufei Ma, Ping-an Zhong (), Bin Xu (), Feilin Zhu, Yao Xiao and Qingwen Lu
Additional contact information
Yufei Ma: Hohai University
Ping-an Zhong: Hohai University
Bin Xu: Hohai University
Feilin Zhu: Hohai University
Yao Xiao: Hohai University
Qingwen Lu: Hohai University

Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), 2020, vol. 34, issue 11, No 1, 3427-3444

Abstract: Abstract The “curse of dimensionality” is a major problem in dynamic programming (DP) algorithms for large-scale hydropower systems. This study proposes a parallel DP algorithm based on Spark (PDPoS) to alleviate the “curse of dimensionality”. Parallel computing experiments are formulated by varying the number of reservoirs, the number of discrete water levels and the number of CPU cores to analyze the quality and efficiency of PDPoS. The methodologies were applied to a cascade reservoir system made up of eight reservoirs in the Yuanshui River Basin in China. The results are as follows. (1) The number of discrete water levels is the dominant factor in the solution quality, while the number of reservoirs is the dominant factor in the solving efficiency. (2) The runtime of PDPoS is markedly affected by the calculational scale (determined by the number of reservoirs and discrete water levels), and the relationship between the number of CPU cores and the runtime is triphasic with increasing calculational scale. (3) The larger the calculational scale is, the better the parallel performance (i.e., the parallel speedup and parallel efficiency). The proposed PDPoS method has strong generality, high parallel performance, and high practical value.

Keywords: Hydropower operation; Curse of dimensionality; Multidimensional dynamic programming; Parallel computing; Spark (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
http://link.springer.com/10.1007/s11269-020-02566-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:waterr:v:34:y:2020:i:11:d:10.1007_s11269-020-02566-9

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11269

DOI: 10.1007/s11269-020-02566-9

Access Statistics for this article

Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA) is currently edited by G. Tsakiris

More articles in Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA) from Springer, European Water Resources Association (EWRA)
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().