Rethinking Urban Water Network Design: A Reinforcement Learning Framework for Long-Term Flexible Planning

Lydia Tsiami (), Christos Makropoulos and Dragan Savic
Additional contact information
Lydia Tsiami: National Technical University of Athens
Christos Makropoulos: National Technical University of Athens
Dragan Savic: KWR Water Research Institute

Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), 2025, vol. 39, issue 13, No 20, 7155-7174

Abstract: Abstract Effectively planning the design of water distribution networks (WDNs) for their whole lifecycle is a complex task for water utilities due to the dynamic nature of WDNs, their long planning horizons, and the deep uncertainty that characterises key design parameters such as future water demand and population growth. Existing flexible design methods, which attempt to address these challenges, rely on static heuristic approaches and predefined decision pathways, requiring re-optimisation whenever new information becomes available. As additional scenarios are introduced, these methods also suffer from exponential increase in complexity, limiting their ability to adapt to emerging information and efficiently explore a wide range of future possibilities. In this work, we introduce a deep reinforcement learning (DRL) framework for the flexible, long-term design of WDNs. By formulating the least-cost staged design problem as a Markov Decision Process and training an agent using Proximal Policy Optimisation, our approach learns cost-effective, sequential interventions across multiple construction stages and future scenarios without relying on predefined decision trees. We evaluate our method on a modified New York Tunnels benchmark across three design tasks, ranging from static single-stage to flexible multi-stage design. Our results show that the DRL agent performs comparably to state-of-the-art heuristics for static and staged deterministic tasks. In the flexible design task, it autonomously devised adaptive strategies, clustered similar scenarios, and maintained high sample efficiency as the number of stages and scenarios increased. These findings highlight DRL as a promising alternative for the lifecycle design of WDNs, establishing a new paradigm for long-term water network planning under deep uncertainty.

Keywords: Reinforcement Learning; Deep Uncertainty; Staged Optimisation; Markov Decision Process; Proximal Policy Optimisation; Flexible Design (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11269-025-04290-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:waterr:v:39:y:2025:i:13:d:10.1007_s11269-025-04290-8

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11269

DOI: 10.1007/s11269-025-04290-8

Access Statistics for this article

Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA) is currently edited by G. Tsakiris

More articles in Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA) from Springer, European Water Resources Association (EWRA)
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().