Deep Reinforcement Learning-Based Rebalancing Policies for Profit Maximization of Relay Nodes in Payment Channel Networks

Nikolaos Papadis () and Leandros Tassiulas ()
Additional contact information
Nikolaos Papadis: Nokia Bell Labs
Leandros Tassiulas: Yale University

A chapter in Mathematical Research for Blockchain Economy, 2023, pp 1-27 from Springer

Abstract: Abstract Payment channel networks (PCNs) are a layer-2 blockchain scalability solution, with its main entity, the payment channel, enabling transactions between pairs of nodes “off-chain,” thus reducing the burden on the layer-1 network. Nodes with multiple channels can serve as relays for multihop payments by providing their liquidity and withholding part of the payment amount as a fee. Relay nodes might after a while end up with one or more unbalanced channels, and thus need to trigger a rebalancing operation. In this paper, we study how a relay node can maximize its profits from fees by using the rebalancing method of submarine swaps. We introduce a stochastic model to capture the dynamics of a relay node observing random transaction arrivals and performing occasional rebalancing operations, and express the system evolution as a Markov Decision Process. We formulate the problem of the maximization of the node’s fortune over time over all rebalancing policies, and approximate the optimal solution by designing a Deep Reinforcement Learning (DRL)-based rebalancing policy. We build a discrete event simulator of the system and use it to demonstrate the DRL policy’s superior performance under most conditions by conducting a comparative study of different policies and parameterizations. Our work is the first to introduce DRL for liquidity management in the complex world of PCNs.

Keywords: Payment channel networks; Lightning Network; Rebalancing; Submarine swaps; Deep reinforcement learning; Soft actor-critic; Optimization; Discrete event simulation; Control (search for similar items in EconPapers)
Date: 2023
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:lnopch:978-3-031-48731-6_1

Ordering information: This item can be ordered from
http://www.springer.com/9783031487316

DOI: 10.1007/978-3-031-48731-6_1

Access Statistics for this chapter

More chapters in Lecture Notes in Operations Research from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().