Deep reinforcement learning-based strategy for maximizing returns from renewable energy and energy storage systems in multi-electricity markets

Javier Cardo-Miota, Hector Beltran, Emilio Pérez, Shafi Khadem and Mohamed Bahloul

Applied Energy, 2025, vol. 388, issue C, No S0306261925002910

Abstract: The integration of Renewable Energy Sources (RES) with Energy Storage Systems (ESS) presents challenges and opportunities in optimizing their participation in electricity markets. This study introduces a novel approach that leverages Deep Reinforcement Learning (RL) algorithms to develop optimal bidding strategies for collocated RES with Battery ESS (BESS) configurations, enabling multi-market participation in both energy and ancillary services (AS) markets. The proposed method uses a Markov Decision Process (MDP) framework to manage BESS utilization dynamically, considering market conditions and technical constraints. As an RL agent, the actor–critic approach known as the Twin Delayed Deep Deterministic (TD3) Policy Gradient algorithm is implemented. A data-driven training process facilitates model learning while minimizing the required training dataset and time. Focused on the Irish context, the case study involves participation in both the day-ahead energy market and reserve services for frequency droop curve response of the DS3 Programme. Historical data from a 7 MW solar PV plant and a 1 MWh BESS are utilized to evaluate the performance. The RL agent dynamically adapts to market dynamics and system constraints, achieving substantial economic benefits compared to benchmark strategies, with an additional 8271€, 166,738€, and 11,369€, respectively.

Keywords: Deep reinforcement learning; Markov decision process; Bidding strategy; Battery energy storage system management; Renewable energy systems; Multi-electricity markets participation (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261925002910
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:388:y:2025:i:c:s0306261925002910

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2025.125561

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().