Parametric study of adaptive reinforcement learning for battery operations in microgrids

Deepak Kumar Panda, Saptarshi Das and Mohammad Abusara

Renewable Energy, 2025, vol. 250, issue C

Abstract: Reinforcement learning (RL) has been increasingly used for efficient energy management systems (EMSs) in microgrids. The battery storage system in the microgrid can be controlled using efficient policies derived from RL. However, little attention has been paid so far to the parametric study, which is a fundamental step for efficient implementation of such RL algorithms. Unlike previous works which focused on the implementation of different RL algorithms, this paper mainly demonstrates the parametric sensitivity study of the RL algorithms. It involves investigating the effects of (1) controllable state discretization, (2) exogenous state discretization, (3) action discretization, (4) exploration and exploitation parameters, and (5) decision intervals. Moreover, the performance of the ε-greedy randomized RL algorithm is compared against the adaptive Q-learning, derived from the adaptive approximate dynamic programming (ADP). In many microgrids utilizing solar energy and battery storage, energy management still relies on manually tuned and inefficient algorithms. This is largely due to the sensitivity of RL algorithm parameters to factors such as the specific EMS problem, environment, action-state discretization, exploration parameter and time step. We show the univariate and multivariate kernel density estimate (KDE) plots to study the RL algorithms’ performance concerning the rewards and variation of the battery state of charge (SoC) and the net power imported from the grid. Overall, the deterministic adaptive RL performs better as compared to the randomized ε-greedy algorithms in terms of rewards and simulation time. Higher discretization levels in the action space affect the convergence rate while lower discretization levels in the state space influence the performance of the algorithm. The proposed parametric analysis can be easily adapted to other EMS in more complex microgrids.

Keywords: Microgrid management; Battery operations; Q-learning; Reinforcement learning convergence; Adaptive RL (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0960148125009127
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:renene:v:250:y:2025:i:c:s0960148125009127

DOI: 10.1016/j.renene.2025.123250

Access Statistics for this article

Renewable Energy is currently edited by Soteris A. Kalogirou and Paul Christodoulides

More articles in Renewable Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().