Application of Deep Reinforcement Learning to At-the-Money S&P 500 Options Hedging

Zofia Bracha, Jakub Michańków and Paweł Sakowski ()
Additional contact information
Zofia Bracha: Faculty of Economic Sciences, University of Warsaw
Jakub Michańków: TripleSun, Krakow
Paweł Sakowski: Faculty of Economic Sciences, University of Warsaw

No 2025-25, Working Papers from Faculty of Economic Sciences, University of Warsaw

Abstract: This paper explores the application of deep Q-learning to hedging at-the-money options on the S&P 500 index. We develop an agent based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, trained to simulate hedging decisions without making explicit model assumptions on price dynamics. The agent was trained on historical intraday prices of S&P 500 call options across years 2004 to 2024, using a single time series of six predictor variables: option price, underlying asset price, moneyness, time to maturity, realized volatility, and current hedge position. A walk-forward procedure was applied for training, which lead to nearly 17 years of out-of-sample evaluation. The performance of the deep reinforcement learning (DRL) agent is benchmarked against the Black–Scholes delta hedging strategy over the same time period. We assess both approaches using metrics such as annualized return, volatility, information ratio, and Sharpe ratio. To test models’ adaptability, we performed simulations across varying market conditions and added constraints such as transaction costs and risk-awareness penalties. Our results show that the DRL agent can outperform traditional hedging methods, particularly in volatile or high-cost environments, highlighting its robustness and flexibility in practical trading contexts. While the agent consistently outperforms delta hedging, its performance deteriorates when the risk-awareness parameter is higher. We also observed that the longer the time interval used for volatility estimation, the more stable the results.

Keywords: Deep learning; Reinforcement learning; Double Deep Q-netwoorks; options market; options hedging; deep hedging (search for similar items in EconPapers)
JEL-codes: C14 C4 C45 C53 C58 G13 (search for similar items in EconPapers)
Pages: 36 pages
Date: 2025
New Economics Papers: this item is included in nep-big and nep-cmp
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.wne.uw.edu.pl/download_file/6259/0 First version, 2025 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:war:wpaper:2025-25

Access Statistics for this paper

More papers in Working Papers from Faculty of Economic Sciences, University of Warsaw Contact information at EDIRC.
Bibliographic data for series maintained by Jacek Rapacz ().