Deep Reinforcement Learning in Agent Based Financial Market Simulation

Iwao Maeda, David deGraw, Michiharu Kitano, Hiroyasu Matsushima, Hiroki Sakaji, Kiyoshi Izumi and Atsuo Kato
Additional contact information
Iwao Maeda: Department of Systems Innovation, School of Engineering, The University of Tokyo, Tokyo 113-8654, Japan
David deGraw: Daiwa Securities Co. Ltd., Tokyo 100-0005, Japan
Michiharu Kitano: Daiwa Institute of Research Ltd., Tokyo 135-8460, Japan
Hiroyasu Matsushima: Department of Systems Innovation, School of Engineering, The University of Tokyo, Tokyo 113-8654, Japan
Hiroki Sakaji: Department of Systems Innovation, School of Engineering, The University of Tokyo, Tokyo 113-8654, Japan
Kiyoshi Izumi: Department of Systems Innovation, School of Engineering, The University of Tokyo, Tokyo 113-8654, Japan
Atsuo Kato: Daiwa Institute of Research Ltd., Tokyo 135-8460, Japan

JRFM, 2020, vol. 13, issue 4, 1-17

Abstract: Prediction of financial market data with deep learning models has achieved some level of recent success. However, historical financial data suffer from an unknowable state space, limited observations, and the inability to model the impact of your own actions on the market can often be prohibitive when trying to find investment strategies using deep reinforcement learning. One way to overcome these limitations is to augment real market data with agent based artificial market simulation. Artificial market simulations designed to reproduce realistic market features may be used to create unobserved market states, to model the impact of your own investment actions on the market itself, and train models with as much data as necessary. In this study we propose a framework for training deep reinforcement learning models in agent based artificial price-order-book simulations that yield non-trivial policies under diverse conditions with market impact. Our simulations confirm that the proposed deep reinforcement learning model with unique task-specific reward function was able to learn a robust investment strategy with an attractive risk-return profile.

Keywords: deep reinforcement learning; financial market simulation; agent based simulation (search for similar items in EconPapers)
JEL-codes: C E F2 F3 G (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
https://www.mdpi.com/1911-8074/13/4/71/pdf (application/pdf)
https://www.mdpi.com/1911-8074/13/4/71/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jjrfmx:v:13:y:2020:i:4:p:71-:d:344491

Access Statistics for this article

JRFM is currently edited by Ms. Chelthy Cheng

More articles in JRFM from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().