A multiagent reinforcement learning framework for off-policy evaluation in two-sided markets

Chengchun Shi, Runzhe Wan, Ge Song, Shikai Luo, Hongtu Zhu and Rui Song

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: The two-sided markets, such as ride-sharing companies, often involve a group of subjects who are making sequential decisions across time and/or location. With the rapid development of smart phones and internet of things, they have substantially transformed the transportation landscape of human beings. In this paper we consider large-scale fleet management in ride-sharing companies that involve multiple units in different areas receiving sequences of products (or treatments) over time. Major technical challenges, such as policy evaluation, arise in those studies because: (i) spatial and temporal proximities induce interference between locations and times, and (ii) the large number of locations results in the curse of dimensionality. To address both challenges simultaneously, we introduce a multiagent reinforcement learning (MARL) framework for carrying policy evaluation in these studies. We propose novel estimators for mean outcomes under different products that are consistent despite the high dimensionality of state-action space. The proposed estimator works favorably in simulation experiments. We further illustrate our method using a real dataset obtained from a two-sided marketplace company to evaluate the effects of applying different subsidizing policies. A Python implementation of our proposed method is available in the Supplementary Material and also at https://github.com/RunzheStat/CausalMARL.

Keywords: reinforcement learning; policy evaluation; multiagent system; spatiotemporal studies; DMS-2003637; EP/W014971/1 (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Pages: 22 pages
Date: 2023-12-01
References: View references in EconPapers View complete reference list from CitEc
Citations:

Published in Annals of Applied Statistics, 1, December, 2023, 17(4), pp. 2701 - 2722. ISSN: 1932-6157

Downloads: (external link)
http://eprints.lse.ac.uk/117174/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:117174

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager ().