Policy Learning with Adaptively Collected Data

Ruohan Zhan, Zhimei Ren, Susan Athey and Zhengyuan Zhou
Additional contact information
Ruohan Zhan: Institute for Computational and Mathematical Engineering, Stanford University
Zhimei Ren: Stanford University
Zhengyuan Zhou: New York University

Research Papers from Stanford University, Graduate School of Business

Abstract: Learning optimal policies from historical data enables the gains from personalization to be realized in a wide variety of applications. The growing policy learning literature focuses on a setting where the treatment assignment policy does not adapt to the data. However, adaptive data collection is becoming more common in practice, from two primary sources: 1) data collected from adaptive experiments that are designed to improve inferential efficiency; 2) data collected from production systems that are adaptively evolving an operational policy to improve performance over time (e.g. contextual bandits). In this paper, we aim to address the challenge of learning the optimal policy with adaptively collected data and provide one of the first theoretical inquiries into this problem. We propose an algorithm based on generalized augmented inverse propensity weighted estimators and establish its finite-sample regret bound. We complement this regret upper bound with a lower bound that characterizes the fundamental difficulty of policy learning with adaptive data. Finally, we demonstrate our algorithm’s effectiveness using both synthetic data and public benchmark datasets.

Date: 2021-05
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.gsb.stanford.edu/faculty-research/work ... ively-collected-data

Related works:
Journal Article: Policy Learning with Adaptively Collected Data (2024)
Working Paper: Policy Learning with Adaptively Collected Data (2022)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ecl:stabus:3963

Access Statistics for this paper

More papers in Research Papers from Stanford University, Graduate School of Business Contact information at EDIRC.
Bibliographic data for series maintained by ().