Necessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problems

Kunal Pattanayak and Vikram Krishnamurthy

Papers from arXiv.org

Abstract: This paper presents an inverse reinforcement learning~(IRL) framework for Bayesian stopping time problems. By observing the actions of a Bayesian decision maker, we provide a necessary and sufficient condition to identify if these actions are consistent with optimizing a cost function. In a Bayesian (partially observed) setting, the inverse learner can at best identify optimality wrt the observed strategies. Our IRL algorithm identifies optimality and then constructs set-valued estimates of the cost function.To achieve this IRL objective, we use novel ideas from Bayesian revealed preferences stemming from microeconomics. We illustrate the proposed IRL scheme using two important examples of stopping time problems, namely, sequential hypothesis testing and Bayesian search. As a real-world example, we illustrate using a YouTube dataset comprising metadata from 190000 videos how the proposed IRL method predicts user engagement in online multimedia platforms with high accuracy. Finally, for finite datasets, we propose an IRL detection algorithm and give finite sample bounds on its error probabilities.

Date: 2020-07, Revised 2023-03
New Economics Papers: this item is included in nep-cbe and nep-upt
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Published in Journal of Machine Learning Research, Vol.24, 2023

Downloads: (external link)
http://arxiv.org/pdf/2007.03481 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2007.03481

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().