Threshold-indexability of restless bandits with real interval state spaces: a performance-metric verification framework and long-run average analysis

José Niño Mora

DES - Working Papers. Statistics and Econometrics. WS from Universidad Carlos III de Madrid. Departamento de Estadística

Abstract: Restless multiarmed bandits are Markov decision process models for allocating a scarce resourceamong projects whose states evolve under active or passive actions. Whittle's index policy is widelyused for such problems, but its application to a given model requires both a proof of indexabilityand a means of computing the index, two analytically challenging tasks. This paper develops aperformance-metric framework for proving threshold-indexability and computing Whittle indicesfor binary-action projects with real interval state spaces. The framework extends discounted partialconservation law (PCL) methods to a criterion-agnostic setting and works directly with rewardand resource metrics of threshold policies, rather than first proving threshold optimality and thenmonotonicity of optimal thresholds in the resource price. The main theorem is a verificationand characterization result: under marginal-resource positivity and a marginal integration-bypartsidentity, threshold-indexability is equivalent to monotonicity and continuity of the marginalproductivity (MP) index, which then equals the Whittle index. The framework is specialized to thediscrete-time long-run average criterion by a vanishing-discount transfer of discounted thresholdmetrics and includes exceptional states where the MP marginal-resource denominator vanishes,handled by continuous extension or vanishing-discount limits. Applications to web crawling andnoisy-channel transmission recover known long-run average Whittle indices. For scalar Kalman-filterbandits, it proves a regular-part average-cost result and reduces the remaining indexability questionto explicit exceptional-state metric-limit conjectures.

Keywords: Restless; bandits; Whittle; index; Indexability; Threshold; policies; Real-valued; state; spaces; Long-run; average; criterion; Partial; conservation; laws (search for similar items in EconPapers)
Date: 2026-05-26
References: Add references at CitEc
Citations:

Downloads: (external link)
https://e-archivo.uc3m.es/rest/api/core/bitstreams ... 2ced83303b3e/content (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cte:wsrepe:50161

Access Statistics for this paper

More papers in DES - Working Papers. Statistics and Econometrics. WS from Universidad Carlos III de Madrid. Departamento de Estadística
Bibliographic data for series maintained by Ana Poveda ().