The Quantile Performance of Statistical Treatment Rules Using Hypothesis Tests to Allocate a Population to Two Treatments

Charles Manski and Aleksey Tetenov

No 44/14, CeMMAP working papers from Institute for Fiscal Studies

Abstract: This paper modifies the Wald development of statistical decision theory to offer new perspective on the performance of certain statistical treatment rules. We study the quantile performance of test rules, ones that use the outcomes of hypothesis tests to allocate a population to two treatments. Let λ denote the quantile used to evaluate performance. Define a test rule to be λ-quantile optimal if it maximizes λ-quantile welfare in every state of nature. We show that a test rule is λ-quantile optimal if and only if its error probabilities are less than λ in all states where the two treatments yield different welfare. We give conditions under which λ-quantile optimal test rules do and do not exist. A sufficient condition for existence of optimal rules is that the state space be finite and the data enable sufficiently precise estimation of the true state. Optimal rules do not exist when the state space is connected and other regularity conditions hold, but near-optimal rules may exist. These nuanced findings differ sharply from measurement of mean performance, as mean optimal test rules generically do not exist. We present further analysis that holds when the data are real-valued and generated by a sampling distribution which satisfies the monotone-likelihood ratio (MLR) property with respect to the average treatment effect. We use the MLR property to characterize the stochastic-dominance admissibility of STRs when the data have a continuous distribution and then generate findings on the quantile admissibility of test rules.

Date: 2014-11-20
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.cemmap.ac.uk/wp-content/uploads/2020/08/CWP4414.pdf (application/pdf)

Related works:
Working Paper: The Quantile Performance Of Statistical Treatment Rules Using Hypothesis Tests To Allocate A Population To Two Treatments (2014)
Working Paper: The Quantile Performance of Statistical Treatment Rules Using Hypothesis Tests to Allocate a Population to Two Treatments (2014)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:azt:cemmap:44/14

DOI: 10.1920/wp.cem.2014.4414

Access Statistics for this paper

More papers in CeMMAP working papers from Institute for Fiscal Studies Contact information at EDIRC.
Bibliographic data for series maintained by Dermot Watson ().