The Quantile Performance of Statistical Treatment Rules Using Hypothesis Tests to Allocate a Population to Two Treatments

Charles Manski and Aleksey Tetenov ()

No CWP44/14, CeMMAP working papers from Centre for Microdata Methods and Practice, Institute for Fiscal Studies

Abstract: This paper modifies the Wald development of statistical decision theory to offer new perspective on the performance of certain statistical treatment rules. We study the quantile performance of test rules, ones that use the outcomes of hypothesis tests to allocate a population to two treatments. Let ? denote the quantile used to evaluate performance. Define a test rule to be ?-quantile optimal if it maximizes ?-quantile welfare in every state of nature. We show that a test rule is ?-quantile optimal if and only if its error probabilities are less than ? in all states where the two treatments yield different welfare. We give conditions under which ?-quantile optimal test rules do and do not exist. A sufficient condition for existence of optimal rules is that the state space be finite and the data enable sufficiently precise estimation of the true state. Optimal rules do not exist when the state space is connected and other regularity conditions hold, but near-optimal rules may exist. These nuanced findings differ sharply from measurement of mean performance, as mean optimal test rules generically do not exist. We present further analysis that holds when the data are real-valued and generated by a sampling distribution which satisfies the monotone-likelihood ratio (MLR) property with respect to the average treatment effect. We use the MLR property to characterize the stochastic-dominance admissibility of STRs when the data have a continuous distribution and then generate findings on the quantile admissibility of test rules.

Date: 2014-11-20
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
https://www.ifs.org.uk/uploads/cemmap/wps/cwp441414.pdf (application/pdf)
Our link check indicates that this URL is bad, the error code is: 404 Not Found (https://www.ifs.org.uk/uploads/cemmap/wps/cwp441414.pdf [302 Found]--> https://ifs.org.uk/uploads/cemmap/wps/cwp441414.pdf)

Related works:
Working Paper: The Quantile Performance of Statistical Treatment Rules Using Hypothesis Tests to Allocate a Population to Two Treatments (2014)
Working Paper: The Quantile Performance Of Statistical Treatment Rules Using Hypothesis Tests To Allocate A Population To Two Treatments (2014)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ifs:cemmap:44/14

Ordering information: This working paper can be ordered from
The Institute for Fiscal Studies 7 Ridgmount Street LONDON WC1E 7AE

Access Statistics for this paper

More papers in CeMMAP working papers from Centre for Microdata Methods and Practice, Institute for Fiscal Studies The Institute for Fiscal Studies 7 Ridgmount Street LONDON WC1E 7AE. Contact information at EDIRC.
Bibliographic data for series maintained by Emma Hyman ().