Tests for statistical significance of a treatment effect in the presence of hidden sub-populations

B. Karmakar (), K. Dhara (), K. Dey (), A. Basu () and A. Ghosh ()

Statistical Methods & Applications, 2015, vol. 24, issue 1, 97-119

Abstract: For testing the statistical significance of a treatment effect, we often compare between two parts of a population; one is exposed to the treatment, and the other is not exposed to it. Standard parametric or nonparametric two-sample tests are commonly used for this comparison. But direct applications of these tests can yield misleading results, especially when the population has some hidden sub-populations, and the effect of this sub-population difference on the response dominates the treatment effect. This problem becomes more evident if these sub-populations have widely different proportions of representatives in the samples obtained from these two parts. In this article, we propose some simple methods to overcome these limitations. These proposed methods first use a suitable clustering algorithm to find the hidden sub-populations, and then they eliminate the sub-population effect by using a suitable transformation of the data. Standard two-sample tests, when they are applied on the transformed data, usually yield better results. We analyze some simulated and real data sets to demonstrate the utility of these proposed methods. Copyright Springer-Verlag Berlin Heidelberg 2015

Keywords: Dunn index; Gap statistic; EM algorithm; Bayesian model averaging; PAM clustering algorithm; Univariate and multivariate two-sample tests (search for similar items in EconPapers)
Date: 2015
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1007/s10260-014-0271-x (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:stmapp:v:24:y:2015:i:1:p:97-119

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10260/PS2

DOI: 10.1007/s10260-014-0271-x

Access Statistics for this article

Statistical Methods & Applications is currently edited by Tommaso Proietti

More articles in Statistical Methods & Applications from Springer, Società Italiana di Statistica
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().