Performance of Random Effects Model Estimators Under Complex Sampling Designs

Yue Jia, Lynne Stokes, Ian Harris and Yan Wang

Journal of Educational and Behavioral Statistics, 2011, vol. 36, issue 1, 6-32

Abstract: In this article, we consider estimation of parameters of random effects models from samples collected via complex multistage designs. Incorporation of sampling weights is one way to reduce estimation bias due to unequal probabilities of selection. Several weighting methods have been proposed in the literature for estimating the parameters of hierarchical models, of which random effects models are a special case. We evaluate the bias of the weighted analysis of variance (ANOVA) estimators of the variance components for a two-level, one-way random effects model. For these estimators, analytic bias expressions are developed and the accuracy of the expressions is evaluated through Monte Carlo simulation. The expressions are used to examine the impact of sample size, the size of the intraclass correlation coefficient (ICC), and the sampling design on the estimators' performance. The sampling designs considered are two-stage, with a general probability design at Level 2 and simple random sampling without replacement (SRS) at Level 1. The study shows that variance component estimators using only first-order weights perform well when both cluster size and ICC are moderate. However, this weighting method should be used with caution for small cluster sizes (less than 20), particularly when ICC is small (less than 0.2). In such scenarios, scaled first-order weighted (SFW) estimators provide an alternative to the difficult-to-use second-order weighted estimators for designs in which SRS is used at the ultimate sampling unit level (Level 1). This article is discussed in the context of large educational survey assessments.

Keywords: random effects model; variance components; estimation bias; ANOVA estimators; complex sampling designs; selection probability; sampling weights; ICC; NAEP (search for similar items in EconPapers)
Date: 2011
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.3102/1076998609359793 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:jedbes:v:36:y:2011:i:1:p:6-32

DOI: 10.3102/1076998609359793

Access Statistics for this article

More articles in Journal of Educational and Behavioral Statistics
Bibliographic data for series maintained by SAGE Publications ().