Avoiding Data Snooping in Multilevel and Mixed Effects Models

David Afshartous and Michael Wolf

No 260, IEW - Working Papers from Institute for Empirical Research in Economics - University of Zurich

Abstract: Multilevel or mixed effects models are commonly applied to hierarchical data; for example, see Goldstein (2003), Raudenbush and Bryk (2002), and Laird and Ware (1982). Although there exist many outputs from such an analysis, the level-2 residuals, otherwise known as random effects, are often of both substantive and diagnostic interest. Substantively, they are frequently used for institutional comparisons or rankings. Diagnostically, they are used to assess the model assumptions at the group level. Current inference on the level-2 residuals, however, typically does not account for data snooping, that is, for the harmful effects of carrying out a multitude of hypothesis tests at the same time. We provide a very general framework that encompasses both of the following inference problems: (1) Inference on the `absolute' level-2 residuals to determine which are significantly different from zero, and (2) Inference on any prespecified number of pairwise comparisons. Thus, the user has the choice of testing the comparisons of interest. As our methods are flexible with respect to the estimation method invoked, the user may choose the desired estimation method accordingly. We demonstrate the methods with the London Education Authority data used by Rasbash et al. (2004), the Wafer data used by Pinheiro and Bates (2000), and the NELS data used by Afshartous and de Leeuw (2004).

Keywords: Data snooping; hierarchical linear models; hypothesis testing; pairwise comparisons; random e�ects; rankings (search for similar items in EconPapers)
JEL-codes: C12 C14 C52 (search for similar items in EconPapers)
Date: 2005-12
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.zora.uzh.ch/id/eprint/52150/1/iewwp260.pdf (application/pdf)

Related works:
Journal Article: Avoiding ‘data snooping’ in multilevel and mixed effects models (2007)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zur:iewwpx:260

Access Statistics for this paper

More papers in IEW - Working Papers from Institute for Empirical Research in Economics - University of Zurich
Bibliographic data for series maintained by Severin Oswald ().