Model-averaged confidence intervals for factorial experiments

David Fletcher and Peter W. Dillingham

Computational Statistics & Data Analysis, 2011, vol. 55, issue 11, 3041-3048

Abstract: We consider the coverage rate of model-averaged confidence intervals for the treatment means in a factorial experiment, when we use a normal linear model in the analysis. Model-averaging provides a useful compromise between using the full model (containing all main effects and interactions) and a "best model" obtained by some model-selection process. Use of the full model guarantees perfect coverage, whereas use of a best model is known to lead to narrow intervals with poor coverage. Model-averaging allows us to achieve good coverage using intervals that are also narrower than those from the full model. We compare four information criteria that might be used for model-averaging in this setting: AIC, AICc, and BIC. In this setting, if the full model is "truth", all the criteria will have perfect coverage rates asymptotically. We use simulation to assess the coverage rates and interval widths likely to be achieved by a confidence interval with a nominal coverage of 95%. Our results suggest that AIC performs best in terms of coverage rate; across a wide range of scenarios and replication levels, it consistently provides coverage rates within 1.5% points of the nominal level, while also leading to reductions in interval-width of up to 30%, compared to the full model. AICc performed worst overall, with a coverage rate that was up to 5.2% points too low. We recommend that model-averaging become standard practise when summarising the results of a factorial experiment in terms of the treatment means, and that AIC be used to perform the model-averaging.

Keywords: Coverage; rate; Information; criterion; Model; uncertainty; Model; weight (search for similar items in EconPapers)
Date: 2011
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0167947311001800
Full text for ScienceDirect subscribers only.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:csdana:v:55:y:2011:i:11:p:3041-3048

Access Statistics for this article

Computational Statistics & Data Analysis is currently edited by S.P. Azen

More articles in Computational Statistics & Data Analysis from Elsevier
Bibliographic data for series maintained by Catherine Liu ().