Methodological Considerations in Using Complex Survey Data: An Applied Example With the Head Start Family and Child Experiences Survey
Debbie L. Hahs-Vaughn (),
Christine M. McWayne,
Rebecca J. Bulotsky-Shearer,
Xiaoli Wen and
Ann-Marie Faria Additional contact information Debbie L. Hahs-Vaughn: Department of Educational Research, University of Central Florida, Orlando, FL, USA
Christine M. McWayne: Tufts University, Boston, MA, USA
Rebecca J. Bulotsky-Shearer: University of Miami, Miami, FL, USA
Xiaoli Wen: National-Louis University, Chicago, IL, USA
Ann-Marie Faria: American Institute for Research, Washington, DC, USA
Abstract:
Complex survey data are collected by means other than simple random samples. This creates two analytical issues: nonindependence and unequal selection probability. Failing to address these issues results in underestimated standard errors and biased parameter estimates. Using data from the nationally representative Head Start Family and Child Experiences Survey (FACES; 1997 and 2000 cohorts), three diverse multilevel models are presented that illustrate differences in results depending on addressing or ignoring the complex sampling issues. Limitations of using complex survey data are reported, along with recommendations for reporting complex sample results.