Monitoring Scale Scores over Time via Quality Control Charts, Model-Based Approaches, and Time Series Techniques

Yi-Hsuan Lee () and Alina Davier

Psychometrika, 2013, vol. 78, issue 3, 557-575

Abstract: Maintaining a stable score scale over time is critical for all standardized educational assessments. Traditional quality control tools and approaches for assessing scale drift either require special equating designs, or may be too time-consuming to be considered on a regular basis with an operational test that has a short time window between an administration and its score reporting. Thus, the traditional methods are not sufficient to catch unusual testing outcomes in a timely manner. This paper presents a new approach for score monitoring and assessment of scale drift. It involves quality control charts, model-based approaches, and time series techniques to accommodate the following needs of monitoring scale scores: continuous monitoring, adjustment of customary variations, identification of abrupt shifts, and assessment of autocorrelation. Performance of the methodologies is evaluated using manipulated data based on real responses from 71 administrations of a large-scale high-stakes language assessment. Copyright The Psychometric Society 2013

Keywords: quality assurance; seasonal adjustment; change point; Shewhart chart; CUSUM chart; hidden Markov model; harmonic regression (search for similar items in EconPapers)
Date: 2013
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (11)

Downloads: (external link)
http://hdl.handle.net/10.1007/s11336-013-9317-5 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:psycho:v:78:y:2013:i:3:p:557-575

Ordering information: This journal article can be ordered from
http://www.springer. ... gy/journal/11336/PS2

DOI: 10.1007/s11336-013-9317-5

Access Statistics for this article

Psychometrika is currently edited by Irini Moustaki

More articles in Psychometrika from Springer, The Psychometric Society
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().