Evaluating Response Shift in Training Evaluation

Valentina Piwowar and Felicitas Thiel

Evaluation Review, 2014, vol. 38, issue 5, 420-448

Abstract: Background: Response shift (RS) can threaten the internal validity of pre–post designs. As RS may indicate a redefinition of the target construct, its occurrence in training evaluation is rather likely. The most common approach to deal with RS is to implement a retrospective pretest (then-test) instead of the traditional pre-test. In health psychology, an adapted measurement invariance approach (MI ad ) was developed as an alternative technique to study RS. Results produced by identifying RS with the two approaches were rarely studied simultaneously or within an experimental framework. Objectives: To study RS in two different treatment conditions and compare results produced by both techniques in identifying various types of RS. We further studied validity aspects of the then-test. Research Design: We evaluated RS by applying the then-test procedure (TP) and the measurement invariance apporach MI ad within an experimental design: Participants either attended a short-term or a long-term classroom management training program. Subjects: Participants were 146 student teachers in their first year of master’s study. Measures: Pre (before training), post, and then self-ratings (after training) on classroom management knowledge were administered. Results: Results indicated that the two approaches do not yield the same results. The MI ad identified more and also group-specific RS as opposed to the findings of the TP, which found less and only little evidence for group-specific RS. Conclusions: Further research is needed to study the usability and validity of the respective approaches. In particular, the usability of the then-test seems to be challenged.

Keywords: response shift; retrospective pretest; training evaluation; pre–post design; classroom management (search for similar items in EconPapers)
Date: 2014
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0193841X14546932 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:evarev:v:38:y:2014:i:5:p:420-448

DOI: 10.1177/0193841X14546932

Access Statistics for this article

More articles in Evaluation Review
Bibliographic data for series maintained by SAGE Publications ().