Ensuring the “health” of a curricular program evaluation: Alignment and analytic quality of two instruments for use in evaluating the effectiveness of an interprofessional collaboration curriculum

Shannon Sampson, Andrew Nelson, Roberto Cardarelli and Karen L. Roper

Evaluation and Program Planning, 2024, vol. 102, issue C

Abstract: To cultivate competencies in interprofessional collaboration (IPC) for patient-centered, team-based care, a multi-faceted training enhancement initiative was implemented at our academic primary care residency site. Evaluation of the activities from previously collected survey data occurred upon a 2-year review. First, the evaluation team scrutinized the instruments for alignment and appropriateness with planned IPC educational learning and behavior objectives. We found the two instruments were well supported by the literature and with appropriate evidence for validation, but were not well aligned to the objectives of this IPC training initiative, reducing appropriateness of potential inferences of the findings for this context. Second, the team assessed the analytic quality of survey results in item difficulty distribution and item fit to the requirements of a Rasch measurement model. This revealed low person separation due to high overall item agreement. Most residents agreed with most items, so the measures lacked the precision necessary to capture change in residents’ IPC competency. Our instrument review serves as a reminder of the need to gather validity evidence for the use of any existing tool within a new context, and offers a generalizable strategy to evaluate data sources for appropriateness and quality within a specific program.

Keywords: Medical education evaluation; Interdisciplinary medical education; Primary care education; Testing/assessment; Rasch measurement; Curriculum development/evaluation (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0149718923001544
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:epplan:v:102:y:2024:i:c:s0149718923001544

DOI: 10.1016/j.evalprogplan.2023.102377

Access Statistics for this article

Evaluation and Program Planning is currently edited by Jonathan A. Morell

More articles in Evaluation and Program Planning from Elsevier
Bibliographic data for series maintained by Catherine Liu ().