The Reliability and Validity of Using Regression Residuals to Measure Institutional Effectiveness in Promoting Degree Completion

Aaron S. Horn () and Giljae Lee ()
Additional contact information
Aaron S. Horn: Midwestern Higher Education Compact
Giljae Lee: Chungbuk National University

Research in Higher Education, 2016, vol. 57, issue 4, No 4, 469-496

Abstract: Abstract A relatively simple way of measuring institutional effectiveness in relation to degree completion is to estimate the difference between an actual and predicted graduation rate, but the reliability and validity of this method have not been thoroughly examined. Longitudinal data were obtained from IPEDS for both public and private not-for-profit 4-year institutions (n = 1496). Hierarchical panel regression was used to predict 4- and 6-year graduation rates based on structural, demographic, financial, and contextual attributes. A direct effects model yielded effectiveness scores that were highly correlated between consecutive data years (r = 0.65–0.80), which indicated acceptable to good test–retest reliability. A test of convergent validity indicated that effectiveness scores were positively associated with students’ perceptions of a supportive campus environment (r = 0.32–0.45). A test of discriminant validity revealed relatively small correlations between effectiveness scores and institutional attributes, such as educational expenditures (r = 0.07–0.16). The modeling of interaction effects in relation to institutional type marginally improved the validity of effectiveness scores among public but not private institutions. The results suggest that correct model specification can yield residual scores that reliably and validly measure institutional effectiveness in promoting timely degree completion.

Keywords: Institutional effectiveness; Graduation rates; Accountability; Performance indicators (search for similar items in EconPapers)
Date: 2016
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s11162-015-9394-7 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:reihed:v:57:y:2016:i:4:d:10.1007_s11162-015-9394-7

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/11162

DOI: 10.1007/s11162-015-9394-7

Access Statistics for this article

Research in Higher Education is currently edited by Robert K. Toutkoushian

More articles in Research in Higher Education from Springer, Association for Institutional Research
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().