Assessing the assessments: Taking stock of learning outcomes data in India

Doug Johnson and Andres Parrado

International Journal of Educational Development, 2021, vol. 84, issue C

Abstract: Data on learning outcomes is essential for tracking progress in achieving education goals, understanding what education policies work (and don’t work), and holding public officials accountable. We assess the accuracy and reliability of India’s two nationally representative surveys on learning outcomes, ASER and NAS, so that users of these datasets may better understand when, and for what purposes, these two datasets can reasonably be used. After restricting our sample to maximize comparability between the two datasets, we find that NAS state averages are significantly higher than ASER state averages and averages from an independently conducted and nationally representative survey (IHDS). In addition, state rankings based on NAS data display almost no correlation with state rankings based on ASER, IHDS, or net state domestic product per capita. We conclude that NAS state averages are likely artificially high and contain little information about states’ relative performance. The presence of severe bias in the NAS data suggests that this data should be used carefully or not at all for comparisons between states, constructing learning profiles, or any other purpose. We then analyze the internal reliability of ASER data using variance decomposition methods. We find that while ASER data is mostly reliable for comparing state averages, it is less reliable for looking at district averages, or changes in district and state averages over time. We conclude that analysts may use ASER data with confidence for comparisons between states in a single year, constructing learning profiles, and assessing learning inequality but should exercise caution when comparing changes in state scores and avoid using ASER district-level data.

Keywords: Learning outcome data; ASER; NAS; Variance decomposition (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0738059321000626
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:injoed:v:84:y:2021:i:c:s0738059321000626

DOI: 10.1016/j.ijedudev.2021.102409

Access Statistics for this article

International Journal of Educational Development is currently edited by Stephen P Heyneman

More articles in International Journal of Educational Development from Elsevier
Bibliographic data for series maintained by Catherine Liu ().