Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis

Jesse Bruhn, Michael Gilraine, Jens Ludwig and Sendhil Mullainathan

No 34484, NBER Working Papers from National Bureau of Economic Research, Inc

Abstract: Much of the data collected in education is effectively thrown away. Students answer individual test questions, but administrators and researchers only see aggregate performance. All the item-level data are lost. Ex ante it is not clear this destroys much useful information, since the aggregate might be a sufficient statistic. Using data from Texas for 5 million students and 1.31 billion student-item responses, we show that in fact aggregation does destroy a great deal of valuable information in education: (1) Even conditional on a summary test measure, there is additional information in the item-level data; (2) This additional information is relevant for the student outcomes that education decisions seek to optimize; and (3) This information can be made practically useful for schools. Given how inexpensive storing, transmitting and analyzing such data would be, large gains could be had in education by simply using all the data we currently collect.

JEL-codes: I20 I21 J24 (search for similar items in EconPapers)
Date: 2025-11
New Economics Papers: this item is included in nep-edu and nep-lma
Note: ED
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.nber.org/papers/w34484.pdf (application/pdf)
Access to the full text is generally limited to series subscribers, however if the top level domain of the client browser is in a developing country or transition economy free access is provided. More information about subscriptions and free access is available at http://www.nber.org/wwphelp.html. Free access is also available to older working papers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nbr:nberwo:34484

Ordering information: This working paper can be ordered from
http://www.nber.org/papers/w34484
The price is Paper copy available by mail.

Access Statistics for this paper

More papers in NBER Working Papers from National Bureau of Economic Research, Inc National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.. Contact information at EDIRC.
Bibliographic data for series maintained by ().