Correcting for sampling problems in PISA and the improvement in Portuguese students' performance

Adriana Ferro, Pedro Freitas, Luis Nunes, Ana Reis and Carmo Seabra

Nova SBE Working Paper Series from Universidade Nova de Lisboa, Nova School of Business and Economics

Abstract: PISA uses a complex sampling procedure based on stratification variables chosen by the participating countries authorities. In this paper we analyse the representativeness of the samples used in terms of the distribution of students per grade and track of studies for Portugal. For the three exam years under analysis (2006, 2009 and 2012) a meaningful bias between the effective student distribution and PISA samples was found. We provide recalculated PISA scores that correct for the sample bias. We find that from 2009 to 2012, contrary to the stagnation denoted in the PISA results, the recalculated scores show an improvement in the Portuguese student performance. We also decompose the evolution of the scores into two effects: (i) change in the student population distribution by grade and track, and type of school; and (ii) evolution in the performance of each type of student. The results show that for the whole period the evolution of the students scores is the main driver of the increase in PISA results, with the change in the population structure playing a weaker role. Although we focus on the Portuguese case, similar problems of representativeness are expected to arise in other countries with high retention rates, affecting the grade of study of 15-year-olds.

Keywords: PISA; sampling; educational achievement; age-based assessment (search for similar items in EconPapers)
JEL-codes: I20 I25 (search for similar items in EconPapers)
Pages: 17 pages
Date: 2015
New Economics Papers: this item is included in nep-edu
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://run.unl.pt/bitstream/10362/82838/1/WP591.pdf

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:unl:unlfep:wp591

Access Statistics for this paper

More papers in Nova SBE Working Paper Series from Universidade Nova de Lisboa, Nova School of Business and Economics Contact information at EDIRC.
Bibliographic data for series maintained by Susana Lopes ().