No More Grading: Incentive Compatible Peer Assessment in a Project-Based Course

Ethan Ligon

Department of Agricultural & Resource Economics, UC Berkeley, Working Paper Series from Department of Agricultural & Resource Economics, UC Berkeley

Abstract: We describe a system for grading collaborative student work that hasbeen developed and refined over eight years of a project-basedundergraduate course. Students complete four team projects persemester, with teams reassigned each round. After each project, everystudent evaluates other teams' deliverables, assesses their ownteammates' contributions, and predicts the scores they themselves willreceive. These three streams of assessment data are combined usingsingular value decomposition (SVD) applied to residualized evaluationmatrices. Residualization removes evaluator-specific biases (thetendency to rate generously or harshly); SVD then extracts thedominant latent quality axis from the bias-corrected data, weightingquestions by their informativeness. Scores are aggregated via mediansfor robustness to outlier evaluators. Each student's composite gradereflects five components: team quality, individual contribution asassessed by teammates, evaluator discrimination (rewarding carefulassessment of others), and two measures of self-assessment accuracy(penalizing the gap between predicted and actual peer ratings). Wemap composite scores to letter grades using an anchor-point methodthat keys grade boundaries to the median performance of the top fivestudents, spacing bands at one-third standard deviation intervals. The system also feeds assessment information forward into teamcomposition. Teams are formed via a stratified round-robin algorithmthat spreads technical skill evenly across groups; for subsequentprojects, self-reported skill is replaced by peer-assessed skill. Aconflict-avoidance mechanism uses "would you work with this personagain" responses to keep incompatible students apart. We present themathematical foundations in enough detail for replication, discussincentive properties (robustness to strategic manipulation,encouragement of honest and thoughtful evaluation), and illustrate theprocedure with a worked numerical example.

Keywords: Education; incentive-compatible; peer assessment; project-based learning; ai-proof education; grading (search for similar items in EconPapers)
Date: 2026-04-03
New Economics Papers: this item is included in nep-ppm
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.escholarship.org/uc/item/2183z6g8.pdf;origin=repeccitec (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cdl:agrebk:qt2183z6g8

Access Statistics for this paper

More papers in Department of Agricultural & Resource Economics, UC Berkeley, Working Paper Series from Department of Agricultural & Resource Economics, UC Berkeley Contact information at EDIRC.
Bibliographic data for series maintained by Lisa Schiff ().