Evidence-based policy or beauty contest? An LLM-based meta-analysis of EU cohesion policy evaluations

Zareh Asatryan (), Carlo Birkholz and Friedrich Heinemann
Additional contact information
Zareh Asatryan: ZEW Mannheim
Carlo Birkholz: University of Mannheim
Friedrich Heinemann: University of Heidelberg

International Tax and Public Finance, 2025, vol. 32, issue 2, No 10, 625-655

Abstract: Abstract Independent and high-quality evaluations of government policies are an important input for designing evidence-based policy. Institutional frictions and lack of incentives to write such evaluations, on the other hand, carry the risk of turning the system into a costly beauty contest. We study one of the most advanced markets of policy evaluations in the world, the evaluations of EU Cohesion Policy interventions by the EU Member States. We use a large language model to quantify the findings of about 2,300 evaluations, and complement this data with our own survey of the evaluation authors. We show that the findings of evaluations are inconsistent with those of the academic literature on the output impacts of Cohesion Policy. Using further variation across Member States, our analysis suggests that the market of evaluations is rather oligopolistic within Member States, that it is very fragmented across the EU, and that there is often a strong involvement of managing authorities in the work of formally independent evaluators. These factors contribute to making the findings of the evaluations overly optimistic (beautiful) risking their relevance for (evidence-based) policy. We conclude by discussing reform options to make the evaluations of EU Cohesion Policy more unbiased and effective.

Keywords: Policy evaluation; EU cohesion policy; Large language model (search for similar items in EconPapers)
JEL-codes: A11 C45 D83 H43 H54 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10797-024-09875-4 Abstract (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:kap:itaxpf:v:32:y:2025:i:2:d:10.1007_s10797-024-09875-4

Ordering information: This journal article can be ordered from
http://www.springer. ... ce/journal/10797/PS2

DOI: 10.1007/s10797-024-09875-4

Access Statistics for this article

International Tax and Public Finance is currently edited by Ronald B. Davies and Kimberly Scharf

More articles in International Tax and Public Finance from Springer, International Institute of Public Finance Contact information at EDIRC.
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().