Evaluating Creative Work with Artificial Intelligence: Evidence from Constrained Innovation Tasks

Valerio Fedele Addis, Giuseppe Attanasi, Giovanni Di Bartolomeo (), Michele Mariella and Valentina Peruzzi

No 197, CIMEO Working Paper Series from Centre for Investigation and Modelling of Experimental Observations (CIMEO)

Abstract: We study whether a large language model can reliably evaluate human creativity in constrained, innovation-like tasks. Using expert-generated creative outputs from a validated experiment with workers in cultural and creative industries, we embed ChatGPT as an evaluator and benchmark its assessments against expert human judgments obtained through the Consensual Assessment Technique. In Study 1, we show that AI-based creativity evaluations exhibit internal consistency comparable to that of expert judges across repeated and independent runs, even under conservative scenarios. Replacing a human judge with an AI evaluator does not reduce inter-rater reliability across drawing, mathematical, and verbal tasks. In Study 2, we find that AI evaluations are systematically structured along fluency, flexibility, originality, and elaboration, with task-specific weighting of these dimensions. Overall, the results indicate that AI can serve as a reliable and structured evaluator of creativity in constrained innovation environments.

Keywords: Artificial intelligence; Creativity evaluation; Constrained creativity tasks; Consensual Assessment Technique; Cultural-and-creative-industry professionals; Innovation-like tasks (search for similar items in EconPapers)
JEL-codes: C91 D83 M14 O31 (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.dipecodir.it/wpcom/data/wp_no_197_2026.pdf (application/pdf)

Related works:
Journal Article: Evaluating creative work with artificial intelligence: Evidence from constrained innovation tasks (2026)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ter:wpaper:00197

Access Statistics for this paper

More papers in CIMEO Working Paper Series from Centre for Investigation and Modelling of Experimental Observations (CIMEO) Contact information at EDIRC.
Bibliographic data for series maintained by Giovanni Di Bartolomeo ().