AI Response Quality in Public Services: Temperature Settings and Contextual Factors

Domenico Trezza (), Giuseppe Luca De Luca Picione and Carmine Sergianni
Additional contact information
Domenico Trezza: Department of Social Sciences, University of Naples Federico II, 80138 Napoli, NA, Italy
Giuseppe Luca De Luca Picione: Department of Economics, Management and Institutions, University of Naples Federico II, 80138 Napoli, NA, Italy
Carmine Sergianni: Department of Economics, Management and Institutions, University of Naples Federico II, 80138 Napoli, NA, Italy

Societies, 2025, vol. 15, issue 5, 1-17

Abstract: This study investigated how generative Artificial Intelligence (AI) systems—now increasingly integrated into public services—respond to different technical configurations, and how these configurations affect the perceived quality of the outputs. Drawing on an experimental evaluation of Govern-AI , a chatbot designed for professionals in the social, educational, and labor sectors, we analyzed the impact of the temperature parameter—which controls the degree of creativity and variability in the responses—on two key dimensions: accuracy and comprehensibility . This analysis was based on 8880 individual evaluations collected from five professional profiles. The findings revealed the following: (1) the high-temperature responses were generally more comprehensible and appreciated, yet less accurate in strategically sensitive contexts; (2) professional groups differed significantly in their assessments, where trade union representatives and regional policy staff expressed more critical views than the others; (3) the type of question —whether operational or informational—significantly influenced the perceived output quality. This study demonstrated that the AI performance was far from neutral: it depended on technical settings, usage contexts, and the profiles of the end users. Investigating these “behind-the-scenes” dynamics is essential for fostering the informed governance of AI in public services, and for avoiding the risk of technology functioning as an opaque black box within decision-making processes.

Keywords: generative artificial intelligence; chatbot; public policy; human–AI interaction; professional users; social policy (search for similar items in EconPapers)
JEL-codes: A13 A14 P P0 P1 P2 P3 P4 P5 Z1 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2075-4698/15/5/127/pdf (application/pdf)
https://www.mdpi.com/2075-4698/15/5/127/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jsoctx:v:15:y:2025:i:5:p:127-:d:1649695

Access Statistics for this article

Societies is currently edited by Ms. Farrah Sun

More articles in Societies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().