LLMs for research: quality, speed and cost. Comparison with Pi, replaying project tickets from Imperial Dragon Harness workflows

Minh Ha-Duong ()
Additional contact information
Minh Ha-Duong: CIRED - Centre International de Recherche sur l'Environnement et le Développement - Cirad - Centre de Coopération Internationale en Recherche Agronomique pour le Développement - EHESS - École des hautes études en sciences sociales - AgroParisTech - Université Paris-Saclay - CNRS - Centre National de la Recherche Scientifique - ENPC - École nationale des ponts et chaussées - IP Paris - Institut Polytechnique de Paris

Working Papers from HAL

Abstract: Twenty hosted and local language-model configurations replay ten tickets randomly sampled from research project histories. A single model completes each task and reviews its own output; three blinded judge models score the deliverable out of 30. The report compares quality, elapsed time and direct cost per ticket, with Pareto frontiers, paired Wilcoxon tests (no comparison survives Holm correction), distributions of paired differences, and a selection guide by confidentiality, delay, budget and available hardware.

Date: 2026-10-09
Note: View the original document on HAL open archive server: https://hal.science/hal-05847479v1
References: Add references at CitEc
Citations:

Published in CIRED. 2026

Downloads: (external link)
https://hal.science/hal-05847479v1/document (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hal:wpaper:hal-05847479

Access Statistics for this paper

More papers in Working Papers from HAL
Bibliographic data for series maintained by CCSD ().