Characterizing LLM scientific concept generation: A multi-dimensional measurement study

Jalil Ahmadpour

PLOS ONE, 2026, vol. 21, issue 9, 1-19

Abstract: Large Language Models (LLMs) can generate text describing scientific concepts, but the characteristics of these outputs remain poorly understood. We present a multi-dimensional characterization framework analyzing 9,666 outputs from seven models (GPT-4.1, GPT-5.2, GPT-5.5, o4-mini, Claude Sonnet 4.5, Claude Opus 4.5, and the open-weight Gemma-3-27B) across five scientific domains. Rather than making claims about creativity or novelty, we measure five independent dimensions: coherence, domain relevance, lexical profile, structural properties, and semantic position using four sentence embedding models spanning 2020–2024. After quality filtering (99.2% coherence, 99.9% domain relevance pass rates), we find that outputs exhibit graduate-level readability (median Flesch-Kincaid grade 16.3) and occupy semantic positions at the 83rd percentile of calibration distributions. All 39 metrics differ significantly across models (Kruskal-Wallis, p

Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0357892 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 57892&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0357892

DOI: 10.1371/journal.pone.0357892

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().