The Readability Paradox: Can We Trust Decisions on AI Detectors?

Akindeji Makinde
Additional contact information
Akindeji Makinde: Educational Research Methodology Department, University of North Carolina Greensboro, US & Department of Information Systems, Federal University of Technology, Akure, Ondo State, Nigeria & Data Science and Analytics Department, North Carolina A&T State University & Educational Research Methodology Department, University of North Carolina & Greensboro & Metallurgical and Material Engineering Department, Federal University of Technology, Akure, Nigeria. & Department of Information and Communication Technology (ICT), Federal University of Technology, Akure, Ondo State, Nigeria & Department of Software Engineering, Federal University of Technology, Akure, Ondo State, Nigeria

Technium Education and Humanities, 2025, vol. 11, 181-195

Abstract: Artificial Intelligence (AI) detectors are increasingly used in various domains, including healthcare, finance, criminal justice, and more, to make critical decisions. These systems are designed to identify patterns, anomalies, or specific features within data to assist or automate decision-making processes. However, the trustworthiness of AI detectors is a growing concern, particularly as these systems can exhibit biases, errors, and lack of transparency. This study evaluates the effectiveness of AI-detection tools and analyzes the linguistic characteristics distinguishing AI-generated and human-written academic texts across six disciplines. A total of 200 research papers were examined, 100 generated using large language and 100 peer-reviewed human-authored articles. Five AI detection tools were assessed for accuracy in classification. The study also conducted a comparative readability analysis using five established indices which include Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Index, SMOG Index, and total Word Count. Results indicate that while human-written papers were correctly identified by most tools with over 80% accuracy, AI-generated papers were frequently misclassified, especially after paraphrasing. Also from the result, the AI-generated texts were significantly shorter and exhibited higher syntactic complexity, with lower readability scores across all indices and disciplines. These findings underscore limitations in current detection tools and highlight notable stylistic differences in how AI and humans generate academic content, with implications for academic integrity policies and future AI writing systems.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://techniumscience.com/index.php/education/article/view/12946 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:tec:educat:ojs:12946

DOI: 10.47577/teh.v11i.12946

Access Statistics for this article

Technium Education and Humanities is currently edited by Anna Maria Golita

More articles in Technium Education and Humanities from Technium Science
Bibliographic data for series maintained by Anna Maria Golita ().