AI Literacy in Higher Education: A Systematic Approach to Questionnaire Development and Validation

Maria Ranieri, Gabriele Biagini and Stefano Cuomo
Additional contact information
Maria Ranieri: University of Florence, Italy
Gabriele Biagini: University of Florence, Italy
Stefano Cuomo: University of Florence, Italy

International Journal of Digital Literacy and Digital Competence (IJDLDC), 2025, vol. 16, issue 1, 1-25

Abstract: This paper presents the development, refinement, and validation of the Critical Artificial Intelligence Literacy Scale, an instrument designed to measure artificial intelligence literacy across four dimensions: knowledge-related, operational, critical, and ethical. The initial version of the questionnaire, based on a robust theoretical framework and expert consultation, included 40 items and was tested with 57 doctoral students. It demonstrated strong psychometric properties (comparative fit index = 0.946, Tucker-Lewis index = 0.92) but showed limitations such as item redundancy (α = 0.947) and low performance of general items. To address these issues, the questionnaire was refined to a concise 24-item version. The revised instrument was evaluated using a sample of 314 first-year student teachers. Exploratory and confirmatory factor analyses confirmed a four-factor structure, with each dimension demonstrating strong reliability (Cronbach's alpha ranging from 0.838 to 0.912) and excellent model fit indices (comparative fit index = 0.960, root mean square error of approximation = 0.0441). The results validate the Critical Artificial Intelligence Literacy Scale as a reliable and efficient tool for assessing artificial intelligence literacy in educational settings.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://services.igi-global.com/resolvedoi/resolve ... 0.4018/IJDLDC.388469 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jdldc0:v:16:y:2025:i:1:p:1-25

Access Statistics for this article

International Journal of Digital Literacy and Digital Competence (IJDLDC) is currently edited by Tonia De Giuseppe

More articles in International Journal of Digital Literacy and Digital Competence (IJDLDC) from IGI Global Scientific Publishing
Bibliographic data for series maintained by Journal Editor ().