Expert Validation of a Qualitative Interview Protocol for ESL Instructors’ Digital Assessment Literacy in Malaysian Higher Education: A Delphi Study

Ameera Irdena Binti Suyansah, Noraini Binti Said, Esther binti Jawing and Wardatul Akmam Binti Din
Additional contact information
Ameera Irdena Binti Suyansah: Faculty of Education and Sport Studies, University Malaysia Sabah, Malaysia
Noraini Binti Said: Faculty of Education and Sport Studies, University Malaysia Sabah, Malaysia
Esther binti Jawing: The Centre for the Promotion of Language Learning, University Malaysia Sabah, Malaysia
Wardatul Akmam Binti Din: Faculty of Education and Sport Studies, University Malaysia Sabah, Malaysia

International Journal of Research and Innovation in Social Science, 2025, vol. 9, issue 7, 2605-2618

Abstract: The integration of technology into educational assessment is a critical component of modern pedagogy, yet many educators lack proficiency in digital assessment methodologies. This study addresses the need for a validated instrument to explore the digital assessment literacy (DAL) of English as a Second Language (ESL) instructors in the Malaysian higher education context. The primary objective was to validate a qualitative interview protocol designed to investigate instructors’ assessment practices and their level of technology integration. The protocol consists of two instruments, one guided by the Teacher Assessment Literacy in Practice (TALiP) framework and the other by the Substitution, Augmentation, Modification, and Redefinition (SAMR) model. A Delphi method was employed to achieve consensus on the instrument’s content validity. Six experts in the fields of digital assessment and ESL, holding PhDs with over five years of relevant experience, were selected to evaluate the interview items. Inter-rater reliability was calculated using the Fleiss’ Kappa statistic. The results indicated an “almost perfect†level of agreement among the expert validators, with a Kappa value of 0.81 for the TALiP instrument and 0.83 for the SAMR instrument. Based on expert feedback, several items were refined for clarity, including the separation of double-barreled questions and the rephrasing of complex terminology, which increased the total number of items in the TALiP protocol from 10 to 15. The study concludes that the validated interview protocol is a robust and reliable instrument for exploring the multifaceted dimensions of DAL among ESL instructors. This tool can be effectively utilized by researchers and educational institutions to inform the design of targeted professional development programs aimed at enhancing digital assessment practices in higher education.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.rsisinternational.org/journals/ijriss/ ... ssue-7/2605-2618.pdf (application/pdf)
https://rsisinternational.org/journals/ijriss/arti ... tion-a-delphi-study/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bcp:journl:v:9:y:2025:issue-7:p:2605-2618

Access Statistics for this article

International Journal of Research and Innovation in Social Science is currently edited by Dr. Nidhi Malhan

More articles in International Journal of Research and Innovation in Social Science from International Journal of Research and Innovation in Social Science (IJRISS)
Bibliographic data for series maintained by Dr. Pawan Verma ().