Educating Low Digital Literacy Users Through Agentic Ai: A Multimodal On-Screen Coaching Assistant

Yvonne Ngu Rui Ging, Kurk Wei Yi, Zahriah Othman, Raja Rina Raja Ikram, Nooraini Yusoff and Universiti Malaysia Kelantan (UMK), Pengkalan Chepa, Kota Bharu, 16100, Kelantan, Malaysia
Additional contact information
Yvonne Ngu Rui Ging: Fakulti Teknologi Maklumat dan Komunikasi Universiti Teknikal Malaysia Melaka (UTeM), Durian Tunggal, Melaka, 76100, Malaysia
Kurk Wei Yi: Fakulti Teknologi Maklumat dan Komunikasi Universiti Teknikal Malaysia Melaka (UTeM), Durian Tunggal, Melaka, 76100, Malaysia
Zahriah Othman: Fakulti Teknologi Maklumat dan Komunikasi Universiti Teknikal Malaysia Melaka (UTeM), Durian Tunggal, Melaka, 76100, Malaysia
Raja Rina Raja Ikram: Fakulti Teknologi Maklumat dan Komunikasi Universiti Teknikal Malaysia Melaka (UTeM), Durian Tunggal, Melaka, 76100, Malaysia

International Journal of Research and Innovation in Social Science, 2025, vol. 9, issue 8, 7666-7673

Abstract: The increasing complexity of mobile applications poses significant challenges for users with limited technical literacy, often leading to frustration and abandonment of tasks. Existing AI-based assistants, such as Google Gemini and Microsoft Copilot, provide contextual help but do not provide detailed, step-by-step instructions specifically designed for mobile devices. This paper introduces Solus, an intelligent on-screen guidance assistant designed to provide real-time, multimodal support directly on Android devices. Unlike conventional assistants that rely on lengthy responses or task automation, Solus employs an STT-LLM-TTS pipeline integrated with real-time screen analysis to deliver interactive voice and visual step-by-step instructions. Developed using an Agile methodology, the system integrates technologies such as Google Cloud AI services, LiveKit Agent Framework, and Porcupine wake word detection. Results from prototype testing demonstrate that Solus reduces cognitive load, improves task efficiency, and empowers users to navigate digital applications independently. The findings highlight Solus’s potential to foster digital inclusion and inform the design of future human-centric AI assistants.

Date: 2025
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.rsisinternational.org/journals/ijriss/ ... ssue-8/7666-7673.pdf (application/pdf)
https://rsisinternational.org/journals/ijriss/arti ... -coaching-assistant/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bcp:journl:v:9:y:2025:issue-8:p:7666-7673

Access Statistics for this article

International Journal of Research and Innovation in Social Science is currently edited by Dr. Nidhi Malhan

More articles in International Journal of Research and Innovation in Social Science from International Journal of Research and Innovation in Social Science (IJRISS)
Bibliographic data for series maintained by Dr. Pawan Verma ().