Designing an Interactive Communication Assistance System for Hearing-Impaired College Students Based on Gesture Recognition and Representation

Yancong Zhu, Juan Zhang, Zhaoxi Zhang, Gina Clepper, Jingpeng Jia and Wei Liu
Additional contact information
Yancong Zhu: School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, China
Juan Zhang: Faculty of Psychology, Beijing Normal University, Beijing 100875, China
Zhaoxi Zhang: School of Engineering and Applied Sciences, University at Buffalo, Buffalo, NY 14260, USA
Gina Clepper: College of Engineering, Purdue University, West Lafayette, IN 47907, USA
Jingpeng Jia: College of Special Education, Beijing Union University, Beijing 100075, China
Wei Liu: Faculty of Psychology, Beijing Normal University, Beijing 100875, China

Future Internet, 2022, vol. 14, issue 7, 1-14

Abstract: Developing a smart classroom can make the modern classroom more efficient and intelligent. Much research has been conducted pertaining to smart classrooms for hearing-impaired college students. However, there have been few significant breakthroughs in mobilizing students’ learning efficiency as measured by information transmission, communication, and interaction in class. This research collects data through nonparticipatory observation and in-depth interviews and analyzes available data on classroom interaction needs of these students. We found that diversified explanations, recordable interactive contents, and interaction between teachers and students could improve the learning effects in the classroom. We also propose a tracking-processing method based on gesture recognition and representation and present a design for a processing system based on AT89C52 microcontroller and Kinect. In this way, sign language can be translated into text and all students can receive the information and participate in the interaction, which greatly improves students’ autonomy and enthusiasm of learning. This design enables deaf students to fully use classroom learning resources, reduces learning time costs, and improves learning efficiency. It can assist teachers in teaching and tutoring students to enhance their experience.

Keywords: hearing-impaired college student; multimedia-equipped classroom; qualitative analysis; gesture recognition and representation; tracking processing (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.mdpi.com/1999-5903/14/7/198/pdf (application/pdf)
https://www.mdpi.com/1999-5903/14/7/198/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:14:y:2022:i:7:p:198-:d:852027

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().