An individualized system of skeletal data-based CNN classifiers for action recognition in manufacturing assembly

Md. Al-Amin, Ruwen Qin (), Md Moniruzzaman, Zhaozheng Yin, Wenjin Tao and Ming C. Leu
Additional contact information
Md. Al-Amin: Missouri University of Science and Technology
Ruwen Qin: Stony Brook University
Md Moniruzzaman: Stony Brook University
Zhaozheng Yin: Stony Brook University
Wenjin Tao: Missouri University of Science and Technology
Ming C. Leu: Missouri University of Science and Technology

Journal of Intelligent Manufacturing, 2023, vol. 34, issue 2, No 13, 633-649

Abstract: Abstract Real-time Action Recognition (ActRgn) of assembly workers can timely assist manufacturers in correcting human mistakes and improving task performance. Yet, recognizing worker actions in assembly reliably is challenging because such actions are complex and fine-grained, and workers are heterogeneous. This paper proposes to create an individualized system of Convolutional Neural Networks (CNNs) for action recognition using human skeletal data. The system comprises six 1-channel CNN classifiers that each is built with one unique posture-related feature vector extracted from the time series skeletal data. Then, the six classifiers are adapted to any new worker through transfer learning and iterative boosting. After that, an individualized fusion method named Weighted Average of Selected Classifiers (WASC) integrates the adapted classifiers as an ActRgn system that outperforms its constituent classifiers. An algorithm of stream data analysis further differentiates the actions for assembly from the background and corrects misclassifications based on the temporal relationship of the actions in assembly. Compared to the CNN classifier directly built with the skeletal data, the proposed system improves the accuracy of action recognition by 28%, reaching 94% accuracy on the tested group of new workers. The study also builds a foundation for immediate extensions for adapting the ActRgn system to current workers performing new tasks and, then, to new workers performing new tasks.

Keywords: Convolutional neural network; Action recognition; Transfer learning; Iterative boosting; Classifier fusion; Smart manufacturing; Deep learning (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10845-021-01815-x Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joinma:v:34:y:2023:i:2:d:10.1007_s10845-021-01815-x

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10845

DOI: 10.1007/s10845-021-01815-x

Access Statistics for this article

Journal of Intelligent Manufacturing is currently edited by Andrew Kusiak

More articles in Journal of Intelligent Manufacturing from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().