Unsupervised exceptional human action detection from repetition of human assembling tasks using entropy signal clustering

Chao-Lung Yang (), Shang-Che Hsu (), Yu-Chung Kang (), Jing-Feng Nian () and Andi Cakravastia ()
Additional contact information
Chao-Lung Yang: National Taiwan University of Science and Technology
Shang-Che Hsu: National Taiwan University of Science and Technology
Yu-Chung Kang: National Taiwan University of Science and Technology
Jing-Feng Nian: National Taiwan University of Science and Technology
Andi Cakravastia: Bandung Institute of Technology

Journal of Intelligent Manufacturing, 2025, vol. 36, issue 6, No 7, 3815 pages

Abstract: Abstract Applying Human Action Recognition (HAR) in manufacturing site to recognize the human assembling tasks, representing as repetitions of human actions, is an emerging research area. However, human unintentional movements or actions beyond the pre-determined tasks are inevitable and cause the difficulty of applying machine learning model for training. Obviously, exhaustively listing all exceptional actions for training a model is not feasible. Therefore, when utilizing a pre-trained model or re-training a model by considering the limited number of exceptional actions leads to the low accuracy of action recognition. To overcome this challenge, in this work, an unsupervised detection framework named Entropy Signal Clustering (ESC) was proposed to detect the exceptional actions from a repetition of actions which are assumed to be basis of assembling tasks. In order to handle frame-basis detection, first, Temporal Self-similarity Matrix (TSM) from the encoder model of RepNet was constructed to represent the similarities of the pair-wise image frames extracted from a video. Second, a calculating process was re-invented to compensate the misleading similarity caused by the padding window on the edges of TSM. Then, an innovative measure of entropy was proposed to create the indication statistics of action similarity. Finally, a clustering-based time-series anomaly detection was proposed to detect the discordance representing the exceptional action in an entropy time-series data against the repetition of actions. To validate the proposed framework, a simulated videos dataset was created. The results show the proposed ESC method is able to obtain accuracies from 85 to 99% and recall of both exceptional (79–100%) and standard of procedure task (92–100%) actions across all demonstration videos comparing with Histogram, Matrix Profile, Local Recurrence Rate based Discord Search (LRRDS), and LRRDS Sampling K-means (LSK) methods.

Keywords: Unsupervised Learning; Exceptional Action Detection; Entropy Information; Time-Series Signal; Anomaly Detection (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10845-024-02420-4 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joinma:v:36:y:2025:i:6:d:10.1007_s10845-024-02420-4

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10845

DOI: 10.1007/s10845-024-02420-4

Access Statistics for this article

Journal of Intelligent Manufacturing is currently edited by Andrew Kusiak

More articles in Journal of Intelligent Manufacturing from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().