Reliability and Validity of Six Selected Observational Methods for Risk Assessment of Hand Intensive and Repetitive Work

Teresia Nyman (), Ida-Märta Rhén, Peter J. Johansson, Kristina Eliasson, Katarina Kjellberg, Per Lindberg, Xuelong Fan and Mikael Forsman
Additional contact information
Teresia Nyman: Department of Medical Sciences, Occupational and Environmental Medicine, Uppsala University, SE-751 85 Uppsala, Sweden
Ida-Märta Rhén: School of Engineering Sciences in Chemistry, Biotechnology and Health, KTH Royal Institute of Technology, SE-141 57 Huddinge, Sweden
Peter J. Johansson: Department of Medical Sciences, Occupational and Environmental Medicine, Uppsala University, SE-751 85 Uppsala, Sweden
Kristina Eliasson: Department of Medical Sciences, Occupational and Environmental Medicine, Uppsala University, SE-751 85 Uppsala, Sweden
Katarina Kjellberg: Centre for Occupational and Environmental Medicine, Stockholm County Council, SE-113 65 Stockholm, Sweden
Per Lindberg: Department of Occupational Health Science and Psychology, Faculty of Health and Occupational Studies, University of Gävle, SE-801 76 Gävle, Sweden
Xuelong Fan: Unit of Occupational Medicine, Institute of Environmental Medicine (IMM), Karolinska Institutet, SE-171 77 Stockholm, Sweden
Mikael Forsman: School of Engineering Sciences in Chemistry, Biotechnology and Health, KTH Royal Institute of Technology, SE-141 57 Huddinge, Sweden

IJERPH, 2023, vol. 20, issue 8, 1-26

Abstract: Risk assessments of hand-intensive and repetitive work are commonly done using observational methods, and it is important that the methods are reliable and valid. However, comparisons of the reliability and validity of methods are hampered by differences in studies, e.g., regarding the background and competence of the observers, the complexity of the observed work tasks and the statistical methodology. The purpose of the present study was to evaluate six risk assessment methods, concerning inter- and intra-observer reliability and concurrent validity, using the same methodological design and statistical parameters in the analyses. Twelve experienced ergonomists were recruited to perform risk assessments of ten video-recorded work tasks twice, and consensus assessments for the concurrent validity were carried out by three experts. All methods’ total-risk linearly weighted kappa values for inter-observer reliability (when all tasks were set to the same duration) were lower than 0.5 (0.15–0.45). Moreover, the concurrent validity values were in the same range with regards to total-risk linearly weighted kappa (0.31–0.54). Although these levels are often considered as being fair to substantial, they denote agreements lower than 50% when the expected agreement by chance has been compensated for. Hence, the risk of misclassification is substantial. The intra-observer reliability was only somewhat higher (0.16–0.58). Regarding the methods ART (Assessment of repetitive tasks of the upper limbs) and HARM (Hand Arm Risk Assessment Method), it is worth noting that the work task duration has a high impact in the risk level calculation, which needs to be taken into account in studies of reliability. This study indicates that when experienced ergonomists use systematic methods, the reliability is low. As seen in other studies, especially assessments of hand/wrist postures were difficult to rate. In light of these results, complementing observational risk assessments with technical methods should be considered, especially when evaluating the effects of ergonomic interventions.

Keywords: ergonomics; repetitive work; hand intensive; risk assessment; observation; reliability (search for similar items in EconPapers)
JEL-codes: I I1 I3 Q Q5 (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1660-4601/20/8/5505/pdf (application/pdf)
https://www.mdpi.com/1660-4601/20/8/5505/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jijerp:v:20:y:2023:i:8:p:5505-:d:1123173

Access Statistics for this article

IJERPH is currently edited by Ms. Jenna Liu

More articles in IJERPH from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().