Quantitative Analysis of the Labeling Quality of Biological Images for Semantic Segmentation Based on Attribute Agreement Analysis

Rong Xiang (), Xinyu Yuan, Yi Zhang and Xiaomin Zhang
Additional contact information
Rong Xiang: College of Quality and Standardization, China Jiliang University, Hangzhou 310018, China
Xinyu Yuan: College of Quality and Standardization, China Jiliang University, Hangzhou 310018, China
Yi Zhang: College of Quality and Standardization, China Jiliang University, Hangzhou 310018, China
Xiaomin Zhang: College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, China

Agriculture, 2025, vol. 15, issue 7, 1-22

Abstract: Semantic segmentation in biological images is increasingly common, particularly in smart agriculture, where deep learning model precision is tied to image labeling quality. However, research has largely focused on improving models rather than analyzing image labeling quality. We proposed a method for quantitatively assessing labeling quality in semantically segmented biological images using attribute agreement analysis. This method evaluates labeling variation, including internal, external, and overall labeling quality, and labeling bias between labeling results and standards through case studies of tomato stem and group-reared pig images, which vary in labeling complexity. The process involves the following three steps: confusion matrix calculation, Kappa value determination, and labeling quality assessment. Initially, two labeling workers were randomly selected to label ten images from each category twice, according to the requirements of the attribute agreement analysis method. Confusion matrices for each image’s dual labeling results were calculated, followed by Kappa value computation. Finally, labeling quality was evaluated by comparing Kappa values against quality criteria. We also introduced a contour ring method to enhance Kappa value differentiation in imbalanced sample scenarios. Three types of representative images were used to test the performance of the proposed method. The results show that attribute agreement analysis effectively quantifies image labeling quality, and the contour ring method improves Kappa value differentiation. The attribute agreement analysis method allows for quantitative analysis of labeling quality based on image labeling difficulty, and Kappa values can also be used as a metric of image labeling difficulty. Dynamic analysis of image labeling variations over time needs further research.

Keywords: semantic segmentation; image labeling; quality evaluation; attribute agreement analysis; Kappa coefficient (search for similar items in EconPapers)
JEL-codes: Q1 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 (search for similar items in EconPapers)
Date: 2025
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2077-0472/15/7/680/pdf (application/pdf)
https://www.mdpi.com/2077-0472/15/7/680/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jagris:v:15:y:2025:i:7:p:680-:d:1618347

Access Statistics for this article

Agriculture is currently edited by Ms. Leda Xuan

More articles in Agriculture from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().