A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Experts

Ingmar Nitze, Jurjen Van der Sluijs, Sophia Barth, Philipp Bernhard, Lingcao Huang, Alexander Kizyakov, Mark J. Lara, Nina Nesterova, Alexandra Runge, Alexandra Veremeeva, Melissa Ward Jones, Chandi Witharana, Zhuoxuan Xia and Anna K. Liljedahl

Permafrost and Periglacial Processes, 2025, vol. 36, issue 1, 83-92

Abstract: Deep‐learning (DL) models have become increasingly beneficial for the detection of retrogressive thaw slumps (RTS) in the permafrost domain. However, comparing accuracy metrics is challenging due to unstandardized labeling guidelines. To address this, we conducted an experiment with 12 international domain experts from a broad range of scientific backgrounds. Using 3 m PlanetScope multispectral imagery, they digitized RTS footprints in two sites. We evaluated label uncertainty by comparing manually outlined RTS labels using Intersection‐over‐Union (IoU) and F1 metrics. At the Canadian Peel Plateau site, we see good agreement, particularly in the active parts of RTS. Differences were observed in the interpretation of the debris tongue and the stable vegetated sections of RTS. At the Russian Bykovsky site, we observed a larger mismatch. Here, the same differences were documented, but several participants mistakenly identified non‐RTS features. This emphasizes the importance of site‐specific knowledge for reliable label creation. The experiment highlights the need for standardized labeling procedures and definition of their scientific purpose. The most similar expert labels outperformed the accuracy metrics reported in the literature, highlighting human labeling capabilities with proper training, site knowledge, and clear guidelines. These findings lay the groundwork for DL‐based RTS monitoring in the pan‐Arctic.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/ppp.2249

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wly:perpro:v:36:y:2025:i:1:p:83-92

Access Statistics for this article

More articles in Permafrost and Periglacial Processes from John Wiley & Sons
Bibliographic data for series maintained by Wiley Content Delivery ().