One versus all: identifiability with a multi-hazard and multiclass building damage imagery dataset and a deep learning neural network

Olalekan R. Sodeinde (), Magaly Koch, Babak Moaveni and Laurie G. Baise ()
Additional contact information
Olalekan R. Sodeinde: Tufts University
Magaly Koch: Tufts University
Babak Moaveni: Tufts University
Laurie G. Baise: Tufts University

Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, 2024, vol. 120, issue 9, No 10, 8337-8366

Abstract: Abstract This paper analyzed the quality of the xBD image-training dataset for identifying building damage across a variety of natural hazards using deep learning convolutional neural networks. Specifically, we evaluated the pros and cons of combining training datasets across multiple natural hazards and provided recommendations on using the provided training dataset to optimize classification accuracy for building damage detection. The xBD dataset was rebalanced, using random over-sampling and under-sampling methods. Random over-sampling randomly duplicates the minority class, while random under-sampling randomly cuts-off the majority class. With the balanced dataset, we used the xBD baseline architecture as a starting point in the classification and find that it overfit to the no damage class; therefore, we improved the base classification algorithm by modifying the top layers of ResNet50. We found that not all classes (destroyed, major damage, minor damage, and no damage) were uniformly identifiable across natural hazards; therefore, we retrained the weights from ImageNet, adding five new convolution, batch normalization, and max pooling layers on top of ResNet50. One dropout layer, with a rate of 0.5 was also added in-between the fully connected layers to reduce overfitting and improve performance. We also evaluate the identifiability of the four damage classes in the xbd dataset. Because classification performance was significantly higher for the “no damage” class as compared to “minor”, “major”, and “destroyed” classes, we evaluated merging classes. We kept the “no damage” class and created a second merged class (“damaged”) representing “minor damage,” “major damage,” and “destroyed.” We used the same architecture for the multiclass classification and the binary classification but without the ImageNet weights. Based on this work, we recommend that users be aware of performance differences across natural hazards and across damage classes. Earthquake building damage is extremely limited in the training data and, as a result, application of the trained algorithm on earthquake data cannot be evaluated given the xBD dataset. Building damage due to volcano and tsunami are also poorly represented in the training data, and do not have sufficient data for model validation (especially within all damage classes). Wind hazards are well-represented and therefore application of the algorithm trained using either the wind-only data or the multi-hazard dataset is reliable. The multi-class algorithm trained with wind hazard specific data slightly outperforms a multihazard trained multiclass model (F1 score 0.70 vs. 0.67). Both models have similar performance across all four classes (F1 > 0.5). For flood, fire, and tsunami hazards, we recommend using the binary damage classes as identifiability is low for at least two of the classes in each hazard. For flood building damage, binary classification performance resulted in a significantly higher F1 score when trained with the flood specific dataset versus the multihazard data (0.72 vs. 0.54). On the other hand, for fire building damage, classification performance is slightly higher when the model is trained on multi-hazard data, rather than trained using a fire specific dataset (F1 score 0.46 vs. 0.42).

Keywords: Damage assessment; Disaster response; Remote sensing; Machine learning; Deep learning; Neural networks; Convolution neural network; Imbalanced dataset (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11069-024-06500-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:nathaz:v:120:y:2024:i:9:d:10.1007_s11069-024-06500-9

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11069

DOI: 10.1007/s11069-024-06500-9

Access Statistics for this article

Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards is currently edited by Thomas Glade, Tad S. Murty and Vladimír Schenk

More articles in Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards from Springer, International Society for the Prevention and Mitigation of Natural Hazards
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().