Multimodal AI for Landslide Disaster Response: Fusing IoT Prediction, UAV Detection, and Social Media Triage

Yash V. Shinde, Jay J. Shinde and Tejas V. Joshi

International Journal of Scientific Research in Science and Technology, 2026, vol. 13, issue 3, 1082-1093

Abstract: Finding a way to predict landslides, detect the physical damage, and organize rescues on the fly is incredibly difficult for emergency teams. This paper presents "Multimodal AI for Landslide Disaster Response: Fusing IoT Prediction, UAV Detection, and Social Media Triage", an end-to-end machine learning framework designed to manage the complete timeline of an emergency. Rather than just sticking to one specific data type, the approach we took breaks disaster management down into three separate phases. To start the pipeline, we apply a Deep Neural Network (DNN) to IoT sensor data so we can forecast landslide risks well before the soil actually gives way. Right after an event hits, a Vision Transformer (ViT-B/16) steps in to look over incoming drone imagery and spot any structural destruction. From there, the system handles the logistics of the aftermath by merging computer vision (ResNet50) with natural language processing (DistilBERT) to read through distress posts that citizens put on social media. To ensure rescue teams are deployed effectively, we incorporated the YOLOv8x algorithm to physically count the number of victims directly from uploaded images. We realized that blindly trusting a neural network's classification is dangerous, so our software actually relies on YOLO's bounding-box tensors to act as a strict safety check. Because of this built-in rule, if a social media post claims there is a critical emergency but the object detector spots zero victims in the photo, the system actively blocks the automated dispatch. This effectively stops AI hallucinations and keeps rescue resources from being wasted. When we ran the evaluations, our sensor prediction model hit 97.8% accuracy, and the drone image classifier came in at 91.42%. These integrated models operate within a single Streamlit dashboard, providing a clear and reliable tool for emergency dispatchers. Future expansions of this work could involve integrating retrieval-augmented generation (RAG) and autonomous AI agents to further optimize rescue routing.

Keywords: Landslide Disaster Response; Disaster Response Fusing; Response Fusing IoT; Fusing IoT Prediction; IoT Prediction UAV (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://ijsrst.com/home/article/view/IJSRST26133243 Abstract page (text/html)
https://ijsrst.com/home/article/download/IJSRST26133243/IJSRST26133243 Full text (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:etm:ijsrst:v13:y2026:i3:id:1700

DOI: 10.32628/IJSRST26133243

Access Statistics for this article

More articles in International Journal of Scientific Research in Science and Technology from Technoscience Academy
Bibliographic data for series maintained by Pankaj Sharma ().