Models for Internal Clustering Validation Indexes Based on Hadoop-MapReduce

Soumeya Zerabi, Souham Meshoul and Samia Chikhi Boucherkha
Additional contact information
Soumeya Zerabi: Abdelhamid Mehri Constantine 2 University, Algeria
Souham Meshoul: Princess Noura University, Saudi Arabia
Samia Chikhi Boucherkha: Abdelhamid Mehri Constantine 2 University, Algeria

International Journal of Distributed Systems and Technologies (IJDST), 2020, vol. 11, issue 3, 42-67

Abstract: Cluster validation aims to both evaluate the results of clustering algorithms and predict the number of clusters. It is usually achieved using several indexes. Traditional internal clustering validation indexes (CVIs) are mainly based in computing pairwise distances which results in a quadratic complexity of the related algorithms. The existing CVIs cannot handle large data sets properly and need to be revisited to take account of the ever-increasing data set volume. Therefore, design of parallel and distributed solutions to implement these indexes is required. To cope with this issue, the authors propose two parallel and distributed models for internal CVIs namely for Silhouette and Dunn indexes using MapReduce framework under Hadoop. The proposed models termed as MR_Silhouette and MR_Dunn have been tested to solve both the issue of evaluating the clustering results and identifying the optimal number of clusters. The results of experimental study are very promising and show that the proposed parallel and distributed models achieve the expected tasks successfully.

Date: 2020
References: Add references at CitEc
Citations:

Downloads: (external link)
http://services.igi-global.com/resolvedoi/resolve. ... 018/IJDST.2020070103 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jdst00:v:11:y:2020:i:3:p:42-67

Access Statistics for this article

International Journal of Distributed Systems and Technologies (IJDST) is currently edited by Nik Bessis

More articles in International Journal of Distributed Systems and Technologies (IJDST) from IGI Global
Bibliographic data for series maintained by Journal Editor ().