Detection of stragglers and optimal rescheduling of slow running tasks in big data environment using LFCSO-LVQ classifier and enhanced PSO algorithm

Hetal A. Joshiara, Chirag S. Thaker, Sanjay M. Shah and Darshan B. Choksi

International Journal of Data Analysis Techniques and Strategies, 2022, vol. 14, issue 1, 1-21

Abstract: This paper plans to implement intelligent techniques in finding straggler tasks along with speculating their way of execution. Here, the LFCSO-LVQ is proposed to effectively identify the slow running tasks as of a bunch of user tasks, and the enhanced PSO is proposed for performing optimal rescheduling of the identified SR tasks. Initially, the collected data are preprocessed by means of identifying homogenous and heterogeneous tasks. After that, the Apache Spark split the preprocessed tasks into several sub-tasks. The features are extracted as of these subtasks for SR task prediction. An information gain-based linear discriminant analysis is proposed for feature selection approach that reduces the classifier's training time. Subsequent to FS, the selected ones are inputted to LFCSO-LVQ, which envisages the SR tasks of the dataset centred on the chosen features. After that, EPSO reschedules these predicted tasks to the other fastest nodes of virtual machine.

Keywords: big data; straggler detection; rescheduling of tasks; speculative execution; slow running tasks identification; learning vector quantisation; LVQ; optimal resource scheduling; particle swarm optimisation; PSO. (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.inderscience.com/link.php?id=121505 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ids:injdan:v:14:y:2022:i:1:p:1-21

Access Statistics for this article

More articles in International Journal of Data Analysis Techniques and Strategies from Inderscience Enterprises Ltd
Bibliographic data for series maintained by Sarah Parker ().