Dynamic Mapreduce for Job Workloads through Slot Configuration Technique

M. Priyanka and A. Subramanyam

International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 2017, vol. 2, issue 4, 695-699

Abstract: The MapReduce is an open source Hadoop framework implemented for processing and producing distributed large Terabyte data on large clusters. Its primary duty is to minimize the completion time of large sets of MapReduce jobs. Hadoop Cluster only has predefined fixed slot configuration for cluster lifetime. This fixed slot configuration may produce long completion time (Makespan) and low system resource utilization. The current open source Hadoop allows only static slot configuration, like fixed numbers of map slots and reduce slots throughout the cluster lifetime. Such static configuration may lead to long completion length as well as low system resource utilizations. Propose new schemes which use slot ratio between map and reduce tasks as a tunable knob for minimizing the completion length (i.e., makespan) of a given set. By leveraging the workload information of recently completed jobs, schemes dynamically allocates resources (or slots) to map and reduce tasks.. Many scheduling methodologies are discussed that aim to improve execution performance as well as completion time goal.

Keywords: Map Reduce; Makespan; Workload; Dynamic Slot Allocation. (search for similar items in EconPapers)
Date: 2017
Note: Article URL: https://ijsrcseit.com/CSEIT1172488
References: Add references at CitEc
Citations:

Downloads: (external link)
https://ijsrcseit.com/CSEIT1172488 Article URL (text/html)
https://ijsrcseit.com/paper/CSEIT1172488.pdf Full text (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:jbh:ijsrcs:v2:y2017:i4:id:hcseit1172488

Access Statistics for this article

More articles in International Journal of Scientific Research in Computer Science, Engineering and Information Technology from International Journal of Scientific Research in Computer Science, Engineering and Information Technology
Bibliographic data for series maintained by Pankaj Sharma ().