Online MapReduce scheduling problem of minimizing the makespan

Cong Chen (), Yinfeng Xu (), Yuqing Zhu () and Chengyu Sun ()
Additional contact information
Cong Chen: Xi’an Jiaotong University
Yinfeng Xu: Xi’an Jiaotong University
Yuqing Zhu: California State University
Chengyu Sun: California State University

Journal of Combinatorial Optimization, 2017, vol. 33, issue 2, No 14, 590-608

Abstract: Abstract MapReduce system is a popular big data processing framework, and the performance of it is closely related to the efficiency of the centralized scheduler. In practice, the centralized scheduler often has little information in advance, which means each job may be known only after being released. In this paper, hence, we consider the online MapReduce scheduling problem of minimizing the makespan, where jobs are released over time. Both preemptive and non-preemptive version of the problem are considered. In addition, we assume that reduce tasks cannot be parallelized because they are often complex and hard to be decomposed. For the non-preemptive version, we prove the lower bound is $$\frac{m+m(\Psi (m)-\Psi (k))}{k+m(\Psi (m)-\Psi (k))}$$ m + m ( Ψ ( m ) - Ψ ( k ) ) k + m ( Ψ ( m ) - Ψ ( k ) ) , higher than the basic online machine scheduling problem, where k is the root of the equation $$k=\big \lfloor {\frac{m-k}{1+\Psi (m)-\Psi (k)}+1 }\big \rfloor $$ k = ⌊ m - k 1 + Ψ ( m ) - Ψ ( k ) + 1 ⌋ and m is the quantity of machines. Then we devise an $$(2-\frac{1}{m})$$ ( 2 - 1 m ) -competitive online algorithm called MF-LPT (Map First-Longest Processing Time) based on the LPT. For the preemptive version, we present a 1-competitive algorithm for two machines.

Keywords: Online algorithm; MapReduce scheduling; Makespan; Big data (search for similar items in EconPapers)
Date: 2017
References: View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://link.springer.com/10.1007/s10878-015-9982-7 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:jcomop:v:33:y:2017:i:2:d:10.1007_s10878-015-9982-7

Ordering information: This journal article can be ordered from
https://www.springer.com/journal/10878

DOI: 10.1007/s10878-015-9982-7

Access Statistics for this article

Journal of Combinatorial Optimization is currently edited by Thai, My T.

More articles in Journal of Combinatorial Optimization from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().