An Analytical Approach for Optimizing the Performance of Hadoop Map Reduce Over RoCE

Geetha J., Uday Bhaskar N and Chenna Reddy P.
Additional contact information
Geetha J.: M.S. Ramaih Institute of Technology, Bengaluru, India
Uday Bhaskar N: Government College (UG & PG), Anantapur, India
Chenna Reddy P.: JNTU-Anantapur, Anatapur, India

International Journal of Information Communication Technologies and Human Development (IJICTHD), 2018, vol. 10, issue 2, 1-14

Abstract: Data intensive systems aim to efficiently process “big” data. Several data processing engines have evolved over past decade. These data processing engines are modeled around the MapReduce paradigm. This article explores Hadoop's MapReduce engine and propose techniques to obtain a higher level of optimization by borrowing concepts from the world of High Performance Computing. Consequently, power consumed and heat generated is lowered. This article designs a system with a pipelined dataflow in contrast to the existing unregulated “bursty” flow of network traffic, the ability to carry out both Map and Reduce tasks in parallel, and a system which incorporates modern high-performance computing concepts using Remote Direct Memory Access (RDMA). To establish the claim of an increased performance measure of the proposed system, the authors provide an algorithm for RoCE enabled MapReduce and a mathematical derivation contrasting the runtime of vanilla Hadoop. This article proves mathematically, that the proposed system functions 1.67 times faster than the vanilla version of Hadoop.

Date: 2018
References: Add references at CitEc
Citations:

Downloads: (external link)
http://services.igi-global.com/resolvedoi/resolve. ... 8/IJICTHD.2018040101 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jicthd:v:10:y:2018:i:2:p:1-14

Access Statistics for this article

International Journal of Information Communication Technologies and Human Development (IJICTHD) is currently edited by Hakikur Rahman

More articles in International Journal of Information Communication Technologies and Human Development (IJICTHD) from IGI Global
Bibliographic data for series maintained by Journal Editor ().