Integrating SDN-Enhanced MPI with Job Scheduler to Support Shared Clusters

Keichi Takahashi (), Susumu Date (), Yasuhiro Watashiba (), Yoshiyuki Kido () and Shinji Shimojo ()
Additional contact information
Keichi Takahashi: Nara Institute of Science and Technology
Susumu Date: Cybermedia Center, Osaka University
Yasuhiro Watashiba: Cybermedia Center, Osaka University
Yoshiyuki Kido: Cybermedia Center, Osaka University
Shinji Shimojo: Cybermedia Center, Osaka University

A chapter in Sustained Simulation Performance 2018 and 2019, 2020, pp 149-159 from Springer

Abstract: Abstract SDN-enhanced MPI is a framework that integrates the network programmability of Software-Defined Networking (SDN) with Message Passing Interface (MPI). The aim of SDN-enhanced MPI is to improve MPI communication performance by dynamically steering the traffic within the interconnect based on the communication pattern of applications. A major limitation in the current implementation of SDN-enhanced MPI is that multiple jobs cannot be executed concurrently. This paper removes this limitation by integrating SDN-enhanced MPI with the job scheduler of the cluster. Specifically, we have developed a plugin for the job scheduler that collects and reports the job information to the interconnect controller. A preliminary evaluation demonstrated that applications could gain up to 2.56× speedup in communication.

Date: 2020
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-030-39181-2_13

Ordering information: This item can be ordered from
http://www.springer.com/9783030391812

DOI: 10.1007/978-3-030-39181-2_13

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().