Storage and Indexing of Fine Grain, Large Scale Data Sets

Ralf Schneider ()
Additional contact information
Ralf Schneider: High Performance Computing Center Stuttgart

A chapter in Sustained Simulation Performance 2013, 2013, pp 89-104 from Springer

Abstract: Abstract In this work a storage scheme for fine grain data sets, which are large in absolute size as well as in file number, is presented. It is meant to deal in parallel with data arising from simulation tasks utilizing high performance computing systems for the independent parallel execution of several thousand serial tasks, resulting in large data sets of terabyte size spread over hundreds of thousands or even millions of files.First of all a short introduction is given to the problem of the multiple parallel execution of serial programs on high performance computing systems. This leads to the description of the idea how to realize a data storage and indexing concept to handle fine grain, large scale data sets. Afterwards the implementation in form of a FORTRAN 2003 library is presented along with the use cases evaluated in the results section.

Keywords: Large-scale Data Sets; Index Concept; High Performance Computer; Simulated Load Cases; Data Storage Concept (search for similar items in EconPapers)
Date: 2013
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-319-01439-5_7

Ordering information: This item can be ordered from
http://www.springer.com/9783319014395

DOI: 10.1007/978-3-319-01439-5_7

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().