TECHNOLOGIES FOR LARGE DATA MANAGEMENT IN SCIENTIFIC COMPUTING

Alberto Pace ()
Additional contact information
Alberto Pace: Data and Storage Services Group, CERN IT Department, CH-1211 Geneva, Switzerland

International Journal of Modern Physics C (IJMPC), 2014, vol. 25, issue 02, 1-72

Abstract: In recent years, intense usage of computing has been the main strategy of investigations in several scientific research projects. The progress in computing technology has opened unprecedented opportunities for systematic collection of experimental data and the associated analysis that were considered impossible only few years ago.This paper focuses on the strategies in use: it reviews the various components that are necessary for an effective solution that ensures the storage, the long term preservation, and the worldwide distribution of large quantities of data that are necessary in a large scientific research project.The paper also mentions several examples of data management solutions used in High Energy Physics for the CERN Large Hadron Collider (LHC) experiments in Geneva, Switzerland which generate more than 30,000 terabytes of data every year that need to be preserved, analyzed, and made available to a community of several tenth of thousands scientists worldwide.

Keywords: Data management; storage; data; scientific computing; big-data; cloud; cloud storage; cloud computing; data analysis; data technologies; hierarchical storage; CERN; LHC; Large Hadron Collider; ATLAS; CMS; ALICE; LHCB; data reliability; data availability; metadata management; data preservation; quality of service; storage pools; data replication; data syncing; data integrity; erasure codes; error correction; data encryption; data security; name server; identity federations; data confidentiality; access control; quota; data mining; cache algorithm; DHT; Distributed hash tables; storage federations; 07.05.Kf; 89.20.Ff; 07.05.Hd; 07.05.-t (search for similar items in EconPapers)
Date: 2014
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.worldscientific.com/doi/abs/10.1142/S0129183114300012
Access to full text is restricted to subscribers

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:ijmpcx:v:25:y:2014:i:02:n:s0129183114300012

Ordering information: This journal article can be ordered from

DOI: 10.1142/S0129183114300012

Access Statistics for this article

International Journal of Modern Physics C (IJMPC) is currently edited by H. J. Herrmann

More articles in International Journal of Modern Physics C (IJMPC) from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().