Debugging Big Data Systems for Big Data Analytics

Ümit Demirbaga, Gagangeet Singh Aujla (), Anish Jindal () and Oğuzhan Kalyon ()
Additional contact information
Ümit Demirbaga: University of Cambridge, Department of Medicine
Gagangeet Singh Aujla: Durham University, Department of Computer Science
Anish Jindal: Durham University, Department of Computer Science
Oğuzhan Kalyon: Newcastle University, Faculty of Medical Sciences

Chapter Chapter 8 in Big Data Analytics, 2024, pp 171-192 from Springer

Abstract: Abstract This chapter unveils the intricate art of debugging big data systems for optimal analytics performance, providing a comprehensive guide to navigating real-world performance challenges. The exploration commences by delineating the critical debugging steps essential for identifying and resolving issues within big data systems. Focussing on the specific problems that can afflict these systems, such as data locality, resource heterogeneity, network issues, resource over-allocation, unnecessary speculation, and poor scheduling policies, the chapter dives into the intricacies of root cause analysis. Emphasising the importance of this analysis in the context of big data analytics, the narrative elucidates the systematic steps involved, accompanied by insightful details on tools and techniques, challenges, and considerations. The chapter explores available diagnosis tools tailored for big data systems, including Mantri, Texas Advanced Computing Centre (TACC) Stats, Data Centre Data Base (DCDB) Wintermute, and AutoDiagn, empowering practitioners to effectively diagnose and address complex issues in their analytics infrastructure.

Date: 2024
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-031-55639-5_8

Ordering information: This item can be ordered from
http://www.springer.com/9783031556395

DOI: 10.1007/978-3-031-55639-5_8

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().