 Big data block impact within big data environmentRon Ziv, Oded Koren and Nir Perel International Journal of Information Technology and Management, 2023, vol. 22, issue 1/2, 140-159 Abstract: Handling data is becoming more and more complex. A higher velocity of data is created as more people have access to data generating devices such as computers, mobile phones, medical devices, home appliances, etc. Data files, such as user activity logs, system logs and so on, are stored in HDFS big data platform in various sizes, which takes into consideration the business requirements, infrastructure parameters, administration decisions, and other factors. Dividing the data files (in various volumes) without taking into consideration the HDFS™ predefined block size, may create performance issues that can affect the system's activity. This paper presents how HDFS™ block design affects the performance of Apache™ Hadoop® big data environment by testing different architectures for reading, writing, and querying identical datasets. We designed three scenarios to illustrate different file divisions on the big data platform. The findings present a significant impact on the performance of a system in accordance with the architecture deployed. Keywords: information management; architecture; HDFS; performance; big data; block partition. (search for similar items in EconPapers) Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:ids:ijitma:v:22:y:2023:i:1/2:p:140-159 Access Statistics for this article More articles in International Journal of Information Technology and Management from Inderscience Enterprises Ltd |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.