Enhanced Resource Efficiency for Association Rule Mining in Cloud Environments via Apache Spark

Sohrab Ansari and Vinod Mahor

International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 2023, vol. 9, issue 6, 369-381

Abstract: Data from various sources, including mobile devices, sensors, and web cams, constantly accumulates and is evaluated in Big Data. These processed data are crucial in various fields, such as research, business, and industry. Apache Spark is a versatile platform for processing both batch and real-time data. Cloud computing provides resources for real-time processing of applications. Association Rule Mining (ARM) is a technology that analyzes the link between objects to identify comparable groupings. FP-Growth is the most widely used algorithm for finding common patterns and locating mining pieces quickly. The aim of this research is to enhance the efficiency of Association Rule Mining by creating rules for big data sets in Big Data environments. The proposed solution enhances association rule efficiency by utilizing the FP-Growth algorithm in a Hadoop Map Reduce setting. FP-Growth is the most used method for discovering and mining frequent patterns. This research introduces the FP-Growth parallel method in Spark Framework. The efficient use of Spark resources through heterogeneous allocation reduces runtime and costs. Apache Spark is a versatile Big Data platform for real-time streaming and batch processing. Cloud computing is used in streaming applications to address real-time processing needs by supplying necessary resources. Using big data apps in a virtualized cloud environment may cause performance issues that impact streaming workloads. The ARM approach identifies highly associated models in item sets. FP expanding is the most common ARM algorithm. The FP-Growth algorithm is implemented in Spark using OpenStack. This article covers OpenStack architecture, needs, configuration, and problems. The analysis evaluates resource utility at full load and no load, and evaluates performance using virtual resource allocation. Using Spark resources efficiently reduces turnaround time and optimizes costs due to their diverse distribution.

Keywords: Association Rule Mining; Cloud System; FP; Big data. (search for similar items in EconPapers)
Date: 2023
Note: Article URL: https://ijsrcseit.com/CSEIT23906780
References: Add references at CitEc
Citations:

Downloads: (external link)
https://ijsrcseit.com/CSEIT23906780 Article URL (text/html)
https://ijsrcseit.com/paper/CSEIT23906780.pdf Full text (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:jbh:ijsrcs:v9:y2023:i6:id:hcseit23906780

Access Statistics for this article

More articles in International Journal of Scientific Research in Computer Science, Engineering and Information Technology from International Journal of Scientific Research in Computer Science, Engineering and Information Technology
Bibliographic data for series maintained by Pankaj Sharma (USA) ().