Performance Evaluation of Query Plan Recommendation with Apache Hadoop and Apache Spark

Elham Azhir, Mehdi Hosseinzadeh, Faheem Khan () and Amir Mosavi ()
Additional contact information
Elham Azhir: Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran 1477893855, Iran
Mehdi Hosseinzadeh: Institute of Research and Development, Duy Tan University, Da Nang 550000, Vietnam
Faheem Khan: Department of Computer Engineering, Gachon University, Seongnam 13120, Korea
Amir Mosavi: Faculty of Civil Engineering, Technische Universität Dresden, 01069 Dresden, Germany

Mathematics, 2022, vol. 10, issue 19, 1-11

Abstract: Access plan recommendation is a query optimization approach that executes new queries using prior created query execution plans (QEPs). The query optimizer divides the query space into clusters in the mentioned method. However, traditional clustering algorithms take a significant amount of execution time for clustering such large datasets. The MapReduce distributed computing model provides efficient solutions for storing and processing vast quantities of data. Apache Spark and Apache Hadoop frameworks are used in the present investigation to cluster different sizes of query datasets in the MapReduce-based access plan recommendation method. The performance evaluation is performed based on execution time. The results of the experiments demonstrated the effectiveness of parallel query clustering in achieving high scalability. Furthermore, Apache Spark achieved better performance than Apache Hadoop, reaching an average speedup of 2x.

Keywords: access plan recommendation; parallel processing; Apache Hadoop; Apache Spark; big data; artificial intelligence; soft computing; cloud computing; data science; MapReduce (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/19/3517/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/19/3517/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:19:p:3517-:d:926004

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().