A Cached-Based Stream-Relation Join Operator for Semi-Stream Data Processing

M. Asif Naeem, Imran Sarwar Bajwa and Noreen Jamil
Additional contact information
M. Asif Naeem: School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology, Auckland, New Zealand
Imran Sarwar Bajwa: School of Computer Science, University of Birmingham, Birmingham, UK
Noreen Jamil: School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology, Auckland, New Zealand

International Journal of Data Warehousing and Mining (IJDWM), 2016, vol. 12, issue 3, 14-31

Abstract: Stream-based join algorithms got a prominent role in the field of real-time data warehouses. One particular type of stream-based joins is a semi-stream join where a single stream is joined with a disk -based relation. Normally the size of this disk-based relation is large enough and cannot be fit into memory, available for join operator. Therefore, the relation is loaded into memory in partitions. A well-known join algorithm called MESHJOIN (Mesh Join) has been presented in the literature to process semi-stream data. However, the algorithm has some limitations. In particular, MESHJOIN does not consider the characteristics of stream data and therefore does not perform well for skewed stream data. This article introduces the concept of caching and based on that presents a novel algorithm called Cached-based Stream-Relation Join (CSRJ). The algorithm exploits skewed distributions more appropriately, and the authors present results for Zipfian distributions of the type that appear in many applications. They test their algorithm using synthetic, TPC-H and real datasets. Their experiments show that CSRJ forms significantly better than MESHJOIN. They also drive the cost model for their algorithm and based on that they tune the algorithm.

Date: 2016
References: Add references at CitEc
Citations:

Downloads: (external link)
http://services.igi-global.com/resolvedoi/resolve. ... 018/IJDWM.2016070102 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jdwm00:v:12:y:2016:i:3:p:14-31

Access Statistics for this article

International Journal of Data Warehousing and Mining (IJDWM) is currently edited by Eric Pardede

More articles in International Journal of Data Warehousing and Mining (IJDWM) from IGI Global
Bibliographic data for series maintained by Journal Editor ().