On Frequency Estimation and Detection of Heavy Hitters in Data Streams

Federica Ventruto, Marco Pulimeno, Massimo Cafaro and Italo Epicoco
Additional contact information
Federica Ventruto: Dpartment of Engineering for Innovation, University of Salento, 73100 Lecce, Italy
Marco Pulimeno: Dpartment of Engineering for Innovation, University of Salento, 73100 Lecce, Italy
Massimo Cafaro: Dpartment of Engineering for Innovation, University of Salento, 73100 Lecce, Italy
Italo Epicoco: Dpartment of Engineering for Innovation, University of Salento, 73100 Lecce, Italy

Future Internet, 2020, vol. 12, issue 9, 1-20

Abstract: A stream can be thought of as a very large set of data, sometimes even infinite, which arrives sequentially and must be processed without the possibility of being stored. In fact, the memory available to the algorithm is limited and it is not possible to store the whole stream of data which is instead scanned upon arrival and summarized through a succinct data structure in order to maintain only the information of interest. Two of the main tasks related to data stream processing are frequency estimation and heavy hitter detection. The frequency estimation problem requires estimating the frequency of each item, that is the number of times or the weight with which each appears in the stream, while heavy hitter detection means the detection of all those items with a frequency higher than a fixed threshold. In this work we design and analyze ACMSS, an algorithm for frequency estimation and heavy hitter detection, and compare it against the state of the art AS ketch algorithm. We show that, given the same budgeted amount of memory, for the task of frequency estimation our algorithm outperforms AS ketch with regard to accuracy. Furthermore, we show that, under the assumptions stated by its authors, AS ketch may not be able to report all of the heavy hitters whilst ACMSS will provide with high probability the full list of heavy hitters.

Keywords: data stream mining; heavy hitters; frequency estimation; sketches (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2020
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/12/9/158/pdf (application/pdf)
https://www.mdpi.com/1999-5903/12/9/158/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:12:y:2020:i:9:p:158-:d:415780

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().