Simulating the Performance of Estimators of the Number of Duplicates in a Database

John C. Handley () and John A. Bunge ()
Additional contact information
John C. Handley: OCLC Online Computer Library Center, Office of Research
John A. Bunge: Cornell University, Department of Economic and Social Statistics

A chapter in Computing Science and Statistics, 1992, pp 366-369 from Springer

Abstract: Abstract A large database consisting of N records is partitioned into K equivalence classes, which are sets of duplicate records. It is desired to estimate K, the number of unique records, based on a small sampling fraction of either individual records or complete equivalence classes. There exists a unique unbiased estimator based on a simple random sample of individual records, and an asymptotically unbiased estimator exists when the sample consists of complete classes drawn without replacement. In both cases, the variance of the estimators must be obtained through simulation. A UNIX software system was written to explore the variance of these estimators for large database-type populations with many singletons. The software system consists of population generators, an efficient sampling program, and programs to compute estimators. Simulation results for both estimators are presented which show that the variance of the unbiased estimator based on sampling individuals is too great to be useful for small sampling fractions but that good estimates can be obtained sampling by class.

Keywords: Class Size; Unbiased Estimator; Sampling Program; Individual Record; Complete Classis (search for similar items in EconPapers)
Date: 1992
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-1-4612-2856-1_55

Ordering information: This item can be ordered from
http://www.springer.com/9781461228561

DOI: 10.1007/978-1-4612-2856-1_55

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().