New ways of specifying data edits

George Petrakos, Claudio Conversano, Gregory Farmakis, Francesco Mola, Roberta Siciliano and Photis Stavropoulos

Journal of the Royal Statistical Society Series A, 2004, vol. 167, issue 2, 249-274

Abstract: Summary. Data editing is the process by which data that are collected in some way (a statistical survey for example) are examined for errors and corrected with the help of software. Edits, the logical conditions that should be satisfied by the data, are specified by subject‐matter experts with a procedure which could be tedious and could lead to mistakes with practical implications. To render the process of edit specification more efficient we provide a new step—the definition of the so‐called abstract data model of a survey—which describes the structure of the phenomenon that is studied in a survey. The existence of this model enables experts to identify all combinations of variables which should be checked by edits and to avoid the definition of conflicting edits. Furthermore, we introduce an automatic data validation strategy—TREEVAL—that consists of fast tree growing to derive automatically the functional form of edits and of a statistical criterion to clean the incoming data. The TREEVAL strategy is cast within a total quality management framework. The application of the methodologies proposed is demonstrated with the help of a real life application.

Date: 2004
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://doi.org/10.1046/j.1467-985X.2003.00745.x

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jorssa:v:167:y:2004:i:2:p:249-274

Ordering information: This journal article can be ordered from
http://ordering.onli ... 1111/(ISSN)1467-985X

Access Statistics for this article

Journal of the Royal Statistical Society Series A is currently edited by A. Chevalier and L. Sharples

More articles in Journal of the Royal Statistical Society Series A from Royal Statistical Society Contact information at EDIRC.
Bibliographic data for series maintained by Wiley Content Delivery ().