Economics at your fingertips  

A novel method for estimating the common signals for consensus across multiple ranked lists

Vendula Švendová and Michael G. Schimek

Computational Statistics & Data Analysis, 2017, vol. 115, issue C, 122-135

Abstract: The ranking of objects, such as journals, institutions or biological entities, is broadly used to assess the relative quality or relevance of such objects. A multiple ranking is performed by a number of assessors (humans or machines) and inference about the nature of the observed rankings is desirable for evaluation, business or scientific purposes. The assessors’ decisions are based on some inherent metric scale and depend on judgement and discriminatory ability, data to which we usually do not have access. An indirect inference approach is proposed that allows one to estimate those signal parameters that might be causal for the observed rankings obtained from several assessors, some of which may not necessarily provide the same decision quality. The order of the values represents a consensus ranking across the observed individual rankings. The standard errors of the estimated signal parameters are obtained through a non-parametric bootstrap. Hence, the signal variability can be evaluated object-wise for the purpose of quantifying the stability of the associated rank positions. As a result, such signal estimates can be used in the meta-analysis of conceptually similar evaluation exercises, studies or experiments, and in any data integration task where measurements on the metric scale are either unavailable, or not directly comparable. The suggested approach is validated on simulated rank data as well as on experimental rank data from current molecular medicine. The proposed algorithms were implemented and all calculations performed in the R environment. The source code is provided.

Keywords: Bootstrap; Distribution function; Indirect inference; Rank data; Signal estimation; Simulation (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations Track citations by RSS feed

Downloads: (external link)
Full text for ScienceDirect subscribers only.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link:

Access Statistics for this article

Computational Statistics & Data Analysis is currently edited by S.P. Azen

More articles in Computational Statistics & Data Analysis from Elsevier
Series data maintained by Dana Niculescu ().

Page updated 2017-10-01
Handle: RePEc:eee:csdana:v:115:y:2017:i:c:p:122-135