A versatile information retrieval framework for evaluating profile strength and similarity

Alexandr A. Kalinin, John Arevalo, Erik Serrano, Loan Vulliard, Hillary Tsang, Michael Bornholdt, Alán F. Muñoz, Suganya Sivagurunathan, Bartek Rajwa, Anne E. Carpenter, Gregory P. Way () and Shantanu Singh ()
Additional contact information
Alexandr A. Kalinin: Broad Institute of MIT and Harvard
John Arevalo: Broad Institute of MIT and Harvard
Erik Serrano: University of Colorado School of Medicine
Loan Vulliard: German Cancer Research Center (DKFZ)
Hillary Tsang: Broad Institute of MIT and Harvard
Michael Bornholdt: Broad Institute of MIT and Harvard
Alán F. Muñoz: Broad Institute of MIT and Harvard
Suganya Sivagurunathan: Broad Institute of MIT and Harvard
Bartek Rajwa: Purdue University
Anne E. Carpenter: Broad Institute of MIT and Harvard
Gregory P. Way: University of Colorado School of Medicine
Shantanu Singh: Broad Institute of MIT and Harvard

Nature Communications, 2025, vol. 16, issue 1, 1-17

Abstract: Abstract Large-scale profiling assays capture a cell population’s state by measuring thousands of biological properties per cell or sample. However, evaluating profile strength and similarity remains challenging due to the high dimensionality and non-linear, heterogeneous nature of measurements. Here, we develop a statistical framework using mean average precision (mAP) as a single, data-driven metric to address this challenge. We validate the mAP framework against established metrics through simulations and real-world data, revealing its ability to capture subtle and meaningful biological differences in cell state. Specifically, we use mAP to assess a sample’s phenotypic activity relative to controls, as well as the phenotypic consistency of groups of perturbations (or samples). We evaluate the framework across diverse datasets and on different profile types (image, protein, mRNA), perturbations (CRISPR, gene overexpression, small molecules), and resolutions (single-cell, bulk). The mAP framework, together with our open-source software package copairs, is useful for evaluating high-dimensional profiling data in biological research and drug discovery.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.nature.com/articles/s41467-025-60306-2 Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-025-60306-2

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-025-60306-2

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().