Variability of a job search indicator induced by operationalization decisions when using digital traces from a meter

Carlos Ochoa and Melanie Revilla

PLOS ONE, 2025, vol. 20, issue 12, 1-19

Abstract: Digital traces —particularly metered data—offer researchers a valuable alternative to surveys for studying online behavior. However, because the concepts being measured are not directly observable in the data, their operationalization requires multiple decisions—for example, which events (e.g., visited websites) represent the concepts and which metrics (e.g., visit counts or time spent) capture their intensity. Using metered data from 600 Netquest panelists in Spain, this study investigates how operationalization choices affect the measurement of job search intensity. By varying metrics—combinations of measurement targets (such as sessions on job platforms or job offer pages) and measurement types (such as visit counts or time spent)—along with other factors (e.g., methods for separating search activity into spells or handling outliers), the study explores 10,080 operationalizations. Results reveal significant variability, with correlations between measurement pairs ranging from 0.14 to 0.91. Metrics sharing the same measurement target (e.g., sessions, job offer pages) demonstrate stronger convergent validity than those sharing only the same measurement type (e.g., time or visits). Other operationalization factors, such as session segmentation methods, also influence results, though less than metric choice. Importantly, operationalization decisions can affect substantive findings.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0338894 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 38894&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0338894

DOI: 10.1371/journal.pone.0338894

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().