Rethinking online monitoring methods for information retrieval systems: From search product to search process

Christine L. Borgman, Sandra G. Hirsh and John Hiller

Journal of the American Society for Information Science, 1996, vol. 47, issue 7, 568-583

Abstract: Searching information retrieval systems is a highly interactive, iterative process that cannot be understood simply by comparing the output of a search session (the “search product”) to a query stated in advance. In this article, we examine evaluation goals and methods for studying information retrieval behavior, drawing examples from our own research and that of others. We limit our review to research that employs online monitoring, also known as transaction log analysis. Online monitoring is one of few methods that can capture detailed data on the search process at a reasonable cost; these data can be used to build quantitative models or to support qualitative interpretations of quantitative results. Monitoring is a data collection technique rather than a research design, and can be employed in experimental or field studies, whether alone or combined with other data collection methods. Based on the research questions of interest, the researcher must determine what variables to collect from each data source, which to treat as independent variables to manipulate, and which to treat as dependent variables to observe effects. Studies of searching behavior often treat search task and searcher characteristics as independent variables and may manipulate other independent variables specific to the research questions addressed. Search outcomes, time, and search paths frequently are treated as dependent variables. We discuss each of these sets of variables, illustrating them with sample results from the literature and from our own research. Our examples are drawn from the Science Library Catalog project, a 7‐year study of children's searching behavior on an experimental retrieval system. We close with a brief discussion of the implications of these results for the design of information retrieval systems. © 1996 John Wiley & Sons, Inc.

Date: 1996
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://doi.org/10.1002/(SICI)1097-4571(199607)47:73.0.CO;2-S

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jamest:v:47:y:1996:i:7:p:568-583

Ordering information: This journal article can be ordered from
https://doi.org/10.1002/(ISSN)1097-4571

Access Statistics for this article

More articles in Journal of the American Society for Information Science from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().