Some independent agreements and resolved disagreements about answer‐providing documents

John O'Connor

American Documentation, 1969, vol. 20, issue 4, 311-319

Abstract: Eighty‐two documents and 30 questions, in documentation and related areas, were compared to find answer‐providing documents (documents from which answers to questions can be inferred). Two judges (documentation experts) made comparisons independently. They discussed their disagreements, attempting to resolve them. In each case the positive judge (who had independently judged a document answer‐providing) was first asked to indicate what answer he inferred, and from what document passage(s). The further discussion depended on the details of each case. There were 32 independent agreements on positive judgments. There were 48 disagreements between independent judgments, all resolved by discussion. Thirty‐four resolutions were agreements on positive judgments, accomplished by pointing out overlooked passages, unnoticed connections, or alternative meanings. Fourteen resolutions were agreements on negative judgments, accomplished by pointing out document misinterpretations, the challenged positive judge being unable to describe an inference and joint work not finding one, or agreement that both judges lacked sufficient background knowledge. In general, the resolution procedures used will resolve a disagreement about whether a document is answerproviding or reduce it to a familiar kind of scientific disagreement (about a passage's meaning, a statement's correctness, or an inference's correctness). This seems better than treating relevance judgements as subjective and not open to rational discussion.

Date: 1969
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/asi.4630200405

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:amedoc:v:20:y:1969:i:4:p:311-319

Ordering information: This journal article can be ordered from
https://doi.org/10.1002/(ISSN)1936-6108

Access Statistics for this article

American Documentation is currently edited by Javed Mostafa

More articles in American Documentation from Wiley Blackwell
Bibliographic data for series maintained by Wiley Content Delivery ().