Extending the Rule Space Methodology to a Semantically-Rich Domain: Diagnostic Assessment in Architecture

Irvin R. Katz, Michael E. Martinez, Kathleen M. Sheehan and Kikumi K. Tatsuoka

Journal of Educational and Behavioral Statistics, 1998, vol. 23, issue 3, 254-278

Abstract: This paper presents a technique for applying the Rule Space methodology of cognitive diagnosis to assessment in a semantically-rich domain. Previous applications of Rule Space—all in simple, well-structured domains—based diagnosis on examinees’ ability to perform individual problem-solving steps. In a complex domain, however, test items might be so different from one another that the problem-solving steps used for one item are unrelated to the steps used to solve another item. The technique presented herein extends Rule Space’s applicability by basing diagnosis on item characteristics that are more abstract than individual problem-solving steps. A cognitive model of problem-solving motivates selection of characteristics in order to maintain the connection between an examinee’s problem-solving skill and his/her diagnosis. To test the extended Rule Space procedure, data were collected from 122 architects of three ability levels (students, architecture interns, and professional architects) on a 22-item test of architectural knowledge. Rule Space provided diagnostic reporting for between 40 and 90% of examinees. The findings support the effectiveness of Rule Space in a complex domain.

Keywords: Keywords: cognitive modeling; diagnostic assessment; item attributes; Rule Space methodology (search for similar items in EconPapers)
Date: 1998
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.3102/10769986023003254 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:jedbes:v:23:y:1998:i:3:p:254-278

DOI: 10.3102/10769986023003254

Access Statistics for this article

More articles in Journal of Educational and Behavioral Statistics
Bibliographic data for series maintained by SAGE Publications ().