Spoken conversational search: Evaluating the effect of system clarifications on user experience through Wizard‐of‐Oz study

Souvick Ghosh and Chirag Shah

Journal of the Association for Information Science & Technology, 2025, vol. 76, issue 5, 819-839

Abstract: Prior research in human–computer interaction suggests that system‐level clarifications are necessary for understanding user intent and communicating effectively with the user. Such clarifications or explanations could contain the system's abstract knowledge of the search or a functional description of the search process (queries and information sources employed). While these interactions may aid the user and the agent in better understanding each other, very few studies have explored the influence of such clarifications on the users' search experience. This research examines whether and how system‐level clarifications (or explanations) affect the user experience when searching through spoken dialogues. We analyzed user satisfaction and preferences in systems with and without explicit clarifications in a within‐subjects Wizard‐of‐Oz user study. We recruited 25 participants and collected user–system interaction data for 50 search sessions. The user feedback was collected using pre‐ and post‐task surveys and exit interviews. Statistical and qualitative analysis of user responses yielded some interesting findings. While Wilcoxon Signed Rank Test found that using explicit system‐level clarifications had no positive influence on the user's search experience, the overall search experience degraded with system clarifications (Z = −2.066, p = 0.04). The user interview data provided valuable insights into how and when clarifications should be offered to the user.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/asi.24974

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jinfst:v:76:y:2025:i:5:p:819-839

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=2330-1635

Access Statistics for this article

More articles in Journal of the Association for Information Science & Technology from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().