Considering the subjectivity to rationalise evaluation approaches: the example of "Spoken Dialogue Systems"

Marianne Laurent (), Philippe Bretier and Ioannis Kanellos ()
Additional contact information
Marianne Laurent: Orange Labs [Lannion] - France Télécom
Philippe Bretier: Orange Labs [Lannion] - France Télécom
Ioannis Kanellos: INFO - Département informatique - UEB - Université européenne de Bretagne - European University of Brittany - Télécom Bretagne - IMT - Institut Mines-Télécom [Paris]

Post-Print from HAL

Abstract: We present in this paper a reading grid aiming at helping the evaluator take into account the subjectivity factor when designing evaluation protocols. Actually, however most contributions to Spoken Dialogue Systems evaluation tend to objectify their approach for rationalising purpose, we believe subjectivity is to be considered to enforce valuable evaluations. The first section shows how closely evaluation processes are dependant on their contexts and on the evaluators' perspectives. We then present an anthropocentric framework that establishes the evaluator as a mediator between the consideration of contextual elements and a rationalising corpus of evaluation procedures. We finally anticipate the benefits brought by our framework at both individual and community levels.

Keywords: SDS; Evaluation; Context; Practice; Subjectivity (search for similar items in EconPapers)
Date: 2010-06-21
Note: View the original document on HAL open archive server: https://hal.science/hal-02295316v1
References: View complete reference list from CitEc
Citations:

Published in Qomex 2010 : Second International Workshop on Quality of Multimedia Experience, Jun 2010, Trondheim, Norway

Downloads: (external link)
https://hal.science/hal-02295316v1/document (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hal:journl:hal-02295316

Access Statistics for this paper

More papers in Post-Print from HAL
Bibliographic data for series maintained by CCSD ().