Automatic, Multimodal Evaluation of Human Interaction

Matthew L. Jensen (), Thomas O. Meservy (), Judee K. Burgoon () and Jay F. Nunamaker ()
Additional contact information
Matthew L. Jensen: University of Oklahoma
Thomas O. Meservy: University of Memphis
Judee K. Burgoon: University of Arizona
Jay F. Nunamaker: University of Arizona

Group Decision and Negotiation, 2010, vol. 19, issue 4, No 4, 367-389

Abstract: Abstract This article outlines an approach for automatically extracting behavioral indicators from video, audio, and text and explores the possibility of using those indicators to predict human-interpretable judgments of involvement, dominance, tension, and arousal. We utilized two-dimensional spatial inputs extracted from video, acoustic properties extracted from audio and verbal content transcribed from face-to-face interactions to construct a set of multimodal features. Multiple predictive models were created using the extracted features as predictors and human-coded perceptions of involvement, tenseness, and arousal as the criterion. These predicted perceptions were then used as independent variables in classifying truth and deception. Though the predicted values for perceptions performed comparably to human-coded perceptions in detecting deception, the results were not satisfying. Thus, the extracted multimodal features were used to predict deception directly. Classification accuracy was substantially higher than typical human deception detection performance. Through this research, we consider the feasibility and validity of the approach and identify how such an approach could contribute to the broader community.

Keywords: Deception; Deception detection; Credibility; Credibility assessment; Multimodal behavior analysis (search for similar items in EconPapers)
Date: 2010
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s10726-009-9171-0 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:grdene:v:19:y:2010:i:4:d:10.1007_s10726-009-9171-0

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10726/PS2

DOI: 10.1007/s10726-009-9171-0

Access Statistics for this article

Group Decision and Negotiation is currently edited by Gregory E. Kersten

More articles in Group Decision and Negotiation from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().