Nonverbal Social Sensing: What Social Sensing Can and Cannot Do for the Study of Nonverbal Behavior From Video

Laetitia Aurelie Renier, Marianne Schmid Mast, Nele Dael and Emmanuelle Patricia Kleinlogel ()
Additional contact information
Laetitia Aurelie Renier: HEC Lausanne - Faculté des Hautes Etudes Commerciales (HEC Lausanne)
Marianne Schmid Mast: HEC Lausanne - Faculté des Hautes Etudes Commerciales (HEC Lausanne)
Nele Dael: HEC Lausanne - Faculté des Hautes Etudes Commerciales (HEC Lausanne)
Emmanuelle Patricia Kleinlogel: CEMOI - Centre d'Économie et de Management de l'Océan Indien - UR - Université de La Réunion, HEC Lausanne - Faculté des Hautes Etudes Commerciales (HEC Lausanne)

Post-Print from HAL

Abstract: The study of nonverbal behavior (NVB), and in particular kinesics (i.e., face and body motions), is typically seen as cost-intensive. However, the development of new technologies (e.g., ubiquitous sensing, computer vision, and algorithms) and approaches to study social behavior [i.e., social signal processing (SSP)] makes it possible to train algorithms to automatically code NVB, from action/motion units to inferences. Nonverbal social sensing refers to the use of these technologies and approaches for the study of kinesics based on video recordings. Nonverbal social sensing appears as an inspiring and encouraging approach to study NVB at reduced costs, making it a more attractive research field. However, does this promise hold? After presenting what nonverbal social sensing is and can do, we discussed the key challenges that researchers face when using nonverbal social sensing on video data. Although nonverbal social sensing is a promising tool, researchers need to be aware of the fact that algorithms might be as biased as humans when extracting NVB or that the automated NVB coding might remain context-dependent. We provided study examples to discuss these challenges and point to potential solutions.

Date: 2021-07-27
Note: View the original document on HAL open archive server: https://hal.univ-reunion.fr/hal-03665960
References: View complete reference list from CitEc
Citations:

Published in Frontiers in Psychology, 2021, 12, pp.606548. ⟨10.3389/fpsyg.2021.606548⟩

Downloads: (external link)
https://hal.univ-reunion.fr/hal-03665960/document (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hal:journl:hal-03665960

DOI: 10.3389/fpsyg.2021.606548

Access Statistics for this paper

More papers in Post-Print from HAL
Bibliographic data for series maintained by CCSD ().