Your Face Mirrors Your Deepest Beliefs—Predicting Personality and Morals through Facial Emotion Recognition

Peter A. Gloor, Andrea Fronzetti Colladon, Erkin Altuntas, Cengiz Cetinkaya, Maximilian F. Kaiser, Lukas Ripperger and Tim Schaefer
Additional contact information
Peter A. Gloor: MIT Center for Collective Intelligence, Cambridge, MA 02142, USA
Andrea Fronzetti Colladon: Department of Engineering, University of Perugia, 06123 Perugia, Italy
Erkin Altuntas: Galaxyadvisors AG, 5000 Aarau, Switzerland
Cengiz Cetinkaya: Department of Data Science, Lucerne University of Applied Sciences and Arts, 6002 Lucerne, Switzerland
Maximilian F. Kaiser: Department of Information Systems, University of Cologne, 50923 Cologne, Germany
Lukas Ripperger: Department of Information Systems, University of Cologne, 50923 Cologne, Germany
Tim Schaefer: Department of Information Systems, University of Cologne, 50923 Cologne, Germany

Future Internet, 2021, vol. 14, issue 1, 1-18

Abstract: Can we really “read the mind in the eyes”? Moreover, can AI assist us in this task? This paper answers these two questions by introducing a machine learning system that predicts personality characteristics of individuals on the basis of their face. It does so by tracking the emotional response of the individual’s face through facial emotion recognition (FER) while watching a series of 15 short videos of different genres. To calibrate the system, we invited 85 people to watch the videos, while their emotional responses were analyzed through their facial expression. At the same time, these individuals also took four well-validated surveys of personality characteristics and moral values: the revised NEO FFI personality inventory, the Haidt moral foundations test, the Schwartz personal value system, and the domain-specific risk-taking scale (DOSPERT). We found that personality characteristics and moral values of an individual can be predicted through their emotional response to the videos as shown in their face, with an accuracy of up to 86% using gradient-boosted trees. We also found that different personality characteristics are better predicted by different videos, in other words, there is no single video that will provide accurate predictions for all personality characteristics, but it is the response to the mix of different videos that allows for accurate prediction.

Keywords: artificial intelligence; facial emotion recognition; personality; moral values; risk-taking; forecasting (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/14/1/5/pdf (application/pdf)
https://www.mdpi.com/1999-5903/14/1/5/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:14:y:2021:i:1:p:5-:d:709020

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().