Predicting Human Strategic Decisions Using Facial Expressions / 2035
Noam Peled, Moshe Bitan, Joseph Keshet, Sarit Kraus

People's facial expressions, whether made consciously or subconsciously, continuously reveal their state of mind. This work proposes a method for predicting people's strategic decisions based on their facial expressions. We designed a new version of the centipede game that introduces an incentive for the human participant to hide her facial expressions. We recorded on video participants who played several games of our centipede version, and concurrently logged their decisions throughout the games. The video snippet of the participants' faces prior to their decisions is represented as a fixed-size vector by estimating the covariance matrix of key facial points which change over time. This vector serves as input to a classifier that is trained to predict the participant's decision. We compare several training techniques, all of which are designed to work with the imbalanced decisions typically made by the players of the game. Furthermore, we investigate adaptation of the trained model to each player individually, while taking into account the player's facial expressions in the previous games. The results show that our method outperforms standard SVM as well as humans in predicting subjects'strategic decisions. To the best of our knowledge, this is the first study to present a methodology for predicting people's strategic decisions when there is an incentive to hide facial expressions.