Driver Frustration Detection from Audio and Video in the Wild

Abdic, Irman; Fridman, Lex; McDuff, Daniel; Marchi, Erik; Reimer, Bryan; Schuller, Björn W.

Driver Frustration Detection from Audio and Video in the Wild

Irman Abdic, Lex Fridman, Daniel McDuff, Erik Marchi, Bryan Reimer, Björn W. Schuller

Download BibTeX Cite

Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. Pages 1354-1359.

Abstract

We present a method for detecting driver frustration from both video and audio streams captured during the driver's interaction with an in-vehicle voice-based navigation system. The video is of the driver's face when the machine is speaking, and the audio is of the driver's voice when he or she is speaking. We analyze a dataset of 20 drivers that contains 596 audio epochs (audio clips, with duration from 1 sec to 15 sec) and 615 video epochs (video clips, with duration from 1 sec to 45 sec). The dataset is balanced across 2 age groups, 2 vehicle systems, and both genders. The model was subject-independently trained and tested using 4-fold cross-validation. We achieve an accuracy of 77.4% for detecting frustration from a single audio epoch and 81.2% for detecting frustration from a single video epoch. We then treat the video and audio epochs as a sequence of interactions and use decision fusion to characterize the trade-off between decision time and classification accuracy, which improved the prediction accuracy to 88.5% after 9 epochs.

Irman Abdic, Lex Fridman, Daniel McDuff, Erik Marchi, Bryan Reimer, and Björn W. Schuller. 2016. Driver Frustration Detection from Audio and Video in the Wild. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-16. IJCAI, 1354–1359. https://ijcai.org/proceedings/2016/195

@inproceedings{ijcai2016p195,
  title     = {Driver Frustration Detection from Audio and Video in the Wild},
  author    = {Abdic, Irman and Fridman, Lex and McDuff, Daniel and Marchi, Erik and Reimer, Bryan and Schuller, Björn W.},
  booktitle = {Proceedings of the Twenty-Fifth International Joint Conference on
               Artificial Intelligence, {IJCAI-16}},
  publisher = {AAAI Press / International Joint Conferences on Artificial Intelligence},
  editor    = {Subbarao Kambhampati},
  pages     = {1354--1359},
  year      = {2016},
  month     = {7},
  url       = {https://ijcai.org/proceedings/2016/195},
}

TY  - CPAPER
TI  - Driver Frustration Detection from Audio and Video in the Wild
AU  - Abdic, Irman
AU  - Fridman, Lex
AU  - McDuff, Daniel
AU  - Marchi, Erik
AU  - Reimer, Bryan
AU  - Schuller, Björn W.
T2  - Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-16
PB  - AAAI Press / International Joint Conferences on Artificial Intelligence
PY  - 2016
SP  - 1354
EP  - 1359
UR  - https://ijcai.org/proceedings/2016/195
ER  -