Human Action Recognition Using a Temporal Hierarchy of Covariance Descriptors on 3D Joint Locations

Hussein, Mohamed Elsayed; Torki, Marwan; Gowayyed, Mohammad Abdelaziz; El-Saban, Motaz

Human Action Recognition Using a Temporal Hierarchy of Covariance Descriptors on 3D Joint Locations

Mohamed Elsayed Hussein, Marwan Torki, Mohammad Abdelaziz Gowayyed, Motaz El-Saban

Download BibTeX Cite

Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. Pages 2466-2471.

Abstract

Human action recognition from videos is a challenging machine vision task with multiple important application domains, such as human-robot/machine interaction, interactive entertainment, multimedia information retrieval, and surveillance. In this paper, we present a novel approach to human action recognition from 3D skeleton sequences extracted from depth data. We use the covariance matrix for skeleton joint locations over time as a discriminative descriptor for a sequence. To encode the relationship between joint movement and time, we deploy multiple covariance matrices over sub-sequences in a hierarchical fashion. The descriptor has a fixed length that is independent from the length of the described sequence. Our experiments show that using the covariance descriptor with an off-the-shelf classification algorithm outperforms the state of the art in action recognition on multiple datasets, captured either via a Kinect-type sensor or a sophisticated motion capture system. We also include an evaluation on a novel large dataset using our own annotation.

Mohamed Elsayed Hussein, Marwan Torki, Mohammad Abdelaziz Gowayyed, and Motaz El-Saban. 2013. Human Action Recognition Using a Temporal Hierarchy of Covariance Descriptors on 3D Joint Locations. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI-13. IJCAI, 2466–2471. https://ijcai.org/proceedings/2013/363

@inproceedings{ijcai2013p363,
  title     = {Human Action Recognition Using a Temporal Hierarchy of Covariance Descriptors on 3D Joint Locations},
  author    = {Hussein, Mohamed Elsayed and Torki, Marwan and Gowayyed, Mohammad Abdelaziz and El-Saban, Motaz},
  booktitle = {Proceedings of the Twenty-Third International Joint Conference on
               Artificial Intelligence, {IJCAI-13}},
  publisher = {AAAI Press / International Joint Conferences on Artificial Intelligence},
  editor    = {Francesca Rossi},
  pages     = {2466--2471},
  year      = {2013},
  month     = {8},
  url       = {https://ijcai.org/proceedings/2013/363},
}

TY  - CPAPER
TI  - Human Action Recognition Using a Temporal Hierarchy of Covariance Descriptors on 3D Joint Locations
AU  - Hussein, Mohamed Elsayed
AU  - Torki, Marwan
AU  - Gowayyed, Mohammad Abdelaziz
AU  - El-Saban, Motaz
T2  - Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI-13
PB  - AAAI Press / International Joint Conferences on Artificial Intelligence
PY  - 2013
SP  - 2466
EP  - 2471
UR  - https://ijcai.org/proceedings/2013/363
ER  -