---
title: "Human Action Recognition Using a Temporal Hierarchy of Covariance Descriptors on 3D Joint Locations"
authors:
  - "Mohamed Elsayed Hussein"
  - "Marwan Torki"
  - "Mohammad Abdelaziz Gowayyed"
  - "Motaz El-Saban"
doi: ""
url: "https://ijcai.org/proceedings/2013/363"
pdf: "https://ijcai.org/proceedings/2013/0363.pdf"
venue: "Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence"
year: 2013
pages: "2466-2471"
track: ""
keywords:
---

# Human Action Recognition Using a Temporal Hierarchy of Covariance Descriptors on 3D Joint Locations

Mohamed Elsayed Hussein, Marwan Torki, Mohammad Abdelaziz Gowayyed, Motaz El-Saban

*Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence*, pages 2466-2471.

## Abstract

Human action recognition from videos is a challenging machine vision task with multiple important application domains, such as human-robot/machine interaction, interactive entertainment, multimedia information retrieval, and surveillance. In this paper, we present a novel approach to human action recognition from 3D skeleton sequences extracted from depth data. We use the covariance matrix for skeleton joint locations over time as a discriminative descriptor for a sequence. To encode the relationship between joint movement and time, we deploy multiple covariance matrices over sub-sequences in a hierarchical fashion. The descriptor has a fixed length that is independent from the length of the described sequence. Our experiments show that using the covariance descriptor with an off-the-shelf classification algorithm outperforms the state of the art in action recognition on multiple datasets, captured either via a Kinect-type sensor or a sophisticated motion capture system. We also include an evaluation on a novel large dataset using our own annotation.