Robust Survey Aggregation with Student-t Distribution and Sparse Representation

Qingtao Tang, Tao Dai, Li Niu, Yisen Wang, Shu-Tao Xia, Jianfei Cai

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence

Main track. Pages 2829-2835.

https://doi.org/10.24963/ijcai.2017/394

PDF BibTeX

Abstract

Most existing survey aggregation methods assume that the sample data follow Gaussian distribution. However, these methods are sensitive to outliers, due to the thin-tailed property of the Gaussian distribution. To address this issue, we propose a robust survey aggregation method based on Student-t distribution and sparse representation. Specifically, we assume that the samples follow Student-$t$ distribution, instead of the common Gaussian distribution. Due to the Student-t distribution, our method is robust to outliers, which can be explained from both Bayesian point of view and non-Bayesian point of view. In addition, inspired by James-Stain estimator (JS) and Compressive Averaging (CAvg), we propose to sparsely represent the global mean vector by an adaptive basis comprising both data-specific basis and combined generic bases. Theoretically, we prove that JS and CAvg are special cases of our method. Extensive experiments demonstrate that our proposed method achieves significant improvement over the state-of-the-art methods on both synthetic and real datasets.

Keywords

  • Machine Learning: Data Mining
  • Machine Learning: Machine Learning
Welcome Future Conferences AI Journal Division IJCAI Proceedings Travel Grant Program IJCAI Awards