Short and Sparse Text Topic Modeling via Self-Aggregation

Quan, Xiaojun; Kit, Chunyu; Ge, Yong; Pan, Sinno Jialin

Short and Sparse Text Topic Modeling via Self-Aggregation

Xiaojun Quan, Chunyu Kit, Yong Ge, Sinno Jialin Pan

Download BibTeX Cite

Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 2270-2275.

Abstract

The overwhelming amount of short text data on social media and elsewhere has posed great challenges to topic modeling due to the sparsity problem. Most existing attempts to alleviate this problem resort to heuristic strategies to aggregate short texts into pseudo-documents before the application of standard topic modeling. Although such strategies cannot be well generalized to more general genres of short texts, the success has shed light on how to develop a generalized solution. In this paper, we present a novel model towards this goal by integrating topic modeling with short text aggregation during topic inference. The aggregation is founded on general topical affinity of texts rather than particular heuristics, making the model readily applicable to various short texts. Experimental results on real-world datasets validate the effectiveness of this new model, suggesting that it can distill more meaningful topics from short texts.

Xiaojun Quan, Chunyu Kit, Yong Ge, and Sinno Jialin Pan. 2015. Short and Sparse Text Topic Modeling via Self-Aggregation. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15. IJCAI, 2270–2275. https://ijcai.org/proceedings/2015/321

@inproceedings{ijcai2015p321,
  title     = {Short and Sparse Text Topic Modeling via Self-Aggregation},
  author    = {Quan, Xiaojun and Kit, Chunyu and Ge, Yong and Pan, Sinno Jialin},
  booktitle = {Proceedings of the Twenty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-15}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Qiang Yang and Michael Wooldridge},
  pages     = {2270--2275},
  year      = {2015},
  month     = {7},
  note      = {Main Track},
  url       = {https://ijcai.org/proceedings/2015/321},
}

TY  - CPAPER
TI  - Short and Sparse Text Topic Modeling via Self-Aggregation
AU  - Quan, Xiaojun
AU  - Kit, Chunyu
AU  - Ge, Yong
AU  - Pan, Sinno Jialin
T2  - Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2015
SP  - 2270
EP  - 2275
UR  - https://ijcai.org/proceedings/2015/321
ER  -