Fusion of Multiple Features and Supervised Learning for Chinese OOV Term Detection and POS Guessing

Zhang, Yuejie; Cen, Lei; Wu, Wei; Jin, Cheng; Xue, Xiangyang

doi:10.5591/978-1-57735-516-8/IJCAI11-321

Fusion of Multiple Features and Supervised Learning for Chinese OOV Term Detection and POS Guessing

Yuejie Zhang, Lei Cen, Wei Wu, Cheng Jin, Xiangyang Xue

Download BibTeX Cite

Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence. Pages 1921-1925.

https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-321

Abstract

In this paper, to support more precise Chinese Out-of-Vocabulary (OOV) term detection and Part-of-Speech (POS) guessing, a unified mechanism is proposed and formulated based on the fusion of multiple features and supervised learning. Besides all the traditional features, the new features for statistical information and global contexts are introduced, as well as some constraints and heuristic rules, which reveal the relationships among OOV term candidates. Our experiments on the Chinese corpora from both People’s Daily and SIGHAN 2005 have achieved the consistent results, which are better than those acquired by pure rule-based or statistics-based models. From the experimental results for combining our model with Chinese monolingual retrieval on the data sets of TREC-9, it is found that the obvious improvement for the retrieval performance can also be obtained.

Yuejie Zhang, Lei Cen, Wei Wu, Cheng Jin, and Xiangyang Xue. 2011. Fusion of Multiple Features and Supervised Learning for Chinese OOV Term Detection and POS Guessing. In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, IJCAI-11. IJCAI, 1921–1925. https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-321

@inproceedings{ijcai2011p321,
  title     = {Fusion of Multiple Features and Supervised Learning for Chinese OOV Term Detection and POS Guessing},
  author    = {Zhang, Yuejie and Cen, Lei and Wu, Wei and Jin, Cheng and Xue, Xiangyang},
  booktitle = {Proceedings of the Twenty-Second International Joint Conference on
               Artificial Intelligence, {IJCAI-11}},
  publisher = {AAAI Press/International Joint Conferences on Artificial Intelligence},
  editor    = {Toby Walsh},
  pages     = {1921--1925},
  year      = {2011},
  month     = {7},
  doi       = {10.5591/978-1-57735-516-8/IJCAI11-321},
  url       = {https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-321},
}

TY  - CPAPER
TI  - Fusion of Multiple Features and Supervised Learning for Chinese OOV Term Detection and POS Guessing
AU  - Zhang, Yuejie
AU  - Cen, Lei
AU  - Wu, Wei
AU  - Jin, Cheng
AU  - Xue, Xiangyang
T2  - Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, IJCAI-11
PB  - AAAI Press/International Joint Conferences on Artificial Intelligence
PY  - 2011
SP  - 1921
EP  - 1925
DO  - 10.5591/978-1-57735-516-8/IJCAI11-321
UR  - https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-321
ER  -