Fusion of Word and Letter Based Metrics for Automatic MT Evaluation

Yang, Muyun; Zhu, Junguo; Li, Sheng; Zhao, Tiejun

Fusion of Word and Letter Based Metrics for Automatic MT Evaluation

Muyun Yang, Junguo Zhu, Sheng Li, Tiejun Zhao

Download BibTeX Cite

Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. Pages 2204-2209.

Abstract

With the progress in machine translation, it becomes more subtle to develop the evaluation metric capturing the systems' differences in comparison to the human translations. In contrast to the current efforts in leveraging more linguistic information to depict translation quality, this paper takes the thread of combining language independent features for a robust solution to MT evaluation metric. To compete with finer granularity of modeling brought by linguistic features, the proposed method augments the word level metrics by a letter based calculation. An empirical study is then conducted over WMT data to train the metrics by ranking SVM. The results reveal that the integration of current language independent metrics can generate well enough performance for a variety of languages. Time-split data validation is promising as a better training setting, though the greedy strategy also works well.

Muyun Yang, Junguo Zhu, Sheng Li, and Tiejun Zhao. 2013. Fusion of Word and Letter Based Metrics for Automatic MT Evaluation. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI-13. IJCAI, 2204–2209. https://ijcai.org/proceedings/2013/325

@inproceedings{ijcai2013p325,
  title     = {Fusion of Word and Letter Based Metrics for Automatic MT Evaluation},
  author    = {Yang, Muyun and Zhu, Junguo and Li, Sheng and Zhao, Tiejun},
  booktitle = {Proceedings of the Twenty-Third International Joint Conference on
               Artificial Intelligence, {IJCAI-13}},
  publisher = {AAAI Press / International Joint Conferences on Artificial Intelligence},
  editor    = {Francesca Rossi},
  pages     = {2204--2209},
  year      = {2013},
  month     = {8},
  url       = {https://ijcai.org/proceedings/2013/325},
}

TY  - CPAPER
TI  - Fusion of Word and Letter Based Metrics for Automatic MT Evaluation
AU  - Yang, Muyun
AU  - Zhu, Junguo
AU  - Li, Sheng
AU  - Zhao, Tiejun
T2  - Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI-13
PB  - AAAI Press / International Joint Conferences on Artificial Intelligence
PY  - 2013
SP  - 2204
EP  - 2209
UR  - https://ijcai.org/proceedings/2013/325
ER  -