A Survey on Out-of-Distribution Evaluation of Neural NLP Models

Xinzhe Li, Ming Liu, Shang Gao, Wray Buntine

Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence

Survey Track. Pages 6683-6691.

https://doi.org/10.24963/ijcai.2023/749

PDF BibTeX

Abstract

Adversarial robustness, domain generalization and dataset biases are three active lines of research contributing to out-of-distribution (OOD) evaluation on neural NLP models.

However, a comprehensive, integrated discussion of the three research lines is still lacking in the literature. This survey will 1) compare the three lines of research under a unifying definition; 2) summarize their data-generating processes and evaluation protocols for each line of research; and 3) emphasize the challenges and opportunities for future work.

Keywords

  • Survey: Natural Language Processing
  • Survey: Machine Learning
  • Survey: Uncertainty in AI
  • Survey: AI Ethics, Trust, Fairness
Welcome Future Conferences AI Journal Division IJCAI Proceedings Travel Grant Program IJCAI Awards