Proceedings

A Two-Stage Method for Active Learning of Statistical Grammars

Download BibTeX Cite

Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence. Pages 991-995.

© 2005 International Joint Conferences on Artificial Intelligence

Abstract

Active learning reduces the amount of manually annotated sentences necessary when training state-ofthe-art statistical parsers. One popular method, uncertainty sampling, selects sentences for which the parser exhibits low certainty. However, this method does not quantify confidence about the current statistical model itself. In particular, we should be less confident about selection decisions based on low frequency events. We present a novel twostage method which first targets sentences which cannot be reliably selected using uncertainty sampling, and then applies standard uncertainty sampling to the remaining sentences. An evaluation shows that this method performs better than pure uncertainty sampling, and better than an ensemble method based on bagged ensemble members only.

  1. [Argamon-Engelson and Dagan, 1999] Shlomo Argamon- Engelson and Ido Dagan. Committee-based sample selection for probabilistic classifiers. Journal of Artificial Intelligence Research, 11:335-360, 1999.
  2. [Bikel, 2004] Daniel M. Bikel. Intricacies of Collins' parsing model. Computational Linguistics, 30(4):479-511, 2004.
  3. [Black et al., 1991] E. Black, S. Abney, D. Flickinger, C. Gdaniec, R. Grishman, P. Harrison, D. Hindle, R. In- gria, F. Jelinek, J. Klavans, M. Liberman, M. Marcus, S. Roukos, B. Santorini, and T. Strzalkowski. A procedure for quantitatively comparing the syntactic coverage of En- glish grammars. In Proceedings, Speech and Natural Lan- guage Workshop, 1991.
  4. [Brants, 2000] Thorsten Brants. TnT -a statistical part-of- speech tagger. In Proceedings of the 6th Applied Natural Language Processing Conference, 2000.
  5. [Breiman, 1996] Leo Breiman. Bagging predictors. Machine Learning, 24(2):123-14, 1996.
  6. [Charniak, 2000] Eugene Charniak. A maximum-entropy- inspired parser. In Proceedings of the 1st Conference of the North American Chapter of the Association for Com- putational Linguistics, 2000.
  7. [Collins, 1997] Michael Collins. Three generative, lexi- calised models for statistical parsing. In Proceedings of the 35th Annual Meeting of the Association for Computa- tional Linguistics, 1997.
  8. [Henderson and Brill, 2000] John C. Henderson and Eric Brill. Bagging and boosting a treebank parser. In Proceed- ings of the 1st Conference of the North American Chapter of the Association for Computational Linguistics, 2000.
  9. [Hwa, 2000] Rebecca Hwa. Sample selection for statistical grammar induction. In Proceedings of the 2000 Joint SIG- DAT Conference on Empirical Methods in Natural Lan- guage Processing and Very Large Corpora, pages 45-52, 2000.
  10. [Hwa, 2001] Rebecca Hwa. Learning Probabilistic Lexical- ized Grammars for Natural Language Processing. PhD thesis, Harvard University, 2001.
  11. [Lewis and Gale, 1994] David D. Lewis and William A. Gale. A sequential algorithm for training text classifiers. In Proceedings of SIGIR-94, 17th ACM International Con- ference on Research and Development in Information Re- trieval, pages 3-12, 1994.
  12. [Marcus et al., 1993] Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. Building a large annotated corpus of English: The Penn treebank. Computational Lin- guistics, 19(2):313-330, 1993.
  13. [McCallum and Nigam, 1998] Andrew McCallum and Ka- mal Nigam. Employing EM and pool-based active learn- ing for text classification. In Proceedings of the 15th In- ternational Conference on Machine Learning, pages 350- 358, 1998.
  14. [Pereira et al., 1993] Fernando C.N. Pereira, Naftali Tishby, and Lillian Lee. Distributional clustering of English words. In Proceedings of the 31st Annual Meeting of the Association for Computational Linguistics, 1993.
  15. [Seung et al., 1992] H. Sebastian Seung, Manfred Opper, and Haim Sompolinsky. Query by committee. In Com- putational Learning Theory, pages 287-294, 1992.
  16. [Tang et al., 2002] M. Tang, X. Luo, and S. Roukos. Active learning for statistical natural language parsing. In Pro- ceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 120-127, 2002.
  17. [Thompson et al., 1999] C. A. Thompson, M. E. Califf, and R. J. Mooney. Active learning for natural language parsing and information extraction. In Proceedings of the 16th In- ternational Conference on Machine Learning, pages 406- 414, 1999.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards