Proceedings

A Weighted Polynomial Information Gain Kernel for Resolving Prepositional Phrase Attachment Ambiguities with Support Vector Machines

Download BibTeX Cite

Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence. Pages 133-139.

© 2003 International Joint Conferences on Artificial Intelligence

Abstract

We introduce a new kernel for Support Vector Machine learning in a natural language setting. As a case study to incorporate domain knowledge into a kernel, we consider the problem of resolving Prepositional Phrase attachment ambiguities. The new kernel is derived from a distance function that proved to be succesful in memory-based learning. We start with the Simple Overlap Metric from which we derive a Simple Overlap Kernel and extend it with Information Gain Weighting. Finally, we combine it with a polynomial kernel to increase the dimensionality of the feature space. The closure properties of kernels guarantee that the result is again a kernel. This kernel achieves high classification accuracy and is efficient in both time and space usage. We compare our results with those obtained by memory-based and other learning methods. They make clear that the proposed kernel achieves a higher classification accuracy.

  1. [Aha eta/., 1991] D. Aha, D. Kibler, and M. Albert. Instance based learning algorithms. Machine Learning, 6:37-66, 1991.
  2. [Brown eta/., 1999] Michael P.S. Brown, William Noble Grundy, David Lin, Nello Cristianini, Charles Sugnet, Manuel Ares, and David Haussler. Support vector machine classification of microarray gene expression data, 1999. [Chih-Chung and Chi-Jen, 2002] Chang Chih-Chung and Lin Chi-Jen. Libsvm: a library for support vector ma- chines, 2002.
  3. [Collins and Brooks, 1995] M.J Collins and J. Brooks. Prepositional phrase attachment through a backed-off model. In Proceedings of the Third Workshop on Very Large Corpora, Cambridge, 1995.
  4. [Cost and Salzberg, 1993] Scott Cost and Steven Salzberg. A weighted nearest neighbour algorithm for learning with symbolic features . Machine Learning, 10:57-78, 1993.
  5. [Cristianini and Shawe-Taylor, 2000] Nello Cristianini and John Shawe-Taylor. An Introduction to Support Vec- tor Machines and other Kernel-based Learning Methods. Cambridge University Press, 2000.
  6. [Daelemans eta/., 2002] Walter Daelemans, Jakub Zavrel, Ko van der Sloot, and Antal van den Bosch. Timbl: Tilburg memory-based learner, version 4.3. Technical re- port, Til burg University and University of Antwerp, 2002.
  7. [Marcus eta/., 1993] M. Marcus, Santorini B, and M.A. Marcinkiewicz. Building a large annotated corpus of en- glish: The penn treebank. Computational Linguistics, 19(2):313-330, 1993.
  8. [Mitchell, 1997] Tom Mitchell. Machine Learning. The McGraw-Hill Companies, Inc., 1997.
  9. [Ramon, 2002] Jan Ramon. Clustering and instance based learning in first order logic. PhD thesis, K.U. Leuven, Belgium, 2002. [Ratnaparkhi eta/., 1994]
  10. Adwait Ratnaparkhi, J. Reynar, and S. Roukos. A maximum entropy model for prepo- sitional phrase attachment. In Proceedings of the ARPA Workshop on Human Language Technology, Plainsboro, NJ, 1994.
  11. [Ratnaparkhi, 1998] Adwait Ratnaparkhi. Maximum En- tropy Models for Natural Language Ambiguity Resolution. PhD thesis, University of Pennsylvania, Philadelphia, PA, 1998.
  12. [Sang, 2002] Erik F. Tjong Kim Sang. Introduction to the conll-2002 shared task: Language-independent named en- tity recognition. In Proceedings of CoNLL-2002, Taipei, Taiwan, pages 155-158, 2002.
  13. [Stetina and Nagao, 1997] Jiri Stetina and Makoto Nagao. Corpus based pp attachment ambiguity resolution with a semantic dictionary., 1997. Kyoto University, Yoshida Honmachi, Kyoto 606, Japan.
  14. [Vapnik, 1998] Vladimir Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, New York, 1998.
  15. [Veenstra eta/., 2000] Jorn Veenstra, Antal van den Bosch, Sabine Buchholz, Walter Daelemans, and Jakub Zavrel. Memory-based word senese disambiguation. Computers and the Humanities, 34: 1-2: 171 -177, 2000.
  16. [Zavrel eta/., 1997] Jakub Zavrel, Walter Daelemans, and Jorn Veenstra. Resolving pp attachment ambiguities with memory-based learning. In Proceedings CoNNL, Madrid, pages 136-144. Computational Linguistics, Tilburg Uni- versity, 1997.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards