Proceedings

Multi-Class Classifiers and their Underlying Shared Structure

Download BibTeX Cite

Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence. Pages 1267-1271.

© 2009 International Joint Conferences on Artificial Intelligence

Abstract

Multi-class problems have a richer structure than binary classification problems. Thus, they can potentially improve their performance by exploiting the relationship among class labels. While for the purposes of providing an automated classification result this class structure does not need to be explicitly unveiled, for human level analysis or interpretation this is valuable. We develop a multi-class large margin classifier that extracts and takes advantage of class relationships. We provide a bi-convex formulation that explicitly learns a matrix that captures these class relationships and is de-coupled from the feature weights. Our representation can take advantage of the class structure to compress themodel by reducing the number of classifiers employed, maintaining high accuracy even with large compression. In addition, we present an efficient formulation in terms of speed and memory.Volkan Vural, Glenn Fung, Romer Rosales, Jennifer G. Dy

  1. [Al-Khayyal and Falk, 1983] F. A. Al-Khayyal and J. E. Falk. Jointly constrained biconvex programming. MOR, 8(2):273-286, 1983.
  2. [Allwein et al., 2000] E. L. Allwein, R. E. Schapire, and Y. Singer. Reducing multiclass to binary: A unifying approach for margin classifiers. JMLR, pages 113-141, 2000.
  3. [Amit et al., 2007] Y. Amit, M. Fink, N. Srebro, and S. Ullman. Uncovering shared structures in multiclass classification. In ICML 2007, pages 17-24, 2007.
  4. [Bezdek and Hathaway, 2003] J. Bezdek and R. Hathaway. Conver- gence of alternating optimization. Neural, Parallel Sci. Comput., 11(4):351-368, 2003.
  5. [Bordes et al., 2007] A. Bordes, L. Bottou, P. Gallinari, and J. We- ston. Solving multiclass support vector machines with larank. In ICML 07, pages 89-96, 2007.
  6. [Bredensteiner and Bennett, 1999] E. Bredensteiner and K. Ben- nett. Multicategory classification by support vector machines. Computational Optimization and Applications, 12:53-79, 1999.
  7. [Crammer and Singer, 2001] K. Crammer and Y. Singer. On the al- gorithmic implementation of multiclass kernel-based vector ma- chines. JMLR, 2:265-292, 2001.
  8. [Friedman, 1996] J. Friedman. Another approach to polychoto- mous classifcation. Technical report, Stanford University, De- partment of Statistics, 1996.
  9. [Fung and Mangasarian, 2001] Glenn Fung and O. L. Mangasarian. Proximal support vector machine classifiers. In Knowledge Dis- covery and Data Mining, pages 77-86, 2001.
  10. [Fung et al., 2004] Glenn Fung, Murat Dundar, Jinbo Bi, and Bharat Rao. A fast iterative algorithm for fisher discriminant us- ing heterogeneous kernels. In ICML '04, page 40, 2004.
  11. [Lee et al., 2004] Y. Lee, Y. Lin, and G. Wahba. Multicategory sup- port vector machines, theory, and application to the classifica- tion of microarray data and satellite radiance data. Journal of the American Statistical Association, 99(465), 2004.
  12. [Platt et al., 2000] J. Platt, N. Cristianini, and J. Shawe-Taylor. Large margin dags for multiclass classification. In S.A. Solla, T.K. Leen, and K.-R. Mueller, editors, Advances in Neural Infor- mation Processing Systems 12, pages 547-553, 2000.
  13. [Platt, 1999] John C. Platt. Fast training of support vector machines using sequential minimal optimization. MIT Press, 1999.
  14. [Tsochantaridis et al., 2005] I. Tsochantaridis, T. Joachims, T. Hof- mann, and Y. Altun. Large margin methods for structured and interdependent output variables. JMLR, 6:1453-1484, 2005.
  15. [Weston and Watkins, 1999] J. Weston and C. Watkins. Support vector machines for multiclass pattern recognition. In Proceed- ings of the Seventh European Symposium On Artificial Neural Networks, pages 219-224, 1999.
  16. [Yuan and Lin, 2006] Ming Yuan and Yi Lin. Model selection and estimation in regression with grouped variables. Journal Of The Royal Statistical Society Series B, 68(1):49-67, 2006.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards