Proceedings

Discriminative Semi-Supervised Feature Selection via Manifold Regularization

Download BibTeX Cite

Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence. Pages 1303-1307.

© 2009 International Joint Conferences on Artificial Intelligence

Abstract

We consider the problem of semi-supervised feature selection, where we are given a small amount of labeled examples and a large amount of unlabeled examples. Since a small number of labeled samples are usually insufficient for identifying the relevant features, the critical problem arising from semi-supervised feature selection is how to take advantage of the information underneath the unlabeled data. To address this problem, we propose a novel discriminative semi-supervised feature selection method based on the idea of manifold regularization. The proposed method selects features throughmaximizing the classification margin between different classes and simultaneously exploiting the geometry of the probability distribution that generates both labeled and unlabeled data. We formulate the proposed feature selection method into a convex-concave optimization problem, where the saddle point corresponds to the optimal solution. To find the optimal solution, the level method, a fairly recent optimization method, is employed. We also present a theoretic proof of the convergence rate for the application of the level method to our problem. Empirical evaluation on several benchmark data sets demonstrates the effectiveness of the proposed semi-supervised feature selection method.Zenglin Xu, Rong Jin, Michael R. Lyu, Irwin King

  1. [Belkin et al., 2006] Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Ma- chine Learning Research, 7:2399-2434, 2006.
  2. [Bishop, 1995] C.M. Bishop. Neural Networks for Pattern Recog- nition. Oxford University Press, London, 1995.
  3. [Cortes et al., 2008] Corinna Cortes, Mehryar Mohri, and Afshin Rostamizadeh. Learning sequence kernels. In IEEE Workshop on Machine Learning for Signal Processing, pages 2-8, 2008.
  4. [Dy and Brodley, 2004] Jennifer G. Dy and Carla E. Brodley. Fea- ture selection for unsupervised learning. Journal of Machine Learning Research, 5:845-889, 2004.
  5. [Fung and Mangasarian, 2000] Glenn Fung and Olvi L. Mangasar- ian. Data selection for support vector machine classifiers. In KDD '00: Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 64- 70, New York, NY, USA, 2000. ACM.
  6. [Guyon and Elisseeff, 2003] I. Guyon and A. Elisseeff. An intro- duction to variable and feature selection. Journal of Machine Learning Research, 3:1157-1182, 2003.
  7. [Guyon et al., 2002] Isabelle Guyon, Jason Weston, Stephen Barn- hill, and Vladimir Vapnik. Gene selection for cancer classifica- tion using support vector machines. Machine Learning, 46(1- 3):389-422, 2002.
  8. [Lanckriet et al., 2004] Gert R. G. Lanckriet, Nello Cristianini, Pe- ter Bartlett, Laurent El Ghaoui, and Michael I. Jordan. Learning the kernel matrix with semidefinite programming. Journal of Ma- chine Learning Research, 5:27-72, 2004.
  9. [Rakotomamonjy et al., 2008] Alain Rakotomamonjy, Francis R. Bach, Stéphane Canu, and Yves Grandvalet. SimpleMKL. Jour- nal of Machine Learning Research, 9:1179-1225, 2008.
  10. [Ren et al., 2008] Jiangtao Ren, Zhengyuan Qiu, Wei Fan, Hong Cheng, and Philip S. Yu. Forward semi-supervised feature se- lection. In Proceedings of Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD '08), pages 970-976, 2008.
  11. [Sonnenburg et al., 2006] Sören Sonnenburg, Gunnar Rätsch, Christin Schäfer, and Bernhard Schölkopf. Large scale multi- ple kernel learning. Journal of Machine Learning Research, 7:1531-1565, 2006.
  12. [Weston et al., 2003] Jason Weston, André Elisseeff, Bernhard Schölkopf, and Mike Tipping. Use of the zero norm with lin- ear models and kernel methods. Journal of Machine Learning Research, 3:1439-1461, 2003.
  13. [Xu et al., 2009] Zenglin Xu, Rong Jin, Irwin King, and Michael Lyu. An extended level method for efficient multiple kernel learn- ing. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21 (NIPS), pages 1825-1832. 2009.
  14. [Zhao and Liu, 2007] Zheng Zhao and Huan Liu. Semi-supervised feature selection via spectral analysis. In SDM, pages 641-646, 2007.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards