Proceedings

Non-Negative Multiple Matrix Factorization

Download BibTeX Cite

Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. Pages 1713-1719.

© 2013 International Joint Conferences on Artificial Intelligence

Abstract

Non-negative Matrix Factorization (NMF) is a traditional unsupervised machine learning technique for decomposing a matrix into a set of bases and coefficients under the non-negative constraint. NMF with sparse constraints is also known for extracting reasonable components from noisy data. However, NMF tends to give undesired results in the case of highly sparse data, because the information included in the data is insufficient to decompose. Our key idea is that we can ease this problem if complementary data are available that we could integrate into the estimation of the bases and coefficients. In this paper, we propose a novel matrix factorization method called Non-negative Multiple Matrix Factorization (NMMF), which utilizes complementary data as auxiliary matrices that share the row or column indices of the target matrix. The data sparse- ness is improved by decomposing the target and auxiliary matrices simultaneously, since auxiliary matrices provide information about the bases and coefficients. We formulate NMMF as a generalization of NMF, and then present a parameter estimation procedure derived from the multiplicative up- date rule. We examined NMMF in both synthetic and real data experiments. The effect of the auxiliary matrices appeared in the improved NMMF performance. We also confirmed that the bases that NMMF obtained from the real data were intuitive and reasonable thanks to the non-negative constraint.

  1. [Abdallah and Plumbley, 2004] S.A. Abdallah and M.D. Plumbley. Polyphonic music transcription by non-negative sparse coding of power spectra. In Proc. ISMIR, 2004.
  2. [Aharon et al., 2006] M. Aharon, M. Elad, and A.M. Bruck- stein. On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them. Linear algebra and its ap- plications, 416(1):48-67, 2006.
  3. [Blei et al., 2003] D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993-1022, 2003.
  4. [Cai et al., 2011] D. Cai, X. He, and J. Han. Graph regular- ized non-negative matrix factorization for data representa- tion. IEEE Trans., 33:1548-1560, 2011.
  5. [Cao et al., 2007] B. Cao, D. Shen, J.T. Sun, X. Wang, Q. Yang, and Z. Chen. Detect and track latent factors with online nonnegative matrix factorization. In Proc. IJCAI, 2007. IJCAI
  6. [Cemgil, 2009] A.T. Cemgil. Bayesian inference for nonneg- ative matrix factorisation models. Computational Intelli- gence and Neuroscience, 2009.
  7. [Cichocki et al., 2007] A. Cichocki, R. Zdunek, and S. Amari. Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization. In Proc. ICA, 2007.
  8. [Cichocki et al., 2009] A. Cichocki, A. H. Phan R. Zdunek, and S. Amari. Nonnegative Matrix and Tensor Factoriza- tions: Applications to Exploratory Multi-way Data Analy- sis, John Wiley. Wiley, 2009.
  9. [Dikmen and Févotte, 2012] O. Dikmen and C. Févotte. Maximum marginal likelihood estimation for nonnegative dictionary learning in the Gamma-Poisson model. IEEE Trans., 2012.
  10. [Ding et al., 2006] C. Ding, T. Li, and W. Peng. NMF and PLSI: Equivalence and a hybrid algorithm. In Proc. SIGIR, 2006.
  11. [Duh et al., 2012] K. Duh, T. Hirao, A. Kimura, K. Ishiguro, T. Iwata, and C. A. Yeung. Creating stories: Social cura- tion of Twitter messages. In Proc. ICWSM, 2012.
  12. [Hoffman et al., 2011] M. D. Hoffman, D. M. Blei, and P. R. Cook. Bayesian nonparametric matrix factorization for recorded music. In Proc. ICML, 2011.
  13. [Hofmann, 1999] T. Hofmann. Probabilistic latent semantic indexing. In Proc. SIGIR, 1999.
  14. [Hoyer, 2004] P.O. Hoyer. Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research, 5:1457-1469, 2004.
  15. [Ishiguro et al., 2012] K. Ishiguro, A. Kimura, and K. Takeuchi. Towards automatic image understand- ing and mining via social curation. In Proc. ICDM, 2012.
  16. [Koren et al., 2009] Y. Koren, R. Bell, and C. Volinsky. Ma- trix factorization techniques for recommender systems. IEEE, 2009.
  17. [Lee and Seung, 1999] D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401:788-791, October 1999.
  18. [Lee et al., 2000] D. D. Lee, Daniel D., and S. H. Sebastian. Algorithms for non-negative matrix factorization. In Proc. NIPS, 2000.
  19. [Lin et al., 2011] J. Lin, R. Snow, and W. Morgan. Smooth- ing techniques for adaptive online language models: topic tracking in tweet streams. In Proc. SIGKDD, 2011.
  20. [Liu et al., 2010] C. Liu, H. Yang, J. Fan, L. He, and Y. Wang. Distributed nonnegative matrix factorization for web-scale dyadic data analysis on MapReduce. In Proc. WWW, 2010.
  21. [Liu et al., 2012] H. Liu, Z. Yang, Z. Wu, and X. Li. A- optimal non-negative projection for image representation. In Proc. CVPR, 2012.
  22. [Ma et al., 2008] H. Ma, H. Yang, M. R. Lyu, and I. King. Sorec: Social recommendation using probabilistic matrix factorization. In Proc. CIKM, 2008.
  23. [Mei et al., 2008] Q. Mei, D. Cai, D. Zhang, and C. Zhai. Topic modeling with network regularization. In Proc. WWW, 2008.
  24. [Nakano et al., 2011] M. Nakano, J. L. Roux, H. Kameoka, T. Nakamura, N. Ono, and S. Sagayama. Bayesian non- parametric spectrogram modeling based on infinite fac- torial infinite hidden Markov model. In Proc. WASPAA, 2011.
  25. [Noel et al., 2012] J. Noel, S. Sanner, K. Tran, P. Christen, L. Xie, E. V. Bonilla, E. Abbasnejad, and N. D. Penna. New objective functions for social collaborative filtering. In Proc. WWW, 2012.
  26. [Purushotham et al., 2012] S. Purushotham, Y. Liu, and C. C. J. Kuo. Collaborative topic regression with social matrix factorization for recommendation systems. In Proc. ICML, 2012.
  27. [Salakhutdinov and Mnih, 2008] R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. In Proc. NIPS, 2008.
  28. [Schmidt et al., 2009] M. N. Schmidt, O. Winther, and L. K. Hansen. Bayesian non-negative matrix factorization. In Proc. ICASSP, 2009.
  29. [Smaragdis and Brown, 2003] P. Smaragdis and J.C. Brown. Non-negative matrix factorization for polyphonic music transcription. In Proc. WASPA, 2003.
  30. [Steyvers et al., 2004] M. Steyvers, P. Smyth, and T. Grif- fiths. Probabilistic author-topic models for information discovery. In Proc. SIGKDD, 2004.
  31. [Wang and Blei, 2011] C. Wang and D. M. Blei. Collabora- tive topic modeling for recommending scientific articles. In Proc. SIGKDD, 2011.
  32. [Wang et al., 2011] F. Wang, C. Tan, A.C. König, and P. Li. Efficient document clustering via online nonnegative ma- trix factorizations. In Proc. SIAM, 2011.
  33. [Xu et al., 2003] W. Xu, X. Liu, and Y. Gong. Document clustering based on non-negative matrix factorization. In Proc. SIGIR, 2003.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards