Proceedings

Learning Finite Beta-Liouville Mixture Models via Variational Bayes for Proportional Data Clustering

Download BibTeX Cite

Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. Pages 1323-1328.

© 2013 International Joint Conferences on Artificial Intelligence

Abstract

During the past decade, finite mixture modeling has become a well-established technique in data analysis and clustering. This paper focus on developing a variational inference framework to learn finite Beta-Liouville mixture models that have been proposed recently as an efficient way for proportional data clustering. In contrast to the conventional expectation maximization (EM) algorithm, commonly used for learning finite mixture models, the proposed algorithm has the advantages that it is more efficient from a computational point of view and by preventing over- and under-fitting problems. Moreover, the complexity of the mixture model (i.e. the number of components) can be determined automatically and simultaneously with the parameters estimation in a closed form as part of the Bayesian inference procedure. The merits of the proposed approach are shown using both artificial data sets and two interesting and challenging real applications namely dynamic textures clustering and facial expression recognition.

  1. [Attias, 1999a] H. Attias. Inferring parameters and structure of latent variable models by variational Bayes. In Proc. of the Conference on Uncertainty in Artificial Intelligence (UAI), pages 21-30, 1999.
  2. [Attias, 1999b] H. Attias. A variational Bayes framework for graphical models. In Proc. of Advances in Neural Informa- tion Processing Systems (NIPS), pages 209-215, 1999.
  3. [Bishop, 2006] C. M. Bishop. Pattern Recognition and Ma- chine Learning. Springer, 2006.
  4. [Black and Yacoob, 1997] M.J. Black and Y. Yacoob. Rec- ognizing facial expressions in image sequences using lo- cal parameterized models of image motion. International Journal of Computer Vision, 25(1):23-48, 1997.
  5. [Bouguila and Ziou, 2006] N. Bouguila and D. Ziou. Unsu- pervised selection of a finite Dirichlet mixture model: An mml-based approach. IEEE Transactions on Knowledge and Data Engineering, 18(8):993-1009, 2006.
  6. [Bouguila and Ziou, 2007] N. Bouguila and D. Ziou. High- dimensional unsupervised selection and estimation of a fi- nite generalized Dirichlet mixture model based on mini- mum message length. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 29(10):1716-1731, 2007.
  7. [Bouguila, 2012] N. Bouguila. Hybrid genera- tive/discriminative approaches for proportional data modeling and classification. IEEE Transactions on Knowledge and Data Engineering, 24(12):2184-2202, 2012.
  8. [Constantinopoulos et al., 2006] C. Constantinopoulos, M.K. Titsias, and A. Likas. Bayesian feature and model selection for gaussian mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(6):1013 -1018, 2006.
  9. [Corduneanu and Bishop, 2001] A. Corduneanu and C. M. Bishop. Variational Bayesian model selection for mix- ture distributions. In Proc. of the 8th International Con- ference on Artificial Intelligence and Statistics (AISTAT), pages 27-34, 2001.
  10. [Doretto et al., 2003] G. Doretto, A. Chiuso, Y. Wu, and S. Soatto. Dynamic textures. International Journal of Computer Vision, 51(2):91-109, 2003.
  11. [Figueiredo and Jain, 2002] M.A.T. Figueiredo and A.K. Jain. Unsupervised learning of finite mixture models. IEEE Transactions on Pattern Analysis and Machine In- telligence, 24(3):381 -396, 2002.
  12. [Jordan et al., 1998] M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. An introduction to variational methods for graphical models. In Learning in Graphical Models, pages 105-162, 1998.
  13. [Kanade et al., 2000] T. Kanade, J.F. Cohn, and Y. Tian. Comprehensive database for facial expression analysis. In Proc. of the IEEE International Conference on Auto- matic Face and Gesture Recognition (AFGR), pages 46- 53, 2000.
  14. [Koelstra et al., 2010] S. Koelstra, M. Pantic, and I. Patras. A dynamic texture-based approach to recognition of facial actions and their temporal models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(11):1940- 1954, 2010.
  15. [Kwatra et al., 2003] V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick. Graphcut textures: Image and video syn- thesis using graph cuts. ACM Transactions on Graphics, SIGGRAPH 2003, 22(3):277-286, July 2003.
  16. [Lucey et al., 2010] P. Lucey, J.F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In Pro. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 94-101, 2010.
  17. [Mackay, 1995] D. J. C. Mackay. Probable networks and plausible predictions -a review of practical Bayesian methods for supervised neural networks. Network: Com- putation in Neural Systems, 6(3):469-505, 1995.
  18. [McLachlan and Peel, 2000] G. McLachlan and D. Peel. Fi- nite Mixture Models. New York: Wiley, 2000.
  19. [Pantic and Rothkrantz, 1999] M. Pantic and L.J.M. Rothkrantz. An expert system for multiple emotional classification of facial expressions. In Proc. of the 11th IEEE International Conference on Tools with Artificial Intelligence (ICTAI), pages 113 -120, 1999.
  20. [Péteri et al., 2010] R. Péteri, S. Fazekas, and M.J. Huiskes. DynTex : a comprehensive database of dynamic textures. Pattern Recognition Letters, 31(12):1627-1632, 2010.
  21. [Ravichandran et al., 2013] A. Ravichandran, R. Chaudhry, and R. Vidal. Categorizing dynamic textures using a bag of dynamical systems. IEEE Transactions on Pattern Analy- sis and Machine Intelligence, 35(2):342 -353, 2013.
  22. [Shan et al., 2009] C. Shan, S. Gong, and P.W. McOwan. Fa- cial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing, 27(6):803 -816, 2009.
  23. [Wang and Titterington, 2006] B. Wang and D. M. Tittering- ton. Convergence properties of a general algorithm for cal- culating variational Bayesian estimates for a normal mix- ture model. Bayesian Analysis, 1(3):625-650, 2006.
  24. [Watanabe and Watanabe, 2007] K. Watanabe and S. Watan- abe. Stochastic complexity for mixture of exponential families in generalized variational bayes. Theoretical Computer Science, 387(1):4-17, 2007.
  25. [Watanabe et al., 2009] K. Watanabe, S. Akaho, S. Omachi, and M. Okada. Variational Bayesian mixture model on a subspace of exponential family distributions. IEEE Trans- actions on Neural Networks, 20(11):1783-1796, 2009.
  26. [Zhao and Pietikainen, 2007] G. Zhao and M. Pietikainen. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 29(6):915 -928, 2007.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards