{"type":"ScholarlyArticle","title":"Variational Mixtures of Gaussian Processes for Classification","authors":[{"name":"Chen Luo","slug":"chen-luo","given":"Chen","family":"Luo","affiliation":"Department of Computer Science and Technology , East China Normal University , 3663 North Zhongshan Road , Shanghai 200062 , P. R. China"},{"name":"Shiliang Sun","slug":"shiliang-sun","given":"Shiliang","family":"Sun","affiliation":"Department of Computer Science and Technology , East China Normal University , 3663 North Zhongshan Road , Shanghai 200062 , P. R. China"}],"abstract":"Gaussian Processes (GPs) are powerful tools for machine learning which have been applied to both classification and regression. The mixture models of GPs were later proposed to further improve GPs for data modeling. However, these models are formulated for regression problems. In this work, we propose a new Mixture of Gaussian Processes for Classification (MGPC). Instead of the Gaussian likelihood for regression, MGPC employs the logistic function as likelihood to obtain the class probabilities, which is suitable for classification problems. The posterior distribution of latent variables is approximated through variational inference. The hyperparameters are optimized through the variational EM method and a greedy algorithm. Experiments are performed on multiple real-world datasets which show improvements over five widely used methods on predictive performance. The results also indicate that for classification MGPC is significantly better than the regression model with mixtures of GPs, different from the existing consensus that their single model counterparts are comparable.","doi":"10.24963/ijcai.2017/642","url":"https://ijcai.org/proceedings/2017/642","pdf":"https://ijcai.org/proceedings/2017/0642.pdf","markdown":"https://ijcai.org/proceedings/2017/642.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/642","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Uncertainty in AI","track":"Main track","number":642,"pages":{"first":4603,"last":4609},"keywords":["Uncertainty in AI: Approximate Probabilistic Inference","Uncertainty in AI: Graphical Models"],"videos":[],"references":[{"position":1,"raw":"[Bishop and Svenskn, 2003] Christopher M. Bishop and Markus Svenskn. Bayesian hierarchical mixtures of ex- perts. In Proceedings of the 19th Conference on Uncer- tainty in Artificial Intelligence, pages 57-64, 2003.","title":"Bayesian hierarchical mixtures of experts","authors":"Christopher M Bishop, Markus Svenskn","year":"2003","venue":"Proceedings of the 19th Conference on Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bishop, 2006] Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006.","title":"Pattern Recognition and Machine Learning","authors":"Christopher M Bishop","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Blei and Jordan, 2006] David M. Blei and Michael I. Jor- dan. Variational inference for Dirichlet process mixtures. Bayesian Analysis, 1:121-144, 2006.","title":"Variational inference for Dirichlet process mixtures","authors":"David M Blei, Michael I Jordan","year":"2006","venue":"Bayesian Analysis","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blei et al., 2016] David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe. Variational inference: A review for statisticians. ArXiv e-prints:1601.00670, 2016.","title":"Variational inference: A review for statisticians","authors":"David M Blei, Alp Kucukelbir, Jon D Mcauliffe","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ferguson, 1973] Thomas S. Ferguson. A Bayesian analysis of some nonparametric problems. The Annals of Statistics, 1:209-230, 1973.","title":"A Bayesian analysis of some nonparametric problems","authors":"Thomas S Ferguson","year":"1973","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gal et al., 2015] Yarin Gal, Yutian Chen, and Zoubin Ghahramani. Latent Gaussian processes for distribution estimation of multivariate categorical data. In Proceed- ings of the 32nd International Conference on International Conference on Machine Learning, pages 645-654, 2015.","title":"Latent Gaussian processes for distribution estimation of multivariate categorical data","authors":"Yarin Gal, Yutian Chen, Zoubin Ghahramani","year":"2015","venue":"Proceedings of the 32nd International Conference on International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hensman et al., 2015] James Hensman, Alexander G. de G. Matthews, and Zoubin Ghahramani. Scalable variational Gaussian process classification. In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, pages 351-360, 2015.","title":"Scalable variational Gaussian process classification","authors":"James Hensman, Alexander G De, G Matthews, Zoubin Ghahramani","year":"2015","venue":"Proceedings of the 18th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kapoor et al., 2010] Ashish Kapoor, Kristen Grauman, Raquel Urtasun, and Trevor Darrell. Gaussian processes for object categorization. International Journal of Com- puter Vision, 88:169-188, 2010.","title":"Gaussian processes for object categorization","authors":"Ashish Kapoor, Kristen Grauman, Raquel Urtasun, Trevor Darrell","year":"2010","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lichman, 2013] M. Lichman. UCI machine learning repos- itory, 2013.","title":"UCI machine learning repository","authors":"M Lichman","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Meeds and Osindero, 2006] Edward Meeds and Simon Osindero. An alternative infinite mixture of Gaussian pro- cess experts. Advances in Neural Information Processing Systems, 18:883-890, 2006.","title":"Edward Meeds and Simon Osindero. An alternative infinite mixture of Gaussian process experts","authors":"Osindero Meeds","year":"2006","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Rasmussen and Ghahramani, 2002] Carl Edward Ras- mussen and Zoubin Ghahramani. Infinite mixtures of Gaussian process experts. Advances in Neural Information Processing Systems, 14:881-888, 2002.","title":"Carl Edward Rasmussen and Zoubin Ghahramani. Infinite mixtures of Gaussian process experts","authors":"Ghahramani Rasmussen","year":"2002","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Rasmussen and Williams, 2006] Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006.","title":"Gaussian Processes for Machine Learning","authors":"Carl Edward Rasmussen, Christopher K I Williams","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Sethuraman, 1994] Jayaram Sethuraman. A constructive definition of Dirichlet priors. Statistica Sinica, 4:639-650, 1994.","title":"A constructive definition of Dirichlet priors","authors":"Jayaram Sethuraman","year":"1994","venue":"Statistica Sinica","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Smola and Bartlett, 2001] Alex J. Smola and Peter L. Bartlett. Sparse greedy Gaussian process regression. Advances in Neural Information Processing Systems, 13:619-625, 2001.","title":"Sparse greedy Gaussian process regression","authors":"Alex J Smola, Peter L Bartlett","year":"2001","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sun and Xu, 2011] Shiliang Sun and Xin Xu. Variational inference for infinite mixtures of Gaussian processes with applications to traffic flow prediction. IEEE Transactions on Intelligent Transportation Systems, 12:466-475, 2011.","title":"Variational inference for infinite mixtures of Gaussian processes with applications to traffic flow prediction","authors":"Xu Sun, Shiliang Sun, Xin Xu","year":"2011","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tresp, 2001] Volker Tresp. Mixtures of Gaussian pro- cess. Advances in Neural Information Processing Systems, 13:654-660, 2001.","title":"Mixtures of Gaussian process","authors":"; Tresp, Volker Tresp","year":"2001","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Yuan and Neubauer, 2009] Chao Yuan and Claus Neubauer. Variational mixture of Gaussian process experts. Advances in Neural Information Processing Systems, 21:1897-1904, 2009.","title":"Variational mixture of Gaussian process experts","authors":"Neubauer Yuan, Chao Yuan, Claus Neubauer","year":"2009","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}