{"type":"ScholarlyArticle","title":"Discriminative Bayesian Nonparametric Clustering","authors":[{"name":"Vu Nguyen","slug":"vu-nguyen","given":"Vu","family":"Nguyen","affiliation":"†Deakin University , Australia ‡Adobe Research , USA"},{"name":"Dinh Phung","slug":"dinh-phung","given":"Dinh","family":"Phung","affiliation":"†Deakin University , Australia ‡Adobe Research , USA"},{"name":"Trung Le","slug":"trung-le","given":"Trung","family":"Le","affiliation":"†Deakin University , Australia ‡Adobe Research , USA"},{"name":"Hung Bui","slug":"hung-bui","given":"Hung","family":"Bui","affiliation":"†Deakin University , Australia ‡Adobe Research , USA"}],"abstract":"We propose a general framework for discriminative Bayesian nonparametric clustering to promote the inter-discrimination among the learned clusters in a fully Bayesian nonparametric (BNP) manner. Our method combines existing BNP clustering and discriminative models by enforcing latent cluster indices to be consistent with the predicted labels resulted from  probabilistic discriminative model. This formulation results in a well-defined generative process wherein we can use either logistic regression or SVM for discrimination. Using the proposed framework, we develop two novel discriminative BNP variants: the discriminative Dirichlet process mixtures, and the discriminative-state infinite HMMs for sequential data. We develop efficient data-augmentation Gibbs samplers for posterior inference. Extensive experiments in image clustering and dynamic location clustering demonstrate that by encouraging discrimination between induced clusters, our model enhances the quality of clustering in comparison with the traditional generative BNP models.","doi":"10.24963/ijcai.2017/355","url":"https://ijcai.org/proceedings/2017/355","pdf":"https://ijcai.org/proceedings/2017/0355.pdf","markdown":"https://ijcai.org/proceedings/2017/355.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/355","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":355,"pages":{"first":2550,"last":2556},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Learning Graphical Models"],"videos":[],"references":[{"position":1,"raw":"[Ambainis and Filmus, 2014] Andris Ambainis and Yuval Filmus. On the coppersmith-winograd method. 2014.","title":"On the coppersmith-winograd method","authors":"Andris Ambainis, Yuval Filmus","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Antoniak, 1974] C.E. Antoniak. Mixtures of Dirichlet pro- cesses with applications to Bayesian nonparametric prob- lems. The Annals of Statistics, 2(6):1152-1174, 1974.","title":"Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems","authors":"C E Antoniak","year":"1974","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Beal et al., 2002] M.J. Beal, Z. Ghahramani, and C.E. Ras- mussen. The infinite hidden Markov model. In Advances in Neural Information Processing Systems (NIPS), vol- ume 1, pages 577-584. MIT, 2002.","title":"The infinite hidden Markov model","authors":"M J Beal, Z Ghahramani, C E Rasmussen","year":"2002","venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2014] Changyou Chen, Jun Zhu, and Xinhua Zhang. Robust bayesian max-margin clustering. In Ad- vances in Neural Information Processing Systems, pages 532-540, 2014.","title":"Robust bayesian max-margin clustering","authors":"Changyou Chen, Jun Zhu, Xinhua Zhang","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[De la Torre and Kanade, 2006] Fernando De la Torre and Takeo Kanade. Discriminative cluster analysis. In Pro- ceedings of the 23rd international conference on Machine learning, pages 241-248. ACM, 2006.","title":"Fernando De la Torre and Takeo Kanade. Discriminative cluster analysis","authors":"Torre De La, Kanade","year":"2006","venue":"Proceedings of the 23rd international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Escobar and West, 1995] M.D. Escobar and M. West. Bayesian density estimation and inference using mix- tures. Journal of the american statistical association, 90(430):577-588, 1995.","title":"Bayesian density estimation and inference using mixtures","authors":"M D Escobar, M West","year":"1995","venue":"Journal of the american statistical association","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ferguson, 1973] Thomas S Ferguson. A Bayesian analysis of some nonparametric problems. The Annals of Statistics, 1(2):209-230, 1973.","title":"A Bayesian analysis of some nonparametric problems","authors":"Thomas S Ferguson","year":"1973","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Frey and Dueck, 2007] B.J. Frey and D. Dueck. Cluster- ing by passing messages between data points. Science, 315:972-976, February 2007.","title":"Clustering by passing messages between data points","authors":"B J Frey, D Dueck","year":"2007","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ghahramani, 2012] Zoubin Ghahramani. Nonparametric bayesian modelling, 2012. Lecture at Machine Learn- ing Summer School, http://videolectures.net/mlss2012- ghahramani-nonparametric-bayesian.","title":"Lecture at Machine Learning Summer School","authors":"Zoubin Ghahramani","year":"2012","venue":null,"doi":null,"url":"http://videolectures.net/mlss2012-ghahramani-nonparametric-bayesian","internal_url":null},{"position":10,"raw":"[Henao et al., 2014] Ricardo Henao, Xin Yuan, and Lawrence Carin. Bayesian nonlinear support vector machines and discriminative factor modeling. In Ad- vances in Neural Information Processing Systems, pages 1754-1762, 2014.","title":"Bayesian nonlinear support vector machines and discriminative factor modeling","authors":"Ricardo Henao, Xin Yuan, Lawrence Carin","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Joulin et al., 2010] Armand Joulin, Francis Bach, and Jean Ponce. Discriminative clustering for image co- segmentation. In Computer Vision and Pattern Recog- nition (CVPR), 2010 IEEE Conference on, pages 1943- 1950. IEEE, 2010.","title":"Discriminative clustering for image cosegmentation","authors":"Armand Joulin, Francis Bach, Jean Ponce","year":"2010","venue":"Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kaski et al., 2005] Samuel Kaski, Janne Sinkkonen, and Arto Klami. Discriminative clustering. Neurocomputing, 69(1):18-41, 2005.","title":"Discriminative clustering","authors":"Kaski","year":"2005","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Krause et al., 2010] Andreas Krause, Pietro Perona, and Ryan G Gomes. Discriminative clustering by regularized information maximization. In Advances in neural infor- mation processing systems, pages 775-783, 2010.","title":"Discriminative clustering by regularized information maximization","authors":"Andreas Krause, Pietro Perona, Ryan G Gomes","year":"2010","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kulis and Jordan, 2012] B. Kulis and M. I. Jordan. Revis- iting k-means: New algorithms via bayesian nonparamet- rics. In Proceedings of the 29th International Conference on Machine Learning (ICML), Edinburgh, UK, 2012.","title":"Revisiting k-means: New algorithms via bayesian nonparametrics","authors":"B Kulis, M I Jordan","year":"2012","venue":"Proceedings of the 29th International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Laurila et al., 2012] Juha K Laurila, Daniel Gatica-Perez, Imad Aad, Olivier Bornet, Trinh-Minh-Tri Do, Olivier Dousse, Julien Eberle, Markus Miettinen, et al. The mo- bile data challenge: Big data for mobile computing re- search. In Pervasive Computing, number EPFL-CONF- 192489, 2012.","title":"The mobile data challenge: Big data for mobile computing research","authors":"Daniel Juha K Laurila, Imad Gatica-Perez, Olivier Aad, Trinh-Minh-Tri Bornet, Olivier Do, Julien Dousse, Markus Eberle, Miettinen","year":"2012","venue":"Pervasive Computing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2014] Chengtao Li, Jun Zhu, and Jianfei Chen. Bayesian max-margin multi-task learning with data aug- mentation. pages 415-423, 2014.","title":"Bayesian max-margin multi-task learning with data augmentation","authors":"Li","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nguyen et al., 2014] T.V. Nguyen, D. Phung, S. Venkatesh, X.L. Nguyen, and H. Bui. Bayesian nonparametric mul- tilevel clustering with group-level contexts. In Proc. of International Conference on Machine Learning (ICML), pages 288-296, Beijing, China, 2014.","title":"Bayesian nonparametric multilevel clustering with group-level contexts","authors":"T V Nguyen, D Phung, S Venkatesh, X L Nguyen, H Bui","year":"2014","venue":"Proc. of International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nguyen et al., 2015] Vu Nguyen, Dinh Phung, Trung Le, and Svetha Venkatesh. Large sample asymptotic for non- parametric mixture model with count data. In Workshop on Advances in Approximate Bayesian Inference at Neural Information Processing Systems (NIPS), 2015.","title":"Large sample asymptotic for nonparametric mixture model with count data","authors":"Dinh Vu Nguyen, Trung Phung, Svetha Le, Venkatesh","year":"2015","venue":"Workshop on Advances in Approximate Bayesian Inference at Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nguyen et al., 2016a] Tu Dinh Nguyen, Vu Nguyen, Trung Le, and Dinh Phung. Distributed data augmented sup- port vector machine on spark. In 23st International Con- ference on Pattern Recognition (ICPR), pages 498-503. IEEE, 2016.","title":"Distributed data augmented support vector machine on spark","authors":"Nguyen","year":"2016","venue":"23st International Conference on Pattern Recognition (ICPR)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nguyen et al., 2016b] Vu Nguyen, Tu Dinh Nguyen, Trung Le, Svetha Venkatesh, and Dinh Phung. One-pass logis- tic regression for label-drift and large-scale classification on distributed systems. In 2016 IEEE 16th International Conference on Data Mining (ICDM), pages 1113-1118. IEEE, 2016.","title":"One-pass logistic regression for label-drift and large-scale classification on distributed systems","authors":"Nguyen","year":"2016","venue":"IEEE 16th International Conference on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Polson et al., 2011] Nicholas G Polson, Steven L Scott, et al. Data augmentation for support vector machines. Bayesian Analysis, 6(1):1-23, 2011.","title":"Data augmentation for support vector machines","authors":"Steven L Nicholas G Polson, Scott","year":"2011","venue":"Bayesian Analysis","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Polson et al., 2013] Nicholas G Polson, James G Scott, and Jesse Windle. Bayesian inference for logistic models using pólya-gamma latent variables. Journal of the American Statistical Association, 108(504):1339-1349, 2013.","title":"Bayesian inference for logistic models using pólya-gamma latent variables","authors":"James G Nicholas G Polson, Jesse Scott, Windle","year":"2013","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sethuraman, 1994] J. Sethuraman. A constructive definition of Dirichlet priors. Statistica Sinica, 4(2):639-650, 1994.","title":"A constructive definition of Dirichlet priors","authors":"J Sethuraman","year":"1994","venue":"Statistica Sinica","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Teh et al., 2006] Y.W. Teh, M.I. Jordan, M.J. Beal, and D.M. Blei. Hierarchical Dirichlet processes. Journal of the American Statistical Association, 101(476):1566- 1581, 2006.","title":"Hierarchical Dirichlet processes","authors":"Y W Teh, M I Jordan, M J Beal, D M Blei","year":"2006","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ye et al., 2008] Jieping Ye, Zheng Zhao, and Mingrui Wu. Discriminative k-means for clustering. In Advances in neural information processing systems, pages 1649-1656, 2008.","title":"Discriminative k-means for clustering","authors":"Ye","year":"2008","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2014] Aonan Zhang, Jun Zhu, and Bo Zhang. Max-margin infinite hidden markov models. pages 315- 323, 2014.","title":"Max-margin infinite hidden markov models","authors":"Zhang","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhu et al., 2014] Jun Zhu, Ning Chen, Hugh Perkins, and Bo Zhang. Gibbs max-margin topic models with data aug- mentation. The Journal of Machine Learning Research, 15(1):1073-1110, 2014.","title":"Gibbs max-margin topic models with data augmentation","authors":"Jun Zhu, Ning Chen, Hugh Perkins, Bo Zhang","year":"2014","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}