{"type":"ScholarlyArticle","title":"Mixture of GANs for Clustering","authors":[{"name":"Yang Yu","slug":"yang-yu","given":"Yang","family":"Yu","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing 210023 , China"},{"name":"Wen-Ji Zhou","slug":"wen-ji-zhou","given":"Wen-Ji","family":"Zhou","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing 210023 , China"}],"abstract":"For data clustering, Gaussian mixture model (GMM) is a typical method that trains several Gaussian models to capture the data. Each Gaussian model then provides the distribution information of a cluster. For clustering of high dimensional and complex data,  more flexible models rather than Gaussian models are desired. Recently, the generative adversarial networks (GANs) have shown effectiveness in capturing complex data distribution. Therefore, GAN mixture model (GANMM) would be a promising alternative of GMM. However, we notice that the non-flexibility of the Gaussian model is essential in the expectation-maximization procedure for training GMM. GAN can have much higher flexibility, which disables the commonly employed expectation-maximization procedure, as that the maximization cannot change the result of the expectation. In this paper, we propose to use the epsilon-expectation-maximization procedure for training GANMM. The experiments show that the proposed GANMM can have good performance on complex data as well as simple data.","doi":"10.24963/ijcai.2018/423","url":"https://ijcai.org/proceedings/2018/423","pdf":"https://ijcai.org/proceedings/2018/0423.pdf","markdown":"https://ijcai.org/proceedings/2018/423.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/423","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":423,"pages":{"first":3047,"last":3053},"keywords":["Machine Learning: Clustering","Machine Learning: Deep Learning","Machine Learning: Unsupervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Leon Bottou. Wasserstein generative adversarial net- works. In Proceedings of the 34th International Conference on Machine Learning, pages 214-223, Sydney, Australia, 2017.","title":"Wasserstein generative adversarial networks","authors":"Arjovsky","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bouveyron and Brunet, 2012] Charles Bouveyron and Camille Brunet. Theoretical and practical considerations on the convergence properties of the Fisher-EM algorithm. Multivariate Analysis, 109:29-41, 2012.","title":"Theoretical and practical considerations on the convergence properties of the Fisher-EM algorithm","authors":"Brunet Bouveyron, Charles Bouveyron, Camille Brunet","year":"2012","venue":"Multivariate Analysis","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and Gopalakrishnan, 1998] Scott Shaobing Chen and Ponani S. Gopalakrishnan. Clustering via the bayesian in- formation criterion with applications in speech recognition. In Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 645- 648, Seattle, Washington, USA, 1998.","title":"Clustering via the bayesian information criterion with applications in speech recognition","authors":"Chen, Gopalakrishnan ; Scott, Shaobing Chen, Ponani S Gopalakrishnan","year":"1998","venue":"Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2016] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: In- terpretable representation learning by information maxi- mizing generative adversarial nets. In Advances in Neu- ral Information Processing Systems 29, pages 2172-2180, Barcelona, Spain, 2016.","title":"Infogan: Interpretable representation learning by information maximizing generative adversarial nets","authors":"Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dua and Karra, 2017] Dheeru Dua and Taniskidou Efi Karra. UCI machine learning repository, 2017.","title":"Dheeru Dua and Taniskidou Efi Karra. UCI machine learning repository","authors":"Karra Dua","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Duda et al., 2012] Richard O. Duda, Peter E. Hart, and David G. Stork. Pattern classification. John Wiley & Sons, 2012.","title":"Pattern classification","authors":"Richard O Duda, Peter E Hart, David G Stork","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fahad et al., 2014] Adil Fahad, Najlaa Alshatri, Zahir Tari, Abdullah Alamri, Ibrahim Khalil, Albert Y. Zomaya, Sebti Foufou, and Abdelaziz Bouras. A survey of clustering algorithms for big data: Taxonomy and empirical analy- sis. IEEE Transactions on Emerging Topics of Computing, 2(3):267-279, 2014.","title":"A survey of clustering algorithms for big data: Taxonomy and empirical analysis","authors":"Adil Fahad, Najlaa Alshatri, Zahir Tari, Abdullah Alamri, Ibrahim Khalil, Albert Y Zomaya, Sebti Foufou, Abdelaziz Bouras","year":"2014","venue":"IEEE Transactions on Emerging Topics of Computing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Frigui and Krishnapuram, 1999] Hichem Frigui and Raghu Krishnapuram. A robust competitive clustering algorithm with applications in computer vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5):450-465, 1999.","title":"Hichem Frigui and Raghu Krishnapuram. A robust competitive clustering algorithm with applications in computer vision","authors":"Krishnapuram Frigui","year":"1999","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sher- jil Ozair, Aaron C. Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances in Neural Informa- tion Processing Systems 27, pages 2672-2680, Montreal, Canada, 2014.","title":"Generative adversarial nets","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C Courville, Yoshua Bengio","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jordan and Xu, 1995] Michael Irwin Jordan and Lei Xu. Convergence results for the EM approach to mixtures of experts architectures. Neural Networks, 8(9):1409-1431, 1995.","title":"Convergence results for the EM approach to mixtures of experts architectures","authors":"Jordan, Xu ; Michael, Irwin Jordan, Lei Xu","year":"1995","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[LeCun et al., 1998] Yann LeCun, Leon Bottou, Yoshua Ben- gio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Lecun","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Kim, 2003] Qing Li and Byeong Man Kim. Cluster- ing approach for hybrid recommender system. In Proceed- ings of the 2003 IEEE / WIC International Conference on Web Intelligence, pages 33-38, Halifax, Canada, 2003.","title":"Clustering approach for hybrid recommender system","authors":"Kim ; ] Qing Li, Byeong Li, Kim Man","year":"2003","venue":"Proceedings of the 2003 IEEE / WIC International Conference on Web Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2015] Baohua Li, Ying Zhang, Zhouchen Lin, and Huchuan Lu. Subspace clustering by mixture of gaussian regression. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, pages 2094- 2102, Boston, MA, USA, 2015.","title":"Subspace clustering by mixture of gaussian regression","authors":"Baohua Li, Ying Zhang, Zhouchen Lin, Huchuan Lu","year":"2015","venue":"Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[McLachlan and Basford, 2004] Geoffrey John McLachlan and Kaye Enid Basford. Mixture models: Inference and Applications to Clustering. Marcel Dekker, 2004.","title":"Mixture models: Inference and Applications to Clustering","authors":"Basford Mclachlan, Geoffrey, John Mclachlan, Kaye Enid Basford","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[McLachlan and Peel, 2004] Geoffrey John McLachlan and David Peel. Finite mixture models. John Wiley & Sons, 2004.","title":"Finite mixture models","authors":"Geoffrey, John Mclachlan, David Peel","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nowozin et al., 2016] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: Training generative neural samplers using variational divergence minimization. In Advances in Neural Information Processing Systems 29, pages 271-279, Barcelona, Spain, 2016.","title":"f-GAN: Training generative neural samplers using variational divergence minimization","authors":"Sebastian Nowozin, Botond Cseke, Ryota Tomioka","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Radford et al., 2015] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv, 1511.06434, 2015.","title":"Unsupervised representation learning with deep convolutional generative adversarial networks","authors":"Radford","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Raghunathan et al., 2017] Aditi Raghunathan, Prateek Jain, and Ravishankar Krishnaswamy. Learning mixture of Gaus- sians with streaming data. In Advances in Neural Informa- tion Processing Systems 30, pages 6608-6617, Long Beach, CA, 2017.","title":"Learning mixture of Gaussians with streaming data","authors":"Aditi Raghunathan, Prateek Jain, Ravishankar Krishnaswamy","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xie et al., 2016] Junyuan Xie, Ross B. Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering anal- ysis. In Proceedings of the 33nd International Conference on Machine Learning, pages 478-487, New York City, NY, 2016.","title":"Unsupervised deep embedding for clustering analysis","authors":"Junyuan Xie, Ross B Girshick, Ali Farhadi","year":"2016","venue":"Proceedings of the 33nd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xu and Jordan, 1996] Lei Xu and Michael Irwin Jordan. On convergence properties of the em algorithm for Gaussian mixtures. Neural computation, 8(1):129-151, 1996.","title":"On convergence properties of the em algorithm for Gaussian mixtures","authors":"Lei Xu, Michael Irwin, Jordan","year":"1996","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu et al., 2016] Ji Xu, Daniel J. Hsu, and Arian Maleki. Global analysis of expectation maximization for mixtures of two Gaussians. In Advances in Neural Information Pro- cessing Systems 29, pages 2676-2684, Barcelona, Spain, 2016.","title":"Global analysis of expectation maximization for mixtures of two Gaussians","authors":"Ji Xu, Daniel J Hsu, Arian Maleki","year":"2016","venue":"Advances in Neural Information Processing Systems 29","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}