{"type":"ScholarlyArticle","title":"Learning General Gaussian Mixture Model with Integral Cosine Similarity","authors":[{"name":"Guanglin Li","slug":"guanglin-li","given":"Guanglin","family":"Li","affiliation":"Guangdong Key Laboratory of Intelligent Information Processing and Shenzhen Key Laboratory of Media Security , Shenzhen University , Shenzhen 518060 , China."},{"name":"Bin Li","slug":"bin-li","given":"Bin","family":"Li","affiliation":"Guangdong Key Laboratory of Intelligent Information Processing and Shenzhen Key Laboratory of Media Security , Shenzhen University , Shenzhen 518060 , China.; Shenzhen Institute of Artificial Intelligence and Robotics for Society , Shenzhen 518129 , China"},{"name":"Changsheng Chen","slug":"changsheng-chen","given":"Changsheng","family":"Chen","affiliation":"Guangdong Key Laboratory of Intelligent Information Processing and Shenzhen Key Laboratory of Media Security , Shenzhen University , Shenzhen 518060 , China.; Shenzhen Institute of Artificial Intelligence and Robotics for Society , Shenzhen 518129 , China"},{"name":"Shunquan Tan","slug":"shunquan-tan","given":"Shunquan","family":"Tan","affiliation":"Guangdong Key Laboratory of Intelligent Information Processing and Shenzhen Key Laboratory of Media Security , Shenzhen University , Shenzhen 518060 , China.; Shenzhen Institute of Artificial Intelligence and Robotics for Society , Shenzhen 518129 , China"},{"name":"Guoping Qiu","slug":"guoping-qiu","given":"Guoping","family":"Qiu","affiliation":"Guangdong Key Laboratory of Intelligent Information Processing and Shenzhen Key Laboratory of Media Security , Shenzhen University , Shenzhen 518060 , China.; Shenzhen Institute of Artificial Intelligence and Robotics for Society , Shenzhen 518129 , China"}],"abstract":"Gaussian mixture model (GMM) is a powerful statistical tool in data modeling, especially for unsupervised learning tasks. Traditional learning methods for GMM such as expectation maximization (EM) require the covariance of the Gaussian components to be non-singular, a condition that is often not satisfied in real-world applications. This paper presents a new learning method called G$^2$M$^2$ (General Gaussian Mixture Model) by fitting an unnormalized Gaussian mixture function (UGMF) to a data distribution. At the core of G$^2$M$^2$ is the introduction of an integral cosine similarity (ICS) function for comparing the UGMF and the unknown data density distribution without having to explicitly estimate it. By maximizing the ICS through Monte Carlo sampling, the UGMF can be made to overlap with the unknown data density distribution such that the two only differ by a constant scalar, and the UGMF can be normalized to obtain the data density distribution. A Siamese convolutional neural network is also designed for optimizing the ICS function. Experimental results show that our method is more competitive in modeling data having correlations that may lead to singular covariance matrices in GMM, and it outperforms state-of-the-art methods in unsupervised anomaly detection.","doi":"10.24963/ijcai.2022/444","url":"https://ijcai.org/proceedings/2022/444","pdf":"https://ijcai.org/proceedings/2022/0444.pdf","markdown":"https://ijcai.org/proceedings/2022/444.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/444","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":444,"pages":{"first":3201,"last":3207},"keywords":["Data Mining: Anomaly/Outlier Detection","Machine Learning: Unsupervised Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986481","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984624","length_sec":349,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Angiulli and Pizzuti, 2002] Fabrizio Angiulli and Clara Piz- zuti. Fast outlier detection in high dimensional spaces. In European conference on principles of data mining and knowledge discovery, pages 15-27. Springer, 2002.","title":"Fabrizio Angiulli and Clara Pizzuti. Fast outlier detection in high dimensional spaces","authors":"Pizzuti Angiulli","year":"2002","venue":"European conference on principles of data mining and knowledge discovery","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ban et al., 2018] Zhihua Ban, Jianguo Liu, and Li Cao. Superpixel segmentation using gaussian mixture model. IEEE Transactions on Image Processing, 27(8):4105- 4117, 2018.","title":"Superpixel segmentation using gaussian mixture model","authors":"Zhihua Ban, Jianguo Liu, Li Cao","year":"2018","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cho et al., 2019] Byung Joon Cho, Jun-Min Lee, and Hyung-Min Park. A beamforming algorithm based on maximum likelihood of a complex gaussian distribution with time-varying variances for robust speech recognition. IEEE Signal Processing Letters, 26(9):1398-1402, 2019.","title":"A beamforming algorithm based on maximum likelihood of a complex gaussian distribution with time-varying variances for robust speech recognition","authors":"Joon Byung, Jun-Min Cho, Hyung-Min Lee, Park","year":"2019","venue":"IEEE Signal Processing Letters","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Davari et al., 2018] Amirabbas Davari, Erchan Aptoula, Berrin Yanikoglu, Andreas Maier, and Christian Riess. Gmm-based synthetic samples for classification of hyper- spectral images with limited training data. IEEE Geo- science and Remote Sensing Letters, 15(6):942-946, 2018.","title":"Gmm-based synthetic samples for classification of hyperspectral images with limited training data","authors":"Davari","year":"2018","venue":"IEEE Geoscience and Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dempster et al., 1977] A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical So- ciety. Series B (Methodological), 39(1):1-38, 1977.","title":"Maximum likelihood from incomplete data via the em algorithm","authors":"A P Dempster, N M Laird, D B Rubin","year":"1977","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Do and Ohsaki, 2021] Bach Do and Makoto Ohsaki. Gaus- sian mixture model for robust design optimization of pla- nar steel frames. Structural and Multidisciplinary Opti- mization, 63(1):137-160, 2021.","title":"Bach Do and Makoto Ohsaki. Gaussian mixture model for robust design optimization of planar steel frames","authors":"Ohsaki Do","year":"2021","venue":"Structural and Multidisciplinary Optimization","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Duda et al., 1973] Richard O Duda, Peter E Hart, and David G Stork. Pattern classification and scene analysis, volume 3. Wiley New York, 1973.","title":"Pattern classification and scene analysis","authors":"Peter E Richard O Duda, David G Hart, Stork","year":"1973","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kolouri et al., 2018] Soheil Kolouri, Gustavo K Rohde, and Heiko Hoffmann. Sliced wasserstein distance for learning gaussian mixture models. In the 30th IEEE Conference on Computer Vision and Pattern Recognition, pages 3427- 3436, 2018.","title":"Sliced wasserstein distance for learning gaussian mixture models","authors":"Soheil Kolouri, Gustavo K Rohde, Heiko Hoffmann","year":"2018","venue":"the 30th IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kriegel et al., 2008] Hans-Peter Kriegel, Matthias Schu- bert, and Arthur Zimek. Angle-based outlier detection in high-dimensional data. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discov- ery and data mining, pages 444-452, 2008.","title":"Angle-based outlier detection in high-dimensional data","authors":"Kriegel","year":"2008","venue":"Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2012] Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation-based anomaly detection. ACM Transac- tions on Knowledge Discovery from Data (TKDD), 6(1):1- 39, 2012.","title":"Isolation-based anomaly detection","authors":"Tony Fei, Kai Liu, Ming Ting, Zhi-Hua Zhou","year":"2012","venue":"ACM Transactions on Knowledge Discovery from Data (TKDD)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2020] Yezheng Liu, Zhe Li, Chong Zhou, Yuanchun Jiang, Jianshan Sun, Meng Wang, and Xiang- nan He. Generative adversarial active learning for unsu- pervised outlier detection. IEEE Transactions on Knowl- edge and Data Engineering, 32(8):1517-1528, 2020.","title":"Generative adversarial active learning for unsupervised outlier detection","authors":"Yezheng Liu, Zhe Li, Chong Zhou, Yuanchun Jiang, Jianshan Sun, Meng Wang, Xiangnan He","year":"2020","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ma et al., 2019] Jiayi Ma, Xingyu Jiang, Junjun Jiang, and Yuan Gao. Feature-guided gaussian mixture model for im- age matching. Pattern Recognition, 92:231-245, 2019.","title":"Feature-guided gaussian mixture model for image matching","authors":"Jiayi Ma, Xingyu Jiang, Junjun Jiang, Yuan Gao","year":"2019","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[McLachlan and Krishnan, 1997] G. McLachlan and T. Kr- ishnan. The EM Algorithm and Extensions. Wiley, New York, 1997.","title":"The EM Algorithm and Extensions","authors":"Krishnan Mclachlan, G Mclachlan, T Krishnan","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Moon, 1996] T.K. Moon. The expectation-maximization al- gorithm. IEEE Signal Processing Magazine, 13(6):47-60, 1996.","title":"The expectation-maximization algorithm","authors":"T K Moon","year":"1996","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ng, 2000] Andrew Ng. Cs229 lecture notes. CS229 Lecture notes, 1(1):1-3, 2000.","title":"Cs229 lecture notes","authors":"Andrew Ng","year":"2000","venue":"CS229 Lecture notes","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Qu et al., 2020] Jiahui Qu, Qian Du, Yunsong Li, Long Tian, and Haoming Xia. Anomaly detection in hyper- spectral imagery based on gaussian mixture model. IEEE Transactions on Geoscience and Remote Sensing, pages 1-14, 2020.","title":"Anomaly detection in hyperspectral imagery based on gaussian mixture model","authors":"Jiahui Qu, Qian Du, Yunsong Li, Long Tian, Haoming Xia","year":"2020","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Rousseeuw and Hubert, 2011] Peter J Rousseeuw and Mia Hubert. Robust statistics for outlier detection. Wiley inter- disciplinary reviews: Data mining and knowledge discov- ery, 1(1):73-79, 2011.","title":"Robust statistics for outlier detection","authors":"Hubert ; Rousseeuw, J Peter, Mia Rousseeuw, Hubert","year":"2011","venue":"Wiley interdisciplinary reviews: Data mining and knowledge discovery","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sorenson and Alspach, 1971] Harold W Sorenson and Daniel L Alspach. Recursive bayesian estimation using gaussian sums. Automatica, 7(4):465-479, 1971.","title":"Recursive bayesian estimation using gaussian sums","authors":"W Harold, Daniel L Sorenson, Alspach","year":"1971","venue":"Automatica","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Venegas et al., 2019] Pablo Venegas, Noel Pérez, Diego Benítez, Román Lara-Cueva, and Mario Ruiz. Com- bining filter-based feature selection methods and gaus- sian mixture model for the classification of seismic events from cotopaxi volcano. IEEE Journal of Selected Top- ics in Applied Earth Observations and Remote Sensing, 12(6):1991-2003, 2019.","title":"Combining filter-based feature selection methods and gaussian mixture model for the classification of seismic events from cotopaxi volcano","authors":"Venegas","year":"2019","venue":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu and Wunsch, 2005] Rui Xu and D. Wunsch. Survey of clustering algorithms. IEEE Transactions on Neural Net- works, 16(3):645-678, 2005.","title":"Survey of clustering algorithms","authors":"Rui Xu, D Wunsch","year":"2005","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2014] Xianzhen Xu, Lei Xie, and Shuqing Wang. Multimode process monitoring with pca mixture model. Computers & Electrical Engineering, 40(7):2101-2112, 2014.","title":"Multimode process monitoring with pca mixture model","authors":"Xu","year":"2014","venue":"Computers & Electrical Engineering","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2019] Linxiao Yang, Ngai-Man Cheung, Jiay- ing Li, and Jun Fang. Deep clustering by gaussian mixture variational autoencoders with graph embedding. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 6440-6449, 2019.","title":"Deep clustering by gaussian mixture variational autoencoders with graph embedding","authors":"Yang","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhou et al., 2020] Xiaokang Zhou, Yiyong Hu, Wei Liang, Jianhua Ma, and Qun Jin. Variational lstm enhanced anomaly detection for industrial big data. IEEE Transac- tions on Industrial Informatics, 17(5):3469-3477, 2020.","title":"Variational lstm enhanced anomaly detection for industrial big data","authors":"Xiaokang Zhou, Yiyong Hu, Wei Liang, Jianhua Ma, Qun Jin","year":"2020","venue":"IEEE Transactions on Industrial Informatics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zong et al., 2018] Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In Interna- tional Conference on Learning Representations, 2018.","title":"Deep autoencoding gaussian mixture model for unsupervised anomaly detection","authors":"Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, Haifeng Chen","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zoran et al., 2012] Daniel Zoran, Yair Weiss, et al. Natural images, gaussian mixtures and dead leaves. In the 25th In- ternational Conference on Neural Information Processing Systems, volume 2, pages 1745-1753, 2012.","title":"Natural images, gaussian mixtures and dead leaves","authors":"Yair Daniel Zoran, Weiss","year":"2012","venue":"the 25th International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}