{"type":"ScholarlyArticle","title":"Decorrelated Clustering with Data Selection Bias","authors":[{"name":"Xiao Wang","slug":"xiao-wang","given":"Xiao","family":"Wang","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Shaohua Fan","slug":"shaohua-fan","given":"Shaohua","family":"Fan","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Kun Kuang","slug":"kun-kuang","given":"Kun","family":"Kuang","affiliation":"Zhejiang University"},{"name":"Chuan Shi","slug":"chuan-shi","given":"Chuan","family":"Shi","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Jiawei Liu","slug":"jiawei-liu","given":"Jiawei","family":"Liu","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Bai Wang","slug":"bai-wang","given":"Bai","family":"Wang","affiliation":"Beijing University of Posts and Telecommunications"}],"abstract":"Most of existing clustering algorithms are proposed without considering the selection bias in data. In many real applications, however, one cannot guarantee the data is unbiased. Selection bias might bring the unexpected correlation between features and ignoring those unexpected correlations will hurt the performance of clustering algorithms. Therefore, how to remove those unexpected correlations induced by selection bias is extremely important yet largely unexplored for clustering. In this paper, we propose a novel Decorrelation regularized K-Means algorithm (DCKM) for clustering with data selection bias. Specifically, the decorrelation regularizer aims to learn the global sample weights which are capable of balancing the sample distribution, so as to remove unexpected correlations among features. Meanwhile, the learned weights are combined with k-means, which makes the reweighted k-means cluster on the inherent data distribution without unexpected correlation influence. Moreover, we derive the updating rules to effectively infer the parameters in DCKM. Extensive experiments results on real world datasets well demonstrate that our DCKM algorithm achieves significant performance gains, indicating the necessity of removing unexpected feature correlations induced by selection bias when clustering.","doi":"10.24963/ijcai.2020/301","url":"https://ijcai.org/proceedings/2020/301","pdf":"https://ijcai.org/proceedings/2020/0301.pdf","markdown":"https://ijcai.org/proceedings/2020/301.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/301","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":301,"pages":{"first":2177,"last":2183},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Clustering","Data Mining: Clustering, Unsupervised Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24441/file_24441.m3u8","length_sec":300,"provider":"hls"}],"references":[{"position":1,"raw":"[Bachem et al., 2018] Olivier Bachem, Mario Lucic, and Andreas Krause. Scalable k-means clustering via lightweight coresets. In SIGKDD, pages 1119-1127. ACM, 2018.","title":"Scalable k-means clustering via lightweight coresets","authors":"Bachem","year":"2018","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bay et al., 2006] Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. Surf: Speeded up robust features. In ECCV, pages 404-417. Springer, 2006.","title":"Surf: Speeded up robust features","authors":"Bay","year":"2006","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio and Bergstra, 2009] Yoshua Bengio and James S Bergstra. Slow, decorrelated features for pretraining com- plex cell-like networks. In NIPS, pages 99-107, 2009.","title":"Slow, decorrelated features for pretraining complex cell-like networks","authors":"Yoshua Bengio, James S Bergstra","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cogswell et al., 2016] Michael Cogswell, Faruk Ahmed, Ross Girshick, Larry Zitnick, and Dhruv Batra. Reducing overfitting in deep networks by decorrelating representa- tions. In ICLR, 2016.","title":"Reducing overfitting in deep networks by decorrelating representations","authors":"Michael Cogswell, Faruk Ahmed, Ross Girshick, Larry Zitnick, Dhruv Batra","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ding and He, 2004] Chris Ding and Xiaofeng He. K-means clustering via principal component analysis. In ICML, page 29. ACM, 2004.","title":"K-means clustering via principal component analysis","authors":"Chris Ding, Xiaofeng He","year":"2004","venue":"ICML, page 29","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ding et al., 2005] Chris Ding, Xiaofeng He, and Horst D Si- mon. Nonnegative lagrangian relaxation of k-means and spectral clustering. In ECML, pages 530-538. Springer, 2005.","title":"Nonnegative lagrangian relaxation of k-means and spectral clustering","authors":"Chris Ding, Xiaofeng He, Horst D Simon","year":"2005","venue":"ECML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Du and Shen, 2015] Liang Du and Yi-Dong Shen. Unsuper- vised feature selection with adaptive structure learning. In SIGKDD, pages 209-218. ACM, 2015.","title":"Unsupervised feature selection with adaptive structure learning","authors":"Liang Du, Yi-Dong Shen","year":"2015","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fan et al., 2020] Shaohua Fan, Xiao Wang, Chuan Shi, Emiao Lu, Ken Lin, and Bai Wang. One2multi graph au- toencoder for multi-view graph clustering. In Proceedings of The Web Conference 2020, pages 3070-3076, 2020.","title":"One2multi graph autoencoder for multi-view graph clustering","authors":"Xiao Shaohua Fan, Chuan Wang, Emiao Shi, Ken Lu, Bai Lin, Wang","year":"2020","venue":"Proceedings of The Web Conference 2020","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gong et al., 2012] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In CVPR, pages 2066-2073. IEEE, 2012.","title":"Geodesic flow kernel for unsupervised domain adaptation","authors":"Gong","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hartigan and Wong, 1979] John A Hartigan and Manchek A Wong. Algorithm as 136: A k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 28(1):100-108, 1979.","title":"Algorithm as 136: A k-means clustering algorithm","authors":"Wong ; ] Hartigan, A John, Manchek A Hartigan, Wong","year":"1979","venue":"Journal of the Royal Statistical Society. Series C (Applied Statistics)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Heckman, 1979] James J Heckman. Sample selection bias as a specification error. Econometrica: Journal of the econometric society, pages 153-161, 1979.","title":"Sample selection bias as a specification error","authors":"J James, Heckman","year":"1979","venue":"Econometrica: Journal of the econometric society","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2007] Jiayuan Huang, Arthur Gretton, Karsten Borgwardt, Bernhard Schölkopf, and Alex J Smola. Correcting sample selection bias by unlabeled data. In NIPS, pages 601-608, 2007.","title":"Correcting sample selection bias by unlabeled data","authors":"Huang","year":"2007","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kriegel et al., 2009] Hans-Peter Kriegel, Peer Kröger, and Arthur Zimek. Clustering high-dimensional data: A sur- vey on subspace clustering, pattern-based clustering, and correlation clustering. TKDD, 3(1):1, 2009.","title":"Clustering high-dimensional data: A survey on subspace clustering, pattern-based clustering, and correlation clustering","authors":"Kriegel","year":"2009","venue":"TKDD","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kuang et al., 2018] Kun Kuang, Peng Cui, Susan Athey, Ruoxuan Xiong, and Bo Li. Stable prediction across unknown environments. In SIGKDD, pages 1617-1626. ACM, 2018.","title":"Stable prediction across unknown environments","authors":"Kun Kuang, Peng Cui, Susan Athey, Ruoxuan Xiong, Bo Li","year":"2018","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kuang et al., 2020] Kun Kuang, Ruoxuan Xiong, Peng Cui, Susan Athey, and Bo Li. Stable prediction with model misspecification and agnostic distribution shift. In AAAI, 2020.","title":"Stable prediction with model misspecification and agnostic distribution shift","authors":"Kun Kuang, Ruoxuan Xiong, Peng Cui, Susan Athey, Bo Li","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2017] Jundong Li, Jiliang Tang, and Huan Liu. Reconstruction-based unsupervised feature selection: An embedded approach. In IJCAI, pages 2159-2165, 2017.","title":"Reconstruction-based unsupervised feature selection: An embedded approach","authors":"Li","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/300"},{"position":17,"raw":"[Long et al., 2014] Mingsheng Long, Jianmin Wang, Guiguang Ding, Jiaguang Sun, and Philip S Yu. Transfer joint matching for unsupervised domain adaptation. In CVPR, pages 1410-1417, 2014.","title":"Transfer joint matching for unsupervised domain adaptation","authors":"Mingsheng Long, Jianmin Wang, Guiguang Ding, Jiaguang Sun, Philip S Yu","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Qian and Zhai, 2013] Mingjie Qian and Chengxiang Zhai. Robust unsupervised feature selection. In IJCAI, 2013.","title":"Robust unsupervised feature selection","authors":"Mingjie Qian, Chengxiang Zhai","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/241"},{"position":19,"raw":"[Rodríguez et al., 2017] Pau Rodríguez, Jordi Gonzalez, Guillem Cucurull, Josep M Gonfaus, and Xavier Roca. Regularizing cnns with locally constrained decorrelations. In ICLR, 2017.","title":"Regularizing cnns with locally constrained decorrelations","authors":"Rodríguez","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Schmidt et al., 2018] Melanie Schmidt, Chris Schwiegelshohn, and Christian Sohler. Fair coresets and streaming algorithms for fair k-means clustering. arXiv preprint arXiv:1812.10854, 2018.","title":"Fair coresets and streaming algorithms for fair k-means clustering","authors":"Melanie Schmidt, Chris Schwiegelshohn, Christian Sohler","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shen et al., 2018] Zheyan Shen, Peng Cui, Kun Kuang, Bo Li, and Peixuan Chen. Causally regularized learning with agnostic data selection bias. In ACM MM, pages 411- 419, 2018.","title":"Causally regularized learning with agnostic data selection bias","authors":"Zheyan Shen, Peng Cui, Kun Kuang, Bo Li, Peixuan Chen","year":"2018","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shen et al., 2020] Zheyan Shen, Peng Cui, Tong Zhang, and Kun Kuang. Stable learning of linear models via sample reweighting. In AAAI, 2020.","title":"Stable learning of linear models via sample reweighting","authors":"Zheyan Shen, Peng Cui, Tong Zhang, Kun Kuang","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Venkateswara et al., 2017] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Pan- chanathan. Deep hashing network for unsupervised do- main adaptation. In CVPR, 2017.","title":"Deep hashing network for unsupervised domain adaptation","authors":"Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, Sethuraman Panchanathan","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Von Luxburg, 2007] Ulrike Von Luxburg. A tutorial on spectral clustering. Statistics and computing, 17(4):395- 416, 2007.","title":"A tutorial on spectral clustering","authors":"Ulrike Von, Luxburg","year":"2007","venue":"Statistics and computing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2018] Zijun Zhang, Yining Zhang, and Zong- peng Li. Removing the feature correlation effect of multi- plicative noise. In NIPS, pages 627-636, 2018.","title":"Removing the feature correlation effect of multiplicative noise","authors":"Zijun Zhang, Yining Zhang, Zongpeng Li","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}