{"type":"ScholarlyArticle","title":"Multi-Class Imbalanced Graph Convolutional Network Learning","authors":[{"name":"Min Shi","slug":"min-shi","given":"Min","family":"Shi","affiliation":"Department of Computer & Electrical Engineering and Computer Science , Florida Atlantic University , USA"},{"name":"Yufei Tang","slug":"yufei-tang","given":"Yufei","family":"Tang","affiliation":"Department of Computer & Electrical Engineering and Computer Science , Florida Atlantic University , USA"},{"name":"Xingquan Zhu","slug":"xingquan-zhu","given":"Xingquan","family":"Zhu","affiliation":"Department of Computer & Electrical Engineering and Computer Science , Florida Atlantic University , USA"},{"name":"David Wilson","slug":"david-wilson","given":"David","family":"Wilson","affiliation":"Department of Computer & Electrical Engineering and Computer Science , Florida Atlantic University , USA"},{"name":"Jianxun Liu","slug":"jianxun-liu","given":"Jianxun","family":"Liu","affiliation":"School of Computer Science and Engineering , Hunan University of Science and Technology , China"}],"abstract":"Networked data often demonstrate the Pareto principle (i.e., 80/20 rule) with skewed class distributions, where most vertices belong to a few majority classes and minority classes only contain a handful of instances. When presented with imbalanced class distributions, existing graph embedding learning tends to bias to nodes from majority classes, leaving nodes from minority classes under-trained. In this paper, we propose Dual-Regularized Graph Convolutional Networks (DR-GCN) to handle multi-class imbalanced graphs, where two types of regularization are imposed to tackle class imbalanced representation learning. To ensure that all classes are equally represented, we propose a class-conditioned adversarial training process to facilitate the separation of labeled nodes. Meanwhile, to maintain training equilibrium (i.e., retaining quality of fit across all classes), we force unlabeled nodes to follow a similar latent distribution to the labeled nodes by minimizing their difference in the embedding space. Experiments on real-world imbalanced graphs demonstrate that DR-GCN outperforms the state-of-the-art methods in node classification, graph clustering, and visualization.","doi":"10.24963/ijcai.2020/398","url":"https://ijcai.org/proceedings/2020/398","pdf":"https://ijcai.org/proceedings/2020/0398.pdf","markdown":"https://ijcai.org/proceedings/2020/398.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/398","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":398,"pages":{"first":2879,"last":2885},"keywords":["Machine Learning: Deep Learning: Convolutional networks","Data Mining: Classification, Semi-Supervised Learning","Data Mining: Mining Text, Web, Social Media"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24199/file_24199.m3u8","length_sec":303,"provider":"hls"}],"references":[{"position":1,"raw":"[Ahrendt, 2005] Peter Ahrendt. The multivariate gaussian probability distribution. Technical University of Denmark, Tech. Rep, 2005.","title":"The multivariate gaussian probability distribution","authors":"Peter Ahrendt","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chawla et al., 2002] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: syn- thetic minority over-sampling technique. Journal of artifi- cial intelligence research, 16:321-357, 2002.","title":"Smote: synthetic minority over-sampling technique","authors":"Kevin W Nitesh V Chawla, Lawrence O Bowyer, Philip Hall, Kegelmeyer","year":"2002","venue":"Journal of artificial intelligence research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2019] Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. Multi-label image recognition with graph convolutional networks. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pages 5177-5186, 2019.","title":"Multi-label image recognition with graph convolutional networks","authors":"Xiu-Shen Zhao-Min Chen, Peng Wei, Yanwen Wang, Guo","year":"2019","venue":"Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dong et al., 2018] Qi Dong, Shaogang Gong, and Xiatian Zhu. Imbalanced deep learning by minority class incre- mental rectification. IEEE Trans. on Pattern Analysis and Machine Intelligence, 41(6):1367-1381, 2018.","title":"Imbalanced deep learning by minority class incremental rectification","authors":"Dong","year":"2018","venue":"IEEE Trans. on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Drummond et al., 2003] Chris Drummond, Robert C Holte, et al. C4. 5, class imbalance, and cost sensitivity: why under-sampling beats over-sampling. In Workshop on learning from imbalanced datasets II, volume 11, pages 1-8, 2003.","title":"C4. 5, class imbalance, and cost sensitivity: why under-sampling beats over-sampling","authors":"Chris Drummond, Robert C Holte","year":"2003","venue":"Workshop on learning from imbalanced datasets II","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, pages 1024-1034, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He and Garcia, 2009] Haibo He and Edwardo A Garcia. Learning from imbalanced data. IEEE Transactions on knowledge and data engineering, 21(9):1263-1284, 2009.","title":"Learning from imbalanced data","authors":"Haibo He, Edwardo A Garcia","year":"2009","venue":"IEEE Transactions on knowledge and data engineering","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2016] Chen Huang, Yining Li, Chen Change Loy, and Xiaoou Tang. Learning deep rep- resentation for imbalanced classification. In Proc. of the IEEE conf. on Computer Vision and Pattern Recognition, pages 5375-5384, 2016.","title":"Learning deep representation for imbalanced classification","authors":"Chen Huang, Yining Li, Chen Change Loy, Xiaoou Tang","year":"2016","venue":"Proc. of the IEEE conf. on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Japkowicz and Stephen, 2002] Nathalie Japkowicz and Shaju Stephen. The class imbalance problem: A systematic study. Intell. Data Anal., 6(5):429-449, 2002.","title":"The class imbalance problem: A systematic study","authors":"Nathalie Japkowicz, Shaju Stephen","year":"2002","venue":"Intell. Data Anal","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Joyce, 2011] James M Joyce. Kullback-leibler divergence. Inl. Ency. of Statistical Science, pages 720-722, 2011.","title":"Kullback-leibler divergence","authors":"Joyce James","year":"2011","venue":"Inl. Ency. of Statistical Science","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In Intl. Conf. on Learning Representations, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":"Intl. Conf. on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lee et al., 2018] John Boaz Lee, Ryan A Rossi, Xiang- nan Kong, Sungchul Kim, Eunyee Koh, and Anup Rao. Higher-order graph convolutional networks. arXiv preprint arXiv:1809.07697, 2018.","title":"Higher-order graph convolutional networks","authors":"John Boaz, Lee, Ryan A Rossi, Xiangnan Kong, Sungchul Kim, Eunyee Koh, Anup Rao","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin and Cohen, 2010] Frank Lin and William W Cohen. Semi-supervised classification of network data using very few labels. In Intl. IEEE Conf. on Advances in Social Net- works Analysis and Mining, pages 192-199, 2010.","title":"Semi-supervised classification of network data using very few labels","authors":"Lin, Cohen, Frank Lin, William W Cohen","year":"2010","venue":"Intl. IEEE Conf. on Advances in Social Networks Analysis and Mining","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2008] Xu-Ying Liu, Jianxin Wu, and Zhi-Hua Zhou. Exploratory undersampling for class-imbalance learning. IEEE Trans. on Systems, Man, and Cybernetics, Part B (Cybernetics), 39(2):539-550, 2008.","title":"Exploratory undersampling for class-imbalance learning","authors":"Xu-Ying Liu, Jianxin Wu, Zhi-Hua Zhou","year":"2008","venue":"IEEE Trans. on Systems, Man, and Cybernetics, Part B (Cybernetics)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Kai Chen, Greg Cor- rado, and Jeffrey Dean. Efficient estimation of word rep- resentations in vector space. ICLR Workshop, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Mikolov","year":"2013","venue":"ICLR Workshop","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mirza and Osindero, 2014] Mehdi Mirza and Simon Osin- dero. Conditional generative adversarial nets. CoRR, abs/1411.1784, 2014.","title":"Conditional generative adversarial nets","authors":"Mehdi Mirza, Simon Osindero","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pan and Zhu, 2013] Shirui Pan and Xingquan Zhu. Graph classification with imbalanced class distributions and noise. In Proc. of the 23rd AAAI Conf. on Artificial In- telligence, pages 1586-1592, 2013.","title":"Graph classification with imbalanced class distributions and noise","authors":"Zhu Pan, Shirui Pan, Xingquan Zhu","year":"2013","venue":"Proc. of the 23rd AAAI Conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2013/236"},{"position":18,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, Steven Skiena, Rami Al-Rfou, and Steven Skiena. Deepwalk: On- line learning of social representations. In Proc. of the ACM SIGKDD Intl. Conf., pages 701-710, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"Proc. of the ACM SIGKDD Intl. Conf","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sun et al., 2009] Yanmin Sun, Andrew KC Wong, and Mo- hamed S Kamel. Classification of imbalanced data: A re- view. Intl. Journal of Pattern Recognition and Artificial Intelligence, 23(04):687-719, 2009.","title":"Classification of imbalanced data: A review","authors":"Yanmin Sun, Andrew Kc Wong, Mohamed S Kamel","year":"2009","venue":"Intl. Journal of Pattern Recognition and Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tai et al., 2015] Kai Sheng Tai, Richard Socher, and Socher Richard Manning, Christopher D, and Christo- pher D Manning. Improved semantic representations from tree-structured long short-term memory networks. In Proc. of the 53rd Annual Meeting of the Association for Compu- tational Linguistics, page 1556-1566, 2015.","title":"Improved semantic representations from tree-structured long short-term memory networks","authors":"Kai Sheng, Tai, Richard Socher, Socher Richard Manning, D Christopher, Christopher D Manning","year":"2015","venue":"Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In Intl. Conf. on Learning Representations, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2018","venue":"Intl. Conf. on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2019] Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. Knowledge graph convolu- tional networks for recommender systems. In Proc. of the intl. conf. on World Wide Web, pages 3307-3313, 2019.","title":"Knowledge graph convolutional networks for recommender systems","authors":"Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, Minyi Guo","year":"2019","venue":"Proc. of the intl. conf. on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wu et al., 2020] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A com- prehensive survey on graph neural networks. IEEE Trans- actions on Neural Networks and Learning Systems, 2020.","title":"A comprehensive survey on graph neural networks","authors":"Wu","year":"2020","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xia et al., 2014] Rongkai Xia, Yan Pan, Lei Du, and Jian Yin. Robust multi-view spectral clustering via low-rank and sparse decomposition. In Proc. of the 28th AAAI conf. on Artificial Intelligence, 2014.","title":"Robust multi-view spectral clustering via low-rank and sparse decomposition","authors":"Xia","year":"2014","venue":"Proc. of the 28th AAAI conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yao et al., 2019] Liang Yao, Chengsheng Mao, and Yuan Luo. Graph convolutional networks for text classification. In Proc. of the 33rd AAAI Conf. on Artificial Intelligence, pages 7370-7377, 2019.","title":"Graph convolutional networks for text classification","authors":"Liang Yao, Chengsheng Mao, Yuan Luo","year":"2019","venue":"Proc. of the 33rd AAAI Conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2019] Yingxue Zhang, Soumyasundar Pal, Mark Coates, and Deniz Ustebay. Bayesian graph con- volutional neural networks for semi-supervised classifica- tion. In Proc. of the 33rd AAAI Conf. on Artificial Intelli- gence, pages 5829-5836, 2019.","title":"Bayesian graph convolutional neural networks for semi-supervised classification","authors":"Zhang","year":"2019","venue":"Proc. of the 33rd AAAI Conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2020] Daokun Zhang, Jie Yin, Xingquan Zhu, and Chengqi Zhang. Network representation learning: A survey. IEEE Trans. on Big Data, 6(1):3-28, 2020.","title":"Network representation learning: A survey","authors":"Zhang","year":"2020","venue":"IEEE Trans. on Big Data","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}