{"type":"ScholarlyArticle","title":"End-to-End Open-Set Semi-Supervised Node Classification with Out-of-Distribution Detection","authors":[{"name":"Tiancheng Huang","slug":"tiancheng-huang","given":"Tiancheng","family":"Huang","affiliation":"Zhejiang University , Hangzhou , China; Westlake University , Hangzhou , China; Westlake Institute for Advanced Study , Hangzhou , China"},{"name":"Donglin Wang","slug":"donglin-wang","given":"Donglin","family":"Wang","affiliation":"Westlake University , Hangzhou , China; Westlake Institute for Advanced Study , Hangzhou , China"},{"name":"Yuan Fang","slug":"yuan-fang","given":"Yuan","family":"Fang","affiliation":"Singapore Management University , Singapore"},{"name":"Zhengyu Chen","slug":"zhengyu-chen","given":"Zhengyu","family":"Chen","affiliation":"Westlake University , Hangzhou , China; Westlake Institute for Advanced Study , Hangzhou , China"}],"abstract":"Out-Of-Distribution (OOD) samples are prevalent in real-world applications. The OOD issue becomes even more severe on graph data, as the effect of OOD nodes can be potentially amplified by propagation through the graph topology. Recent works have considered the OOD detection problem, which is critical for reducing the uncertainty in learning and improving the robustness. However, no prior work considers simultaneously OOD detection and node classification on graphs in an end-to-end manner. In this paper, we study a novel problem of end-to-end open-set semi-supervised node classification (OSSNC) on graphs, which deals with node classification in the presence of OOD nodes. Given the lack of supervision on OOD nodes, we introduce a latent variable to indicate in-distribution or OOD nodes in a variational inference framework, and further propose a novel algorithm named as Learning to Mix Neighbors (LMN) which learns to dampen the influence of OOD nodes through the messaging-passing in typical graph neural networks. Extensive experiments on various datasets show that the proposed method outperforms state-of-the-art baselines in terms of both node classification and OOD detection.","doi":"10.24963/ijcai.2022/290","url":"https://ijcai.org/proceedings/2022/290","pdf":"https://ijcai.org/proceedings/2022/0290.pdf","markdown":"https://ijcai.org/proceedings/2022/290.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/290","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":290,"pages":{"first":2087,"last":2093},"keywords":["Data Mining: Mining Graphs","Machine Learning: Semi-Supervised Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986887","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985030","length_sec":333,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bitterwolf et al., 2022] Julian Bitterwolf, Alexander Meinke, Maximilian Augustin, and Matthias Hein. Re- visiting ood detection: A simple baseline is surprisingly effective. In ICLR 2022 Submitted, 2022.","title":"Revisiting ood detection: A simple baseline is surprisingly effective","authors":"Julian Bitterwolf, Alexander Meinke, Maximilian Augustin, Matthias Hein","year":"2022","venue":"ICLR 2022 Submitted","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2020] Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In ICML, 2020.","title":"Simple and deep graph convolutional networks","authors":"Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, Yaliang Li","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dai et al., 2021] Enyan Dai, Zhimeng Guo, and Suhang Wang. Label-wise message passing graph neural network on heterophilic graphs. In CoRR, 2021.","title":"Label-wise message passing graph neural network on heterophilic graphs","authors":"Dai","year":"2021","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Defferrard et al., 2016] Michaël Defferrard, Xavier Bres- son, and Pierre Vandergheynst. Convolutional neural net- works on graphs with fast localized spectral filtering. In NeurIPS, 2016.","title":"Convolutional neural networks on graphs with fast localized spectral filtering","authors":"Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Denouden et al., 2018] Taylor Denouden, Rick Salay, Krzysztof Czarnecki, Vahdat Abdelzad, Buu Phan, and Sachin Vernekar. Improving reconstruction autoencoder ood detection with mahalanobis distance. CoRR, 2018.","title":"Improving reconstruction autoencoder ood detection with mahalanobis distance","authors":"Taylor Denouden, Rick Salay, Krzysztof Czarnecki, Buu Vahdat Abdelzad, Sachin Phan, Vernekar","year":"2018","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[DeVries and Taylor, 2018] Terrance DeVries and Gra- ham W Taylor. Learning confidence for out-of-distribution detection in neural networks. CoRR, 2018.","title":"Learning confidence for out-of-distribution detection in neural networks","authors":"Terrance Devries, Graham W Taylor","year":"2018","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Geisa et al., 2021] Ali Geisa, Ronak Mehta, Hayden S Helm, Jayanta Dey, Eric Eaton, Jeffery Dick, Carey E Priebe, and Joshua T Vogelstein. Towards a theory of out- of-distribution learning. CoRR, 2021.","title":"Towards a theory of outof-distribution learning","authors":"Ali Geisa, Ronak Mehta, Hayden S Helm, Jayanta Dey, Eric Eaton, Jeffery Dick, Carey E Priebe, Joshua T Vogelstein","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hamilton et al., 2017] William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NeurIPS, 2017.","title":"Inductive representation learning on large graphs","authors":"Rex William L Hamilton, Jure Ying, Leskovec","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hendrycks and Gimpel, 2016] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out- of-distribution examples in neural networks. CoRR, 2016.","title":"A baseline for detecting misclassified and outof-distribution examples in neural networks","authors":"Gimpel Hendrycks, Dan Hendrycks, Kevin Gimpel","year":"2016","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hsu et al., 2020] Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-of-distribution image without learning from out-of- distribution data. In CVPR, 2020.","title":"Generalized odin: Detecting out-of-distribution image without learning from out-ofdistribution data","authors":"Yen-Chang Hsu, Yilin Shen, Hongxia Jin, Zsolt Kira","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2020] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In NeurIPS, 2020.","title":"Open graph benchmark: Datasets for machine learning on graphs","authors":"Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, Jure Leskovec","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kim and Oh, 2020] Dongkwan Kim and Alice Oh. How to find your friendly neighborhood: Graph attention design with self-supervision. In ICLR, 2020.","title":"Dongkwan Kim and Alice Oh. How to find your friendly neighborhood: Graph attention design with self-supervision","authors":"Oh Kim","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In CoRR, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kipf and Welling, 2017] Thomas N Kipf and Max Welling. Semi-supervised classification with graph neural net- works. In ICLR, 2017. [Klicpera et al., 2018] Johannes Klicpera, Aleksandar Bo- jchevski, and Stephan Günnemann. Predict then propa- gate: Gnns meet personalized pagerank. CoRR, 2018.","title":"Semi-supervised classification with graph neural networks","authors":"N Thomas, Max Kipf, Welling","year":"2017","venue":"Predict then propagate: Gnns meet personalized pagerank","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lee et al., 2018] Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrated classifiers for detecting out-of-distribution samples. In ICLR, 2018.","title":"Training confidence-calibrated classifiers for detecting out-of-distribution samples","authors":"Lee","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liang et al., 2018] Shiyu Liang, Yixuan Li, and R Srikant. Enhancing the reliability of out-of-distribution image de- tection in neural networks. In ICLR, 2018.","title":"Enhancing the reliability of out-of-distribution image detection in neural networks","authors":"Shiyu Liang, Yixuan Li, Srikant","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Luo et al., 2021] Dongsheng Luo, Wei Cheng, Wenchao Yu, Bo Zong, Jingchao Ni, Haifeng Chen, and Xiang Zhang. Learning to drop: Robust graph neural network via topological denoising. In WSDM, 2021.","title":"Learning to drop: Robust graph neural network via topological denoising","authors":"Dongsheng Luo, Wei Cheng, Wenchao Yu, Bo Zong, Jingchao Ni, Haifeng Chen, Xiang Zhang","year":"2021","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ma et al., 2021] Xiaoxiao Ma, Jia Wu, Shan Xue, Jian Yang, Quan Z Sheng, and Hui Xiong. A comprehensive survey on graph anomaly detection with deep learning. CoRR, 2021.","title":"A comprehensive survey on graph anomaly detection with deep learning","authors":"Xiaoxiao Ma, Jia Wu, Shan Xue, Jian Yang, Hui Quan Z Sheng, Xiong","year":"2021","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mukhoti et al., 2021] Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. Deep deterministic uncertainty: A simple baseline. In CoRR, 2021.","title":"Deep deterministic uncertainty: A simple baseline","authors":"Jishnu Mukhoti, Andreas Kirsch, Joost Van Amersfoort, Philip Hs Torr, Yarin Gal","year":"2021","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pimentel et al., 2014] Marco AF Pimentel, David A Clifton, Lei Clifton, and Lionel Tarassenko. A review of novelty detection. Signal Processing, 2014.","title":"A review of novelty detection","authors":"A F Marco, David A Pimentel, Lei Clifton, Lionel Clifton, Tarassenko","year":"2014","venue":"Signal Processing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rong et al., 2019] Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. Dropedge: Towards deep graph con- volutional networks on node classification. In ICLR, 2019.","title":"Dropedge: Towards deep graph convolutional networks on node classification","authors":"Yu Rong, Wenbing Huang, Tingyang Xu, Junzhou Huang","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Stadler et al., 2021] Maximilian Stadler, Bertrand Charp- entier, Simon Geisler, Daniel Zügner, and Stephan Günnemann. Graph posterior network: Bayesian predic- tive uncertainty for node classification. In NeurIPS, 2021.","title":"Graph posterior network: Bayesian predictive uncertainty for node classification","authors":"Maximilian Stadler, Bertrand Charpentier, Simon Geisler, Daniel Zügner, Stephan Günnemann","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vernekar et al., 2019] Sachin Vernekar, Ashish Gaurav, Vahdat Abdelzad, Taylor Denouden, Rick Salay, and Krzysztof Czarnecki. Out-of-distribution detection in clas- sifiers via generation. CoRR, 2019.","title":"Out-of-distribution detection in classifiers via generation","authors":"Sachin Vernekar, Ashish Gaurav, Taylor Vahdat Abdelzad, Rick Denouden, Krzysztof Salay, Czarnecki","year":"2019","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wu et al., 2019] Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Sim- plifying graph convolutional networks. In ICML, 2019.","title":"Simplifying graph convolutional networks","authors":"Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, Kilian Weinberger","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2020] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A com- prehensive survey on gnns. TNNLS, 2020.","title":"A comprehensive survey on gnns","authors":"Wu","year":"2020","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xiao et al., 2021] Teng Xiao, Zhengyu Chen, Donglin Wang, and Suhang Wang. Learning how to propagate mes- sages in graph neural networks. In SIGKDD, 2021.","title":"Learning how to propagate messages in graph neural networks","authors":"Teng Xiao, Zhengyu Chen, Donglin Wang, Suhang Wang","year":"2021","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xu et al., 2018] Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In ICML, 2018.","title":"Representation learning on graphs with jumping knowledge networks","authors":"Xu","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhao et al., 2020] Xujiang Zhao, Feng Chen, Shu Hu, and Jin-Hee Cho. Uncertainty aware semi-supervised learning on graph data. In CoRR, 2020.","title":"Uncertainty aware semi-supervised learning on graph data","authors":"Xujiang Zhao, Feng Chen, Shu Hu, Jin-Hee Cho","year":"2020","venue":"CoRR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}