{"type":"ScholarlyArticle","title":"Belief Propagation Network for Hard Inductive Semi-Supervised Learning","authors":[{"name":"Jaemin Yoo","slug":"jaemin-yoo","given":"Jaemin","family":"Yoo","affiliation":"Seoul National University"},{"name":"Hyunsik Jeon","slug":"hyunsik-jeon","given":"Hyunsik","family":"Jeon","affiliation":"Seoul National University"},{"name":"U Kang","slug":"u-kang","given":"U","family":"Kang","affiliation":"Seoul National University"}],"abstract":"Given graph-structured data, how can we train a robust classifier in a semi-supervised setting that performs well without neighborhood information? In this work, we propose belief propagation networks (BPN), a novel approach to train a deep neural network in a hard inductive setting, where the test data are given without neighborhood information. BPN uses a differentiable classifier to compute the prior distributions of nodes, and then diffuses the priors through the graphical structure, independently from the prior computation. This separable structure improves the generalization performance of BPN for isolated test instances, compared with previous approaches that jointly use the feature and neighborhood without distinction. As a result, BPN outperforms state-of-the-art methods in four datasets with an average margin of 2.4% points in accuracy.","doi":"10.24963/ijcai.2019/580","url":"https://ijcai.org/proceedings/2019/580","pdf":"https://ijcai.org/proceedings/2019/0580.pdf","markdown":"https://ijcai.org/proceedings/2019/580.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/580","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":580,"pages":{"first":4178,"last":4184},"keywords":["Machine Learning: Classification","Machine Learning: Semi-Supervised Learning","Uncertainty in AI: Graphical Models","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Akoglu, 2014] Leman Akoglu. Quantifying political polar- ity based on bipartite opinion networks. In ICWSM, 2014.","title":"Leman Akoglu. Quantifying political polarity based on bipartite opinion networks","authors":"Akoglu","year":"2014","venue":"ICWSM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Atwood and Towsley, 2016] James Atwood and Don Towsley. Diffusion-convolutional neural networks. In NIPS, pages 1993-2001, 2016.","title":"Diffusion-convolutional neural networks","authors":"Towsley Atwood, James Atwood, Don Towsley","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Belkin et al., 2006] Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled ex- amples. Journal of Machine Learning Research, 7:2399- 2434, 2006.","title":"Manifold regularization: A geometric framework for learning from labeled and unlabeled examples","authors":"Mikhail Belkin, Partha Niyogi, Vikas Sindhwani","year":"2006","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chau et al., 2011] Duen Horng Chau, Carey Nachenberg, Jeffrey Wilhelm, Adam Wright, and Christos Faloutsos. Polonium: Tera-scale graph mining for malware detection. In SDM, pages 131-142, 2011.","title":"Polonium: Tera-scale graph mining for malware detection","authors":"Duen Horng, Chau, Carey Nachenberg, Jeffrey Wilhelm, Adam Wright, Christos Faloutsos","year":"2011","venue":"SDM","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2015] Liang-Chieh Chen, Alexander G. Schwing, Alan L. Yuille, and Raquel Urtasun. Learning deep structured models. In ICML, 2015.","title":"Learning deep structured models","authors":"Liang-Chieh Chen, Alexander G Schwing, Alan L Yuille, Raquel Urtasun","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In SIGKDD, pages 855-864, 2016.","title":"Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks","authors":"Leskovec Grover","year":"2016","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ha et al., 2012] Jiwoon Ha, Soon-Hyoung Kwon, Sang- Wook Kim, Christos Faloutsos, and Sunju Park. Top-n rec- ommendation through belief propagation. In CIKM, pages 2343-2346, 2012.","title":"Top-n recommendation through belief propagation","authors":"Ha","year":"2012","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hamilton et al., 2017] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, pages 1025-1035, 2017.","title":"Inductive representation learning on large graphs","authors":"William L Hamilton, Zhitao Ying, Jure Leskovec","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He and McAuley, 2016] Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fash- ion trends with one-class collaborative filtering. In WWW, pages 507-517, 2016.","title":"Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering","authors":"Mcauley He","year":"2016","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jang et al., 2016] Min-Hee Jang, Christos Faloutsos, Sang- Wook Kim, U. Kang, and Jiwoon Ha. PIN-TRUST: fast trust propagation exploiting positive, implicit, and nega- tive information. In CIKM, pages 629-638, 2016.","title":"PIN-TRUST: fast trust propagation exploiting positive, implicit, and negative information","authors":"Jang","year":"2016","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Koller and Friedman, 2009] Daphne Koller and Nir Fried- man. Probabilistic Graphical Models -Principles and Techniques. MIT Press, 2009.","title":"Probabilistic Graphical Models -Principles and Techniques","authors":"Friedman Koller","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2018] Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI, 2018.","title":"Deeper insights into graph convolutional networks for semi-supervised learning","authors":"Li","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liang et al., 2018] Jiongqian Liang, Peter Jacobs, Jiankai Sun, and Srinivasan Parthasarathy. Semi-supervised em- bedding in attributed networks with outliers. In SDM, pages 153-161, 2018.","title":"Semi-supervised embedding in attributed networks with outliers","authors":"Liang","year":"2018","venue":"SDM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liao et al., 2018] Renjie Liao, Marc Brockschmidt, Daniel Tarlow, Alexander L. Gaunt, Raquel Urtasun, and Richard S. Zemel. Graph partition neural networks for semi-supervised classification. In ICLR (Workshop), 2018.","title":"Graph partition neural networks for semi-supervised classification","authors":"Renjie Liao, Marc Brockschmidt, Daniel Tarlow, Alexander L Gaunt, Raquel Urtasun, Richard S Zemel","year":"2018","venue":"ICLR (Workshop)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liao et al., 2019] Renjie Liao, Zhizhen Zhao, Raquel Urta- sun, and Richard S. Zemel. Lanczosnet: Multi-scale deep graph convolutional networks. In ICLR, 2019.","title":"Lanczosnet: Multi-scale deep graph convolutional networks","authors":"Renjie Liao, Zhizhen Zhao, Raquel Urtasun, Richard S Zemel","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[McAuley et al., 2015] Julian J. McAuley, Christopher Tar- gett, Qinfeng Shi, and Anton van den Hengel. Image- based recommendations on styles and substitutes. In SI- GIR, pages 43-52, 2015.","title":"Imagebased recommendations on styles and substitutes","authors":"Julian J Mcauley, Christopher Targett, Qinfeng Shi, Anton Van Den, Hengel","year":"2015","venue":"SI-GIR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ng et al., 2018] Yin Cheng Ng, Nicolò Colombo, and Ri- cardo Silva. Bayesian semi-supervised learning with graph gaussian processes. In NeurIPS, pages 1690-1701, 2018.","title":"Bayesian semi-supervised learning with graph gaussian processes","authors":"Yin Cheng Ng, Nicolò Colombo, Ricardo Silva","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: online learning of social rep- resentations. In KDD, pages 701-710, 2014.","title":"Deepwalk: online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Saad, 2003] Yousef Saad. Iterative methods for sparse lin- ear systems. SIAM, 2003.","title":"Iterative methods for sparse linear systems","authors":"Yousef Saad","year":"2003","venue":"SIAM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sen et al., 2008] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, and Tina Eliassi- Rad. Collective classification in network data. AI Mag- azine, 29(3):93-106, 2008.","title":"Collective classification in network data","authors":"Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, Tina Eliassi-Rad","year":"2008","venue":"AI Magazine","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E. Hin- ton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tamersoy et al., 2014] Acar Tamersoy, Kevin A. Roundy, and Duen Horng Chau. Guilt by association: large scale malware detection by mining file-relation graphs. In KDD, pages 1524-1533, 2014.","title":"Guilt by association: large scale malware detection by mining file-relation graphs","authors":"Acar Tamersoy, Kevin A Roundy, Duen Horng, Chau","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Velickovic et al., 2018] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. ICLR, 2018.","title":"Graph attention networks","authors":"Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Weston et al., 2012] Jason Weston, Frédéric Ratle, Hossein Mobahi, and Ronan Collobert. Deep learning via semi- supervised embedding. In Neural Networks: Tricks of the Trade -Second Edition, pages 639-655. 2012.","title":"Deep learning via semisupervised embedding","authors":"Jason Weston, Frédéric Ratle, Hossein Mobahi, Ronan Collobert","year":"2012","venue":"Neural Networks: Tricks of the Trade -Second Edition","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yang and Pedersen, 1997] Yiming Yang and Jan O. Peder- sen. A comparative study on feature selection in text cate- gorization. In ICML, pages 412-420, 1997.","title":"A comparative study on feature selection in text categorization","authors":"Yiming Yang, Jan O Pedersen","year":"1997","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2016] Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learn- ing with graph embeddings. In ICML, 2016.","title":"Revisiting semi-supervised learning with graph embeddings","authors":"Zhilin Yang, William W Cohen, Ruslan Salakhutdinov","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yoo et al., 2017] Jaemin Yoo, Saehan Jo, and U. Kang. Su- pervised belief propagation: Scalable supervised inference on attributed networks. In ICDM, pages 595-604, 2017.","title":"Supervised belief propagation: Scalable supervised inference on attributed networks","authors":"Jaemin Yoo, Saehan Jo, U Kang","year":"2017","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yoon et al., 2018] KiJung Yoon, Renjie Liao, Yuwen Xiong, Lisa Zhang, Ethan Fetaya, Raquel Urtasun, Richard S. Zemel, and Xaq Pitkow. Inference in prob- abilistic graphical models by graph neural networks. In ICLR (Workshop), 2018.","title":"Inference in probabilistic graphical models by graph neural networks","authors":"Yoon","year":"2018","venue":"ICLR (Workshop)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}