{"type":"ScholarlyArticle","title":"Coloring Graph Neural Networks for Node Disambiguation","authors":[{"name":"George Dasoulas","slug":"george-dasoulas","given":"George","family":"Dasoulas","affiliation":"Huawei Noah's Ark Lab; École Polytechnique , Paris , France"},{"name":"Ludovic Dos Santos","slug":"ludovic-dos-santos","given":"Ludovic","family":"Dos Santos","affiliation":"Huawei Noah's Ark Lab"},{"name":"Kevin Scaman","slug":"kevin-scaman","given":"Kevin","family":"Scaman","affiliation":"Huawei Noah's Ark Lab"},{"name":"Aladin Virmaux","slug":"aladin-virmaux","given":"Aladin","family":"Virmaux","affiliation":"Huawei Noah's Ark Lab"}],"abstract":"In this paper, we show that a simple coloring scheme can improve, both theoretically and empirically, the expressive power of Message Passing Neural Networks (MPNNs). More specifically, we introduce a graph neural network called Colored Local Iterative Procedure (CLIP) that uses colors to disambiguate identical node attributes, and show that this representation is a universal approximator of continuous functions on graphs with node attributes. Our method relies on separability, a key topological characteristic that allows to extend well-chosen neural networks into universal representations. Finally, we show experimentally that CLIP is capable of capturing structural characteristics that traditional MPNNs fail to distinguish, while being state-of-the-art on benchmark graph classification datasets.","doi":"10.24963/ijcai.2020/294","url":"https://ijcai.org/proceedings/2020/294","pdf":"https://ijcai.org/proceedings/2020/0294.pdf","markdown":"https://ijcai.org/proceedings/2020/294.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/294","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":294,"pages":{"first":2126,"last":2132},"keywords":["Machine Learning: Deep Learning","Data Mining: Mining Graphs, Semi Structured Data, Complex Data","Machine Learning: Relational Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24838/file_24838.m3u8","length_sec":260,"provider":"hls"}],"references":[{"position":1,"raw":"[Atwood and Towsley, 2016] James Atwood and Don Towsley. Diffusion-convolutional neural networks. In Advances in Neural Information Processing Systems, 2016.","title":"Diffusion-convolutional neural networks","authors":"Towsley Atwood, James Atwood, Don Towsley","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Babai, 2016] László Babai. Graph isomorphism in quasipolynomial time. In Proceedings of the forty-eighth annual ACM symposium on Theory of Computing, pages 684-697. ACM, 2016.","title":"Graph isomorphism in quasipolynomial time","authors":"László Babai, Babai","year":"2016","venue":"Proceedings of the forty-eighth annual ACM symposium on Theory of Computing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio et al., 2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. PAMI, 2013.","title":"Representation learning: A review and new perspectives","authors":"Yoshua Bengio, Aaron Courville, Pascal Vincent","year":"2013","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bodlaender, 1990] Hans L Bodlaender. Polynomial algorithms for graph isomorphism and chromatic index on partial k-trees. Journal of Algorithms, 11(4):631-643, 1990.","title":"Polynomial algorithms for graph isomorphism and chromatic index on partial k-trees","authors":"L Hans, Bodlaender","year":"1990","venue":"Journal of Algorithms","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bourbaki, 1998] N. Bourbaki. General Topology: Chapters 1-4. Number vol. 4 in Addison-Wesley series in mathematics. Springer, 1998.","title":"General Topology: Chapters 1-4","authors":"N Bourbaki","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bruna et al., 2014] Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann Lecun. Spectral networks and locally connected net- works on graphs. ICLR, 2014.","title":"Spectral networks and locally connected networks on graphs","authors":"Joan Bruna, Wojciech Zaremba, Arthur Szlam, Yann Lecun","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2019] Z. Chen, S. Villar, L. Chen, and J. Bruna. On the equivalence between graph isomorphism testing and function approximation with gnns. NeurIPS 2019, 2019.","title":"On the equivalence between graph isomorphism testing and function approximation with gnns","authors":"Z Chen, S Villar, L Chen, J Bruna","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cybenko, 1989] George Cybenko. Approximation by superposi- tions of a sigmoidal function. Mathematics of control, signals and systems, 1989.","title":"Approximation by superpositions of a sigmoidal function","authors":"George Cybenko","year":"1989","venue":"Mathematics of control, signals and systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Defferrard et al., 2016] M. Defferrard, X. Bresson, and P. Van- dergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In NIPS, 2016.","title":"Convolutional neural networks on graphs with fast localized spectral filtering","authors":"M Defferrard, X Bresson, P Vandergheynst","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Duvenaud et al., 2015] D. K Duvenaud, D. Maclaurin, J. Ipar- raguirre, R. Bombarell, T. Hirzel, A. Aspuru-Guzik, and R. P Adams. Convolutional networks on graphs for learning molecular fingerprints. In NIPS, 2015.","title":"Convolutional networks on graphs for learning molecular fingerprints","authors":"D Duvenaud, D Maclaurin, J Iparraguirre, R Bombarell, T Hirzel, A Aspuru-Guzik, R P Adams","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Frasconi et al., 1998] Paolo Frasconi, Marco Gori, and Alessandro Sperduti. A general framework for adaptive processing of data structures. IEEE transactions on Neural Networks, 1998.","title":"A general framework for adaptive processing of data structures","authors":"Paolo Frasconi, Marco Gori, Alessandro Sperduti","year":"1998","venue":"IEEE transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gilmer et al., 2017] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. ICML, 2017.","title":"Neural message passing for quantum chemistry","authors":"Justin Gilmer, S Samuel, Patrick F Schoenholz, Oriol Riley, George E Vinyals, Dahl","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gori et al., 2005] Marco Gori, Gabriele Monfardini, and Franco Scarselli. A new model for learning in graph domains. In IJCNN, 2005.","title":"A new model for learning in graph domains","authors":"Marco Gori, Gabriele Monfardini, Franco Scarselli","year":"2005","venue":"IJCNN","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In SIGKDD, 2016.","title":"node2vec: Scalable feature learning for networks","authors":"Leskovec Grover, Aditya Grover, Jure Leskovec","year":"2016","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, pages 1024- 1034, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Henaff et al., 2015] Mikael Henaff, Joan Bruna, and Yann LeCun. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163, 2015.","title":"Deep convolutional networks on graph-structured data","authors":"Mikael Henaff, Joan Bruna, Yann Lecun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hornik et al., 1989] Kurt Hornik, Maxwell Stinchcombe, and Hal- bert White. Multilayer feedforward networks are universal ap- proximators. Neural networks, 2(5):359-366, 1989.","title":"Multilayer feedforward networks are universal approximators","authors":"Kurt Hornik, Maxwell Stinchcombe, Halbert White","year":"1989","venue":"Neural networks","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ivanov and Burnaev, 2018] Sergey Ivanov and Evgeny Burnaev. Anonymous walk embeddings. ICML, 2018.","title":"Anonymous walk embeddings","authors":"Sergey Ivanov, Evgeny Burnaev","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kipf and Welling, 2017] Thomas N Kipf and Max Welling. Semi- supervised classification with graph convolutional networks. ICLR, 2017.","title":"Semisupervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kriege et al., 2018] N. M. Kriege, C. Morris, A. Rey, and C. Sohler. A property testing framework for the theoretical expressivity of graph kernels. In IJCAI, 2018.","title":"A property testing framework for the theoretical expressivity of graph kernels","authors":"N M Kriege, C Morris, A Rey, C Sohler","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/325"},{"position":21,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Ge- offrey E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, pages 1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lei et al., 2017] Tao Lei, Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Deriving neural architectures from sequence and graph kernels. ICML, 2017.","title":"Deriving neural architectures from sequence and graph kernels","authors":"Tao Lei, Wengong Jin, Regina Barzilay, Tommi Jaakkola","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Luks, 1982] Eugene M. Luks. Isomorphism of graphs of bounded valence can be tested in polynomial time. Journal of Computer and System Sciences, 25(1):42 -65, 1982.","title":"Isomorphism of graphs of bounded valence can be tested in polynomial time","authors":"Eugene M Luks","year":"1982","venue":"Journal of Computer and System Sciences","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Maron et al., 2019a] Haggai Maron, Heli Ben-Hamu, Hadar Ser- viansky, and Yaron Lipman. Provably powerful graph networks. NeurIPS, 2019.","title":"Provably powerful graph networks","authors":"Maron","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Maron et al., 2019b] Haggai Maron, Heli Ben-Hamu, Nadav Shamir, and Yaron Lipman. Invariant and equivariant graph net- works. In International Conference on Learning Representations, 2019.","title":"Invariant and equivariant graph networks","authors":"Maron","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Maron et al., 2019c] Haggai Maron, Ethan Fetaya, Nimrod Segol, and Yaron Lipman. On the universality of invariant networks. ICML 2019, 2019.","title":"On the universality of invariant networks","authors":"Maron","year":"2019","venue":"ICML 2019","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[McKay, 1981] Brendan D McKay. Practical graph isomorphism. Congressus Numerantium, 30:45-87, 1981.","title":"Practical graph isomorphism","authors":"Brendan D Mckay","year":"1981","venue":"Congressus Numerantium","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Murphy et al., 2019] R. Murphy, B. Srinivasan, V. Rao, and B. Ribeiro. Relational pooling for graph representations. In ICML, 2019.","title":"Relational pooling for graph representations","authors":"R Murphy, B Srinivasan, V Rao, B Ribeiro","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Niepert et al., 2016] Mathias Niepert, Mohamed Ahmed, and Kon- stantin Kutzkov. Learning convolutional neural networks for graphs. In International conference on machine learning, 2016.","title":"Learning convolutional neural networks for graphs","authors":"Mathias Niepert, Mohamed Ahmed, Konstantin Kutzkov","year":"2016","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In SIGKDD, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Pinkus, 1999] Allan Pinkus. Approximation theory of the mlp model in neural networks. Acta numerica, 8:143-195, 1999.","title":"Approximation theory of the mlp model in neural networks","authors":"Allan Pinkus","year":"1999","venue":"Acta numerica","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Rudin, 1987] Walter Rudin. Real and Complex Analysis, 3rd Ed. McGraw-Hill, Inc., New York, NY, USA, 1987.","title":"Real and Complex Analysis","authors":"Walter Rudin","year":"1987","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Scarselli et al., 2009] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 2009.","title":"The graph neural network model","authors":"Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, Gabriele Monfardini","year":"2009","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Shervashidze et al., 2011] Nino Shervashidze, Pascal Schweitzer, Erik Jan van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 2011.","title":"Weisfeiler-lehman graph kernels","authors":"Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, Karsten M Borgwardt","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Sperduti and Starita, 1997] Alessandro Sperduti and Antonina Starita. Supervised neural networks for the classification of struc- tures. IEEE Transactions on Neural Networks, 1997.","title":"Supervised neural networks for the classification of structures","authors":"Alessandro Sperduti, Antonina Starita","year":"1997","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Szegedy et al., 2016] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. In CVPR, 2016.","title":"Rethinking the inception architecture for computer vision","authors":"C Szegedy, V Vanhoucke, S Ioffe, J Shlens, Z Wojna","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Velickovic et al., 2018] Petar Velickovic, Guillem Cucurull, Aran- txa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. ICLR, 2018.","title":"Graph attention networks","authors":"Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Verma and Zhang, 2019] Saurabh Verma and Zhi-Li Zhang. Graph capsule convolutional neural networks. ICLR, 2019.","title":"Graph capsule convolutional neural networks","authors":"Saurabh Verma, Zhi-Li Zhang","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Weisfeiler and Lehman, 1968] Boris Weisfeiler and AA Lehman. A reduction of a graph to a canonical form and an algebra aris- ing during this reduction. Nauchno-Technicheskaya Informatsia, 1968.","title":"A reduction of a graph to a canonical form and an algebra arising during this reduction","authors":"Lehman Weisfeiler, Boris Weisfeiler, Lehman","year":"1968","venue":"Nauchno-Technicheskaya Informatsia","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Xu et al., 2019] Keyulu Xu, Weihua Hu, Jure Leskovec, and Ste- fanie Jegelka. How powerful are graph neural networks? ICLR, 2019.","title":"How powerful are graph neural networks? ICLR","authors":"Keyulu Xu, Weihua Hu, Jure Leskovec, Stefanie Jegelka","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Ying et al., 2018] R. Ying, J. You, C. Morris, X. Ren, W. L Hamil- ton, and J. Leskovec. Hierarchical graph representation learning with differentiable pooling. In NIPS, pages 4805-4815, 2018.","title":"Hierarchical graph representation learning with differentiable pooling","authors":"R Ying, J You, C Morris, X Ren, W L Hamilton, J Leskovec","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zaheer et al., 2017] M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Póc- zos, R. Salakhutdinov, and A. J. Smola. Deep sets. In NIPS, 2017.","title":"Deep sets","authors":"M Zaheer, S Kottur, S Ravanbakhsh, B Póczos, R Salakhutdinov, A J Smola","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhang et al., 2018] M. Zhang, Z. Cui, M. Neumann, and Y. Chen. An end-to-end deep learning architecture for graph classification. In AAAI, 2018.","title":"An end-to-end deep learning architecture for graph classification","authors":"M Zhang, Z Cui, M Neumann, Y Chen","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}