{"type":"ScholarlyArticle","title":"SGAT: Simplicial Graph Attention Network","authors":[{"name":"See Hian Lee","slug":"see-hian-lee","given":"See Hian","family":"Lee","affiliation":"Nanyang Technological University , Singapore"},{"name":"Feng Ji","slug":"feng-ji","given":"Feng","family":"Ji","affiliation":""},{"name":"Wee Peng Tay","slug":"wee-peng-tay","given":"Wee Peng","family":"Tay","affiliation":"Nanyang Technological University , Singapore"}],"abstract":"Heterogeneous graphs have multiple node and edge types and are semantically richer than homogeneous graphs. To learn such complex semantics, many graph neural network approaches for heterogeneous graphs use metapaths to capture multi-hop interactions between nodes. Typically, features from non-target nodes are not incorporated into the learning procedure. However, there can be nonlinear, high-order interactions involving multiple nodes or edges. In this paper, we present Simplicial Graph Attention Network (SGAT), a simplicial complex approach to represent such high-order interactions by placing features from non-target nodes on the simplices. We then use attention mechanisms and upper adjacencies to generate representations. We empirically demonstrate the efficacy of our approach with node classification tasks on heterogeneous graph datasets and further show SGAT's ability in extracting structural information by employing random node features. Numerical experiments indicate that SGAT performs better than other current state-of-the-art heterogeneous graph learning methods.","doi":"10.24963/ijcai.2022/443","url":"https://ijcai.org/proceedings/2022/443","pdf":"https://ijcai.org/proceedings/2022/0443.pdf","markdown":"https://ijcai.org/proceedings/2022/443.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/443","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":443,"pages":{"first":3192,"last":3200},"keywords":["Data Mining: Mining Graphs","Data Mining: Mining Heterogenous Data","Machine Learning: Classification","Machine Learning: Representation learning","Machine Learning: Sequence and Graph Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986742","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984885","length_sec":356,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Barbarossa and Sardellitti, 2020] Sergio Barbarossa and Stefania Sardellitti. Topological signal processing over simplicial complexes. IEEE Trans. on Signal Processing, PP:1-1, 03 2020.","title":"Sergio Barbarossa and Stefania Sardellitti. Topological signal processing over simplicial complexes","authors":"Sardellitti Barbarossa","year":"2020","venue":"IEEE Trans. on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bodnar et al., 2021] Cristian Bodnar, Fabrizio Frasca, Yuguang Wang, Nina Otter, Guido F Montufar, Pietro Lió, and Michael Bronstein. Weisfeiler and lehman go topological: Message passing simplicial networks. In Proc. of the 38th International Conference on Machine Learning (ICML), pages 1026-1037, 2021.","title":"Weisfeiler and lehman go topological: Message passing simplicial networks","authors":"Cristian Bodnar, Fabrizio Frasca, Yuguang Wang, Nina Otter, Pietro Guido F Montufar, Michael Lió, Bronstein","year":"2021","venue":"Proc. of the 38th International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Borgwardt et al., 2020] Karsten Borgwardt, Elisabetta Ghisu, Felipe Llinares-López, Leslie O'Bray, and Bastian Rieck. Graph Kernels: State-of-the-Art and Future Challenges, volume 13. Foundations and Trends® in Machine Learning, 2020.","title":"Graph Kernels: State-of-the-Art and Future Challenges","authors":"Karsten Borgwardt, Elisabetta Ghisu, Felipe Llinares-López, O' Leslie, Bastian Bray, Rieck","year":"2020","venue":"Foundations and Trends® in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bunch et al., 2020] Eric Bunch, Qian You, Glenn Fung, and Vikas Singh. Simplicial 2-complex convolutional neural networks. NeurIPS 2020 Workshop on Topological Data Analysis and Beyond, 2020.","title":"Simplicial 2-complex convolutional neural networks","authors":"Eric Bunch, Qian You, Glenn Fung, Vikas Singh","year":"2020","venue":"NeurIPS 2020 Workshop on Topological Data Analysis and Beyond","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong et al., 2017] Yuxiao Dong, Nitesh V. Chawla, and Ananthram Swami. Metapath2vec: Scalable representa- tion learning for heterogeneous networks. In Proc. of the 23rd ACM SIGKDD International Conference on Knowl. Discovery and Data Mining, page 135-144, 2017.","title":"Metapath2vec: Scalable representation learning for heterogeneous networks","authors":"Dong","year":"2017","venue":"Proc. of the 23rd ACM SIGKDD International Conference on Knowl. Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Douglas, 2011] Brendan L. Douglas. The weisfeiler-lehman method and graph isomorphism testing. arXiv preprint, 2011. arXiv:1101.5211.","title":"The weisfeiler-lehman method and graph isomorphism testing","authors":"Brendan L Douglas","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ebli et al., 2020] Stefania Ebli, Michaël Defferrard, and Gard Spreemann. Simplicial neural networks. NeurIPS Workshop in Topological Data Analysis and Beyond, 2020.","title":"Simplicial neural networks","authors":"Stefania Ebli, Michaël Defferrard, Gard Spreemann","year":"2020","venue":"NeurIPS Workshop in Topological Data Analysis and Beyond","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fang et al., 2019] Yang Fang, Xiang Zhao, Peixin Huang, Weidong Xiao, and Maarten de Rijke. M-hin: Complex embeddings for heterogeneous information networks via metagraphs. In Proc. of the 42nd International ACM SI- GIR Conference on Research and Development in Infor- mation Retrieval, SIGIR'19, page 913-916, 2019.","title":"M-hin: Complex embeddings for heterogeneous information networks via metagraphs","authors":"Yang Fang, Xiang Zhao, Peixin Huang, Weidong Xiao, Maarten De Rijke","year":"2019","venue":"Proc. of the 42nd International ACM SI-GIR Conference on Research and Development in Information Retrieval, SIGIR'19","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fu et al., 2020] Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. MAGNN: Metapath aggregated graph neural network for heterogeneous graph embedding. In Proc. of The Web Conference 2020, page 2331-2341, 2020.","title":"MAGNN: Metapath aggregated graph neural network for heterogeneous graph embedding","authors":"Xinyu Fu, Jiani Zhang, Ziqiao Meng, Irwin King","year":"2020","venue":"Proc. of The Web Conference 2020","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Giusti et al., 2016] Chad Giusti, Robert Ghrist, and Danielle S. Bassett. Two's company, three (or more) is a simplex. Journal of Computational Neuroscience, 41(1):1-14, 2016.","title":"Two's company, three (or more) is a simplex","authors":"Chad Giusti, Robert Ghrist, Danielle S Bassett","year":"2016","venue":"Journal of Computational Neuroscience","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[GUDHI Project, 2015] GUDHI Project. GUDHI User and Reference Manual. GUDHI Editorial Board, 2015.","title":"GUDHI Project. GUDHI User and Reference Manual","authors":null,"year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hamilton et al., 2017] William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Proc. of the 31st International Conference on Neural Information Processing Systems, page 1025-1035, 2017.","title":"Inductive representation learning on large graphs","authors":"William L Hamilton, Rex Ying, Jure Leskovec","year":"2017","venue":"Proc. of the 31st International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hatcher, 2000] Allen Hatcher. Algebraic topology. Cam- bridge Univ. Press, Cambridge, 2000.","title":"Algebraic topology","authors":"Allen Hatcher","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Horn et al., 2022] Max Horn, Edward De Brouwer, Michael Moor, Yves Moreau, Bastian Rieck, and Karsten Borg- wardt. Topological graph neural networks. International Conference on Learning Representations, 2022.","title":"Topological graph neural networks","authors":"Horn","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hussein et al., 2018] Rana Hussein, Dingqi Yang, and Philippe Cudré-Mauroux. Are meta-paths necessary? re- visiting heterogeneous graph embeddings. In Proc. of the 27th ACM International Conference on Information and Knowledge Management, page 437-446, 2018.","title":"Are meta-paths necessary? revisiting heterogeneous graph embeddings","authors":"Rana Hussein, Dingqi Yang, Philippe Cudré-Mauroux","year":"2018","venue":"Proc. of the 27th ACM International Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ji et al., 2022] Feng Ji, Giacomo Kahn, and Wee Peng Tay. Signal processing on simplicial complexes with vertex sig- nals. IEEE Access, pages 1-1, 2022.","title":"Signal processing on simplicial complexes with vertex signals","authors":"Feng Ji, Giacomo Kahn, Wee Peng Tay","year":"2022","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. International Conference on Learning Repre- sentations, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lee et al., 2021] See Hian Lee, Feng Ji, and Wee Peng Tay. Learning on heterogeneous graphs using high-order rela- tions. ICASSP 2021 -2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3175-3179, 2021.","title":"Learning on heterogeneous graphs using high-order relations","authors":"Lee","year":"2021","venue":"ICASSP 2021 -2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Roman et al., 2015] Theodore Roman, Amir Nayyeri, Brit- tany Fasy, and Russell Schwartz. A simplicial complex- based approach to unmixing tumor progression data. BMC bioinformatics, 16:254, 12 2015.","title":"Theodore Roman, Amir Nayyeri, Brittany Fasy, and Russell Schwartz. A simplicial complexbased approach to unmixing tumor progression data","authors":"Roman","year":"2015","venue":"BMC bioinformatics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sankar et al., 2019] Aravind Sankar, Xinyang Zhang, and Kevin Chen-Chuan Chang. Meta-gnn: Metagraph neu- ral network for semi-supervised learning in attributed het- erogeneous information networks. In Proc. of the 2019 IEEE/ACM International Conference on Advances in So- cial Networks Analysis and Mining, 2019.","title":"Meta-gnn: Metagraph neural network for semi-supervised learning in attributed heterogeneous information networks","authors":"Aravind Sankar, Xinyang Zhang, Kevin Chen, -Chuan Chang","year":"2019","venue":"Proc. of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shi et al., 2019] Chuan Shi, Binbin Hu, Wayne Xin Zhao, and Philip S. Yu. Heterogeneous information network em- bedding for recommendation. IEEE Trans. on Knowledge and Data Engineering, 31(2):357-370, February 2019.","title":"Heterogeneous information network embedding for recommendation","authors":"Chuan Shi, Binbin Hu, Wayne Xin Zhao, Philip S Yu","year":"2019","venue":"IEEE Trans. on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Veličković et al., 2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. International Conference on Learning Representations, 2017.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2017","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2019] Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. Hetero- geneous graph attention network. In The World Wide Web Conference, page 2022-2032, 2019.","title":"Heterogeneous graph attention network","authors":"Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, Philip S Yu","year":"2019","venue":"The World Wide Web Conference","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2018] Carl Yang, Yichen Feng, Pan Li, Yu Shi, and Jiawei Han. Meta-graph based HIN spectral embed- ding: methods, analyses and insights. In ICDM, 2018.","title":"Meta-graph based HIN spectral embedding: methods, analyses and insights","authors":"Carl Yang, Yichen Feng, Pan Li, Yu Shi, Jiawei Han","year":"2018","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yun et al., 2019] Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J. Kim. Graph trans- former networks. In Proc. of the 33rd International Con- ference on Neural Information Processing Systems, 2019.","title":"Graph transformer networks","authors":"Yun","year":"2019","venue":"Proc. of the 33rd International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2020] Wentao Zhang, Yuan Fang, Zemin Liu, Min Wu, and Xinming Zhang. mg2vec: Learning relationship-preserving heterogeneous graph representa- tions via metagraph embedding. IEEE Trans. on Knowl- edge and Data Engineering, 2020.","title":"mg2vec: Learning relationship-preserving heterogeneous graph representations via metagraph embedding","authors":"Wentao Zhang, Yuan Fang, Zemin Liu, Min Wu, Xinming Zhang","year":"2020","venue":"IEEE Trans. on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}