{"type":"ScholarlyArticle","title":"Multi-Vector Embedding on Networks with Taxonomies","authors":[{"name":"Yue Fan","slug":"yue-fan","given":"Yue","family":"Fan","affiliation":"Key Laboratory of Machine Perception (MOE) , School of Artificial Intelligence , Peking University , Beijing , China"},{"name":"Xiuli Ma","slug":"xiuli-ma","given":"Xiuli","family":"Ma","affiliation":"Key Laboratory of Machine Perception (MOE) , School of Artificial Intelligence , Peking University , Beijing , China"}],"abstract":"A network can effectively depict close relationships among its nodes, with labels in a taxonomy describing the nodes' rich attributes. Network embedding aims at learning a representation vector for each node and label to preserve their proximity, while most existing methods suffer from serious underfitting when dealing with datasets with dense node-label links. For instance, a node could have dozens of labels describing its diverse properties, causing the single node vector overloaded and hard to fit all the labels. We propose HIerarchical Multi-vector Embedding (HIME), which solves the underfitting problem by adaptively learning multiple 'branch vectors' for each node to dynamically fit separate sets of labels in a hierarchy-aware embedding space. Moreover, a 'root vector' is learned for each node based on its branch vectors to better predict the sparse but valuable node-node links with the knowledge of its labels. Experiments reveal HIME’s comprehensive advantages over existing methods on tasks such as proximity search, link prediction and hierarchical classification.","doi":"10.24963/ijcai.2022/408","url":"https://ijcai.org/proceedings/2022/408","pdf":"https://ijcai.org/proceedings/2022/0408.pdf","markdown":"https://ijcai.org/proceedings/2022/408.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/408","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":408,"pages":{"first":2944,"last":2950},"keywords":["Data Mining: Mining Heterogenous Data","Machine Learning: Multi-label","Multidisciplinary Topics and Applications: Bioinformatics","Machine Learning: Representation learning","Data Mining: Networks"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987044","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985187","length_sec":684,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Ashburner et al., 2000] Michael Ashburner, Catherine A Ball, Judith A Blake, David Botstein, Heather Butler, J Michael Cherry, Allan P Davis, Kara Dolinski, Selina S Dwight, Janan T Eppig, et al. Gene ontology: tool for the unification of biology. Nature genetics, 25(1):25-29, 2000.","title":"Gene ontology: tool for the unification of biology","authors":"Michael Ashburner, Catherine A Ball, Judith A Blake, David Botstein, Heather Butler, J Michael Cherry, Allan P Davis, Kara Dolinski, Selina S Dwight, Janan T Eppig","year":"2000","venue":"Nature genetics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Davis et al., 2020] Allan Peter Davis, Cynthia J Grondin, Robin J Johnson, Daniela Sciaky, Jolene Wiegers, Thomas C Wiegers, and Carolyn J Mattingly. Compara- tive Toxicogenomics Database (CTD): update 2021. Nu- cleic Acids Research, 49(D1):D1138-D1143, 10 2020.","title":"Comparative Toxicogenomics Database (CTD): update 2021","authors":"Allan Peter, Davis, Cynthia J Grondin, Robin J Johnson, Daniela Sciaky, Jolene Wiegers, Thomas C Wiegers, Carolyn J Mattingly","year":"2020","venue":"Nucleic Acids Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Epasto and Perozzi, 2019] Alessandro Epasto and Bryan Perozzi. Is a single embedding enough? learning node rep- resentations that capture multiple social contexts. In The World Wide Web Conference, page 394-404. Association for Computing Machinery, 2019.","title":"Is a single embedding enough? learning node representations that capture multiple social contexts","authors":"Alessandro Epasto, Bryan Perozzi","year":"2019","venue":"The World Wide Web Conference","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pages 855-864, 2016.","title":"node2vec: Scalable feature learning for networks","authors":"Leskovec Grover, Aditya Grover, Jure Leskovec","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, volume 30, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hao et al., 2019] Junheng Hao, Muhao Chen, Wenchao Yu, Yizhou Sun, and Wei Wang. Universal representation learning of knowledge bases by jointly embedding in- stances and ontological concepts. In Proceedings of the 25th ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, page 1709-1719, 2019.","title":"Universal representation learning of knowledge bases by jointly embedding instances and ontological concepts","authors":"Hao","year":"2019","venue":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huntley et al., 2015] Rachael P Huntley, Tony Sawford, Prudence Mutowo-Meullenet, Aleksandra Shypitsyna, Carlos Bonilla, Maria J Martin, and Claire O'Donovan. The goa database: gene ontology annotation updates for 2015. Nucleic acids research, 43(D1):D1057-D1063, 2015.","title":"The goa database: gene ontology annotation updates for 2015","authors":"P Rachael, Tony Huntley, Prudence Sawford, Aleksandra Mutowo-Meullenet, Carlos Shypitsyna, Maria J Bonilla, Claire O' Martin, Donovan","year":"2015","venue":"Nucleic acids research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2015] Zhi-Ping Liu, Canglin Wu, Hongyu Miao, and Hulin Wu. RegNetwork: an integrated database of transcriptional and post-transcriptional regulatory net- works in human and mouse. Database, 2015, 09 2015. bav095.","title":"RegNetwork: an integrated database of transcriptional and post-transcriptional regulatory networks in human and mouse","authors":"Zhi-Ping Liu, Canglin Wu, Hongyu Miao, Hulin Wu","year":"2015","venue":"Database","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu et al., 2019] Ninghao Liu, Qiaoyu Tan, Yuening Li, Hongxia Yang, Jingren Zhou, and Xia Hu. Is a single vector enough? exploring node polysemy for network em- bedding. In Proceedings of the 25th ACM SIGKDD Inter- national Conference on Knowledge Discovery and Data Mining, page 932-940, 2019.","title":"Is a single vector enough? exploring node polysemy for network embedding","authors":"Liu","year":"2019","venue":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Nickel and Kiela, 2017] Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical rep- resentations. In Advances in Neural Information Process- ing Systems, volume 30, 2017.","title":"Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations","authors":"Kiela Nickel","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Nickel and Kiela, 2018] Maximilian Nickel and Douwe Kiela. Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In Proceedings of the 35th International Conference on Machine Learning, vol- ume 80, pages 3776-3785, 2018.","title":"Maximilian Nickel and Douwe Kiela. Learning continuous hierarchies in the lorentz model of hyperbolic geometry","authors":"Kiela Nickel","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Park et al., 2020] Chanyoung Park, Carl Yang, Qi Zhu, Donghyun Kim, Hwanjo Yu, and Jiawei Han. Unsuper- vised differentiable multi-aspect network embedding. In Proceedings of the 26th ACM SIGKDD International Con- ference on Knowledge Discovery and Data Mining, pages 1435-1445, 2020.","title":"Unsupervised differentiable multi-aspect network embedding","authors":"Park","year":"2020","venue":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Petri et al., 2014] Victoria Petri, Pushkala Jayaraman, Marek Tutaj, G. Thomas Hayman, Jennifer R. Smith, Jeff De Pons, Stanley JF Laulederkind, Timothy F. Lowry, Ra- jni Nigam, Shur-Jen Wang, Mary Shimoyama, Melinda R. Dwinell, Diane H. Munzenmaier, Elizabeth A. Worthey, and Howard J. Jacob. The pathway ontology -updates and applications. Journal of Biomedical Semantics, 5(1):7, Feb 2014.","title":"The pathway ontology -updates and applications","authors":"Petri","year":"2014","venue":"Journal of Biomedical Semantics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ren et al., 2016] Xiang Ren, Wenqi He, Meng Qu, Lifu Huang, Heng Ji, and Jiawei Han. Afet: Automatic fine- grained entity typing by hierarchical partial-label embed- ding. In Proceedings of the 2016 conference on empiri- cal methods in natural language processing, pages 1369- 1378, 2016.","title":"Afet: Automatic finegrained entity typing by hierarchical partial-label embedding","authors":"Xiang Ren, Wenqi He, Meng Qu, Lifu Huang, Ji Heng, Jiawei Han","year":"2016","venue":"Proceedings of the 2016 conference on empirical methods in natural language processing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Szklarczyk et al., 2021] Damian Szklarczyk, Annika L Gable, Katerina C Nastou, David Lyon, Rebecca Kirsch, Sampo Pyysalo, Nadezhda T Doncheva, Marc Legeay, Tao Fang, Peer Bork, et al. The string database in 2021: cus- tomizable protein-protein networks, and functional char- acterization of user-uploaded gene/measurement sets. Nu- cleic acids research, 49(D1):D605-D612, 2021.","title":"The string database in 2021: customizable protein-protein networks, and functional characterization of user-uploaded gene/measurement sets","authors":"Damian Szklarczyk, Annika L Gable, Katerina C Nastou, David Lyon, Rebecca Kirsch, Sampo Pyysalo, T Nadezhda, Marc Doncheva, Tao Legeay, Peer Fang, Bork","year":"2021","venue":"Nucleic acids research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tang et al., 2015] Jian Tang, Meng Qu, and Qiaozhu Mei. Pte: Predictive text embedding through large-scale hetero- geneous text networks. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discov- ery and data mining, pages 1165-1174, 2015.","title":"Pte: Predictive text embedding through large-scale heterogeneous text networks","authors":"Jian Tang, Meng Qu, Qiaozhu Mei","year":"2015","venue":"Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2018] Hongwei Wang, Jia Wang, Jialin Wang, Miao Zhao, Weinan Zhang, Fuzheng Zhang, Xing Xie, and Minyi Guo. Graphgan: Graph representation learning with generative adversarial nets. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.","title":"Graphgan: Graph representation learning with generative adversarial nets","authors":"Hongwei Wang, Jia Wang, Jialin Wang, Miao Zhao, Weinan Zhang, Fuzheng Zhang, Xing Xie, Minyi Guo","year":"2018","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2019] Junshan Wang, Zhicong Lu, Guojia Song, Yue Fan, Lun Du, and Wei Lin. Tag2vec: Learning tag representations in tag networks. In The World Wide Web Conference, page 3314-3320, 2019.","title":"Tag2vec: Learning tag representations in tag networks","authors":"Junshan Wang, Zhicong Lu, Guojia Song, Yue Fan, Lun Du, Wei Lin","year":"2019","venue":"The World Wide Web Conference","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yang et al., 2020] Carl Yang, Jieyu Zhang, and Jiawei Han. Co-embedding network nodes and hierarchical labels with taxonomy based generative adversarial networks. In 2020 IEEE International Conference on Data Mining (ICDM), pages 721-730. IEEE, 2020.","title":"Co-embedding network nodes and hierarchical labels with taxonomy based generative adversarial networks","authors":"Carl Yang, Jieyu Zhang, Jiawei Han","year":"2020","venue":"IEEE International Conference on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}