{"type":"ScholarlyArticle","title":"A Single Vector Is Not Enough: Taxonomy Expansion via Box Embeddings (Extended Abstract)","authors":[{"name":"Song Jiang","slug":"song-jiang","given":"Song","family":"Jiang","affiliation":"University of California , Los Angeles ,"},{"name":"Qiyue Yao","slug":"qiyue-yao","given":"Qiyue","family":"Yao","affiliation":"University of California , Los Angeles ,"},{"name":"Qifan Wang","slug":"qifan-wang","given":"Qifan","family":"Wang","affiliation":"Meta AI"},{"name":"Yizhou Sun","slug":"yizhou-sun","given":"Yizhou","family":"Sun","affiliation":"University of California , Los Angeles ,"}],"abstract":"Taxonomies support various practical web applications such as product navigation in online shopping and user profile tagging on social platforms. Most existing methods for expanding taxonomies encode entities into vector embeddings (i.e., single points). However, we argue that vectors are insufficient to model the ``is-a'' hierarchy in taxonomy (asymmetrical relation), because two points can only represent pairwise similarity (symmetrical relation). To this end, we propose to project taxonomy entities into boxes (i.e., hyperrectangles). Two boxes can be \"contained\", \"disjoint\" and \"intersecting\", thus naturally representing an asymmetrical taxonomic hierarchy. Upon box embeddings, we propose a novel model BoxTaxo for taxonomy expansion. The core of BoxTaxo is to learn boxes for entities to capture their child-parent hierarchies. Extensive experiments on two benchmarks demonstrate the effectiveness of BoxTaxo compared to vector based models.","doi":"10.24963/ijcai.2024/934","url":"https://ijcai.org/proceedings/2024/934","pdf":"https://ijcai.org/proceedings/2024/0934.pdf","markdown":"https://ijcai.org/proceedings/2024/934.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/934","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Sister Conferences Best Papers","track":"Sister Conferences Best Papers","number":934,"pages":{"first":8421,"last":8426},"keywords":["Knowledge Representation and Reasoning: KRR: Semantic Web"],"videos":[],"references":[{"position":1,"raw":"[Aly et al., 2019] Rami Aly, Shantanu Acharya, Alexan- der Ossa, Arne Köhn, Chris Biemann, and Alexander Panchenko. Every child should have parents: A taxonomy refinement algorithm based on hyperbolic term embed- dings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4811- 4817, Florence, Italy, July 2019. Association for Compu- tational Linguistics.","title":"Every child should have parents: A taxonomy refinement algorithm based on hyperbolic term embeddings","authors":"Rami Aly, Shantanu Acharya, Alexander Ossa, Arne Köhn, Chris Biemann, Alexander Panchenko","year":"2019","venue":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chang et al., 2018] Haw-Shiuan Chang, Ziyun Wang, Luke Vilnis, and Andrew McCallum. Distributional inclusion vector embedding for unsupervised hypernymy detection. In Proceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Lin- guistics: Human Language Technologies, pages 485-495, New Orleans, Louisiana, June 2018. Association for Com- putational Linguistics.","title":"Distributional inclusion vector embedding for unsupervised hypernymy detection","authors":"Chang","year":"2018","venue":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Clark et al., 2020] Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. ELECTRA: pre-training text encoders as discriminators rather than generators. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.","title":"ELECTRA: pre-training text encoders as discriminators rather than generators","authors":"Kevin Clark, Minh-Thang Luong, Quoc V Le, Christopher D Manning","year":"2020","venue":"8th International Conference on Learning Representations, ICLR 2020","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dasgupta et al., 2020] Shib Dasgupta, Michael Boratko, Dongxu Zhang, Luke Vilnis, Xiang Li, and Andrew Mc- Callum. Improving local identifiability in probabilistic box embeddings. Advances in Neural Information Processing Systems, 33, 2020.","title":"Improving local identifiability in probabilistic box embeddings","authors":"Shib Dasgupta, Michael Boratko, Dongxu Zhang, Luke Vilnis, Xiang Li, Andrew Mc-Callum","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understand- ing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapo- lis, Minnesota, June 2019. Association for Computational Linguistics.","title":"BERT: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fu et al., 2014] Ruiji Fu, Jiang Guo, Bing Qin, Wanxiang Che, Haifeng Wang, and Ting Liu. Learning semantic hier- archies via word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Lin- guistics, pages 1199-1209, 2014.","title":"Learning semantic hierarchies via word embeddings","authors":"Ruiji Fu, Jiang Guo, Bing Qin, Wanxiang Che, Haifeng Wang, Ting Liu","year":"2014","venue":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ganea et al., 2018] Octavian Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic entailment cones for learn- ing hierarchical embeddings. In International Conference on Machine Learning, pages 1646-1655. PMLR, 2018.","title":"Hyperbolic entailment cones for learning hierarchical embeddings","authors":"Ganea","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gonçalves et al., 2019] Rafael S Gonçalves, Matthew Hor- ridge, Rui Li, Yu Liu, Mark A Musen, Csongor I Nyulas, Evelyn Obamos, Dhananjay Shrouty, and David Temple. Use of owl and semantic web technologies at pinterest. In International Semantic Web Conference, pages 418-435. Springer, 2019.","title":"Use of owl and semantic web technologies at pinterest","authors":"Matthew Rafael S Gonçalves, Rui Horridge, Yu Li, Liu, Mark A Musen, I Csongor, Evelyn Nyulas, Dhananjay Obamos, David Shrouty, Temple","year":"2019","venue":"International Semantic Web Conference","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"Marti A Hearst. Automatic acquisition of hy- ponyms from large text corpora. In COLING 1992 Volume 2: The 14th International Conference on Computational Linguistics, 1992.","title":"Automatic acquisition of hyponyms from large text corpora","authors":"A Marti, Hearst","year":"1992","venue":"The 14th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jiang et al., 2022] Minhao Jiang, Xiangchen Song, Jieyu Zhang, and Jiawei Han. Taxoenrich: Self-supervised tax- onomy completion via structure-semantic representations. In Proceedings of the ACM Web Conference 2022, pages 925-934, 2022.","title":"Taxoenrich: Self-supervised taxonomy completion via structure-semantic representations","authors":"Jiang","year":"2022","venue":"Proceedings of the ACM Web Conference 2022","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lees et al., 2020] Alyssa Lees, Chris Welty, Shubin Zhao, Jacek Korycki, and Sara Mc Carthy. Embedding semantic taxonomies. In Proceedings of the 28th International Con- ference on Computational Linguistics, pages 1279-1291, 2020.","title":"Embedding semantic taxonomies","authors":"Alyssa Lees, Chris Welty, Shubin Zhao, Jacek Korycki, Sara Mc Carthy","year":"2020","venue":"Proceedings of the 28th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2018] Xiang Li, Luke Vilnis, Dongxu Zhang, Michael Boratko, and Andrew McCallum. Smoothing the geometry of probabilistic box embeddings. In Interna- tional Conference on Learning Representations, 2018.","title":"Smoothing the geometry of probabilistic box embeddings","authors":"Xiang Li, Luke Vilnis, Dongxu Zhang, Michael Boratko, Andrew Mccallum","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv:1907.11692, 2019.","title":"Roberta: A robustly optimized bert pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2021] Zichen Liu, Hongyuan Xu, Yanlong Wen, Ning Jiang, Haiying Wu, and Xiaojie Yuan. Temp: Taxonomy expansion with dynamic margin loss through taxonomy-paths. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3854-3863, 2021.","title":"Temp: Taxonomy expansion with dynamic margin loss through taxonomy-paths","authors":"Liu","year":"2021","venue":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ma et al., 2021] Mingyu Derek Ma, Muhao Chen, Te-Lin Wu, and Nanyun Peng. HyperExpan: Taxonomy expan- sion with hyperbolic representation learning. In Findings of the Association for Computational Linguistics: EMNLP 2021, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.","title":"HyperExpan: Taxonomy expansion with hyperbolic representation learning","authors":"Derek Mingyu, Muhao Ma, Te-Lin Chen, Nanyun Wu, Peng","year":"2021","venue":"Findings of the Association for Computational Linguistics: EMNLP 2021","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Manzoor et al., 2020] Emaad Manzoor, Rui Li, Dhananjay Shrouty, and Jure Leskovec. Expanding taxonomies with implicit edge semantics. In Proceedings of The Web Con- ference 2020, pages 2044-2054, 2020.","title":"Expanding taxonomies with implicit edge semantics","authors":"Manzoor","year":"2020","venue":"Proceedings of The Web Conference 2020","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mao et al., 2020] Yuning Mao, Tong Zhao, Andrey Kan, Chenwei Zhang, Xin Luna Dong, Christos Faloutsos, and Jiawei Han. Octet: Online catalog taxonomy enrichment with self-supervision. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discov- ery & Data Mining, pages 2247-2257, 2020.","title":"Octet: Online catalog taxonomy enrichment with self-supervision","authors":"Yuning Mao, Tong Zhao, Andrey Kan, Chenwei Zhang, Xin, Luna Dong, Christos Faloutsos, Jiawei Han","year":"2020","venue":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed rep- resentations of words and phrases and their composition- ality. Advances in neural information processing systems, 26, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nickel and Kiela, 2017] Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical rep- resentations. Advances in neural information processing systems, 30, 2017.","title":"Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations","authors":"Kiela Nickel","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Onoe et al., 2021] Yasumasa Onoe, Michael Boratko, An- drew McCallum, and Greg Durrett. Modeling fine-grained entity types with box embeddings. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online, August 2021. Association for Computational Lin- guistics.","title":"Andrew McCallum, and Greg Durrett. Modeling fine-grained entity types with box embeddings","authors":"Onoe","year":"2021","venue":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Qiu et al., 2020] Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. Pre-trained models for natural language processing: A survey. Science China Technological Sciences, 63(10):1872-1897, 2020.","title":"Pre-trained models for natural language processing: A survey","authors":"Qiu","year":"2020","venue":"Science China Technological Sciences","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shen et al., 2020] Jiaming Shen, Zhihong Shen, Chenyan Xiong, Chi Wang, Kuansan Wang, and Jiawei Han. Taxoexpan: Self-supervised taxonomy expansion with position-enhanced graph neural network. In Proceedings of The Web Conference 2020, 2020.","title":"Taxoexpan: Self-supervised taxonomy expansion with position-enhanced graph neural network","authors":"Jiaming Shen, Zhihong Shen, Chenyan Xiong, Chi Wang, Kuansan Wang, Jiawei Han","year":"2020","venue":"Proceedings of The Web Conference 2020","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Snow et al., 2004] Rion Snow, Daniel Jurafsky, and Andrew Ng. Learning syntactic patterns for automatic hypernym discovery. Advances in neural information processing sys- tems, 17, 2004.","title":"Learning syntactic patterns for automatic hypernym discovery","authors":"Rion Snow, Daniel Jurafsky, Andrew Ng","year":"2004","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Venn, 1880] John Venn. I. on the diagrammatic and me- chanical representation of propositions and reasonings. The London, Edinburgh, and Dublin philosophical mag- azine and journal of science, 10(59):1-18, 1880.","title":"on the diagrammatic and mechanical representation of propositions and reasonings","authors":"John Venn, . I","year":"1880","venue":"The London, Edinburgh, and Dublin philosophical magazine and journal of science","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vilnis et al., 2018] Luke Vilnis, Xiang Li, Shikhar Murty, and Andrew McCallum. Probabilistic embedding of knowledge graphs with box lattice measures. In Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Mel- bourne, Australia, July 2018. Association for Computa- tional Linguistics.","title":"Probabilistic embedding of knowledge graphs with box lattice measures","authors":"Vilnis","year":"2018","venue":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2021] Suyuchen Wang, Ruihui Zhao, Xi Chen, Yefeng Zheng, and Bang Liu. Enquire one's parent and child before decision: Fully exploit hierarchical structure for self-supervised taxonomy expansion. In Proceedings of the Web Conference 2021, pages 3291-3304, 2021.","title":"Enquire one's parent and child before decision: Fully exploit hierarchical structure for self-supervised taxonomy expansion","authors":"Wang","year":"2021","venue":"Proceedings of the Web Conference 2021","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2020] Yue Yu, Yinghao Li, Jiaming Shen, Hao Feng, Jimeng Sun, and Chao Zhang. Steam: Self- supervised taxonomy expansion with mini-paths. In Pro- ceedings of the 26th ACM SIGKDD International Con- ference on Knowledge Discovery & Data Mining, pages 1026-1035, 2020.","title":"Steam: Selfsupervised taxonomy expansion with mini-paths","authors":"Yue Yu, Yinghao Li, Jiaming Shen, Hao Feng, Jimeng Sun, Chao Zhang","year":"2020","venue":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}