{"type":"ScholarlyArticle","title":"MEIM: Multi-partition Embedding Interaction Beyond Block Term Format for Efficient and Expressive Link Prediction","authors":[{"name":"Hung-Nghiep Tran","slug":"hung-nghiep-tran","given":"Hung-Nghiep","family":"Tran","affiliation":"National Institute of Informatics , Tokyo , Japan"},{"name":"Atsuhiro Takasu","slug":"atsuhiro-takasu","given":"Atsuhiro","family":"Takasu","affiliation":"National Institute of Informatics , Tokyo , Japan; The Graduate University for Advanced Studies , SOKENDAI , Tokyo , Japan"}],"abstract":"Knowledge graph embedding aims to predict the missing relations between entities in knowledge graphs. Tensor-decomposition-based models, such as ComplEx, provide a good trade-off between efficiency and expressiveness, that is crucial because of the large size of real world knowledge graphs. The recent multi-partition embedding interaction (MEI) model subsumes these models by using the block term tensor format and provides a systematic solution for the trade-off. However, MEI has several drawbacks, some of which carried from its subsumed tensor-decomposition-based models. In this paper, we address these drawbacks and introduce the Multi-partition Embedding Interaction iMproved beyond block term format (MEIM) model, with independent core tensor for ensemble effects and soft orthogonality for max-rank mapping, in addition to multi-partition embedding. MEIM improves expressiveness while still being highly efficient, helping it to outperform strong baselines and achieve state-of-the-art results on difficult link prediction benchmarks using fairly small embedding sizes. The source code is released at https://github.com/tranhungnghiep/MEIM.","doi":"10.24963/ijcai.2022/314","url":"https://ijcai.org/proceedings/2022/314","pdf":"https://ijcai.org/proceedings/2022/0314.pdf","markdown":"https://ijcai.org/proceedings/2022/314.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/314","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":314,"pages":{"first":2262,"last":2269},"keywords":["Machine Learning: Relational Learning","Data Mining: Knowledge Graphs and Knowledge Base Completion","Machine Learning: Representation learning","Natural Language Processing: Embeddings"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986888","length_sec":85,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985031","length_sec":323,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Balažević et al., 2019] Ivana Balažević, Carl Allen, and Timothy M. Hospedales. TuckER: Tensor Factorization for Knowledge Graph Completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Lan- guage Processing, pages 5185-5194, 2019.","title":"TuckER: Tensor Factorization for Knowledge Graph Completion","authors":"Ivana Balažević, Carl Allen, Timothy M Hospedales","year":"2019","venue":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating Embeddings for Modeling Multi-Relational Data. In Advances in Neural Information Processing Systems, pages 2787-2795, 2013.","title":"Translating Embeddings for Modeling Multi-Relational Data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chami et al., 2020] Ines Chami, Adva Wolf, Da-Cheng Juan, Frederic Sala, Sujith Ravi, and Christopher Ré. Low- Dimensional Hyperbolic Knowledge Graph Embeddings. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 6901-6914, 2020.","title":"Low-Dimensional Hyperbolic Knowledge Graph Embeddings","authors":"Ines Chami, Adva Wolf, Da-Cheng Juan, Frederic Sala, Sujith Ravi, Christopher Ré","year":"2020","venue":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[De Lathauwer, 2008] Lieven De Lathauwer. Decomposi- tions of a Higher-Order Tensor in Block Terms-Part II: Definitions and Uniqueness. SIAM Journal on Matrix Analysis and Applications, 30(3):1033-1066, 2008.","title":"Decompositions of a Higher-Order Tensor in Block Terms-Part II: Definitions and Uniqueness","authors":"Lieven De, Lathauwer","year":"2008","venue":"SIAM Journal on Matrix Analysis and Applications","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dettmers et al., 2018] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2D Knowledge Graph Embeddings. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, pages 1811-1818, 2018.","title":"Convolutional 2D Knowledge Graph Embeddings","authors":"Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel","year":"2018","venue":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ebisu and Ichise, 2019] Takuma Ebisu and Ryutaro Ichise. Generalized Translation-based Embedding of Knowledge Graph. IEEE Transactions on Knowledge and Data Engi- neering, 32(5):941-951, 2019.","title":"Generalized Translation-based Embedding of Knowledge Graph","authors":"Ichise ; ] Takuma Ebisu, Ryutaro Ebisu, Ichise","year":"2019","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ha et al., 2016] David Ha, Andrew M. Dai, and Quoc V. Le. HyperNetworks. In International Conference on Learning Representations, page 18, 2016.","title":"International Conference on Learning Representations","authors":"David Ha, Andrew M Dai, V Quoc, Le, Hypernetworks","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In International Conference on Machine Learning, pages 448-456, 2015.","title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift","authors":"Sergey Ioffe, Christian Szegedy","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kazemi and Poole, 2018] Seyed Mehran Kazemi and David Poole. SimplE Embedding for Link Prediction in Knowl- edge Graphs. In Advances in Neural Information Process- ing Systems, pages 4289-4300, 2018.","title":"SimplE Embedding for Link Prediction in Knowledge Graphs","authors":"Seyed Mehran, Kazemi, David Poole","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kingma and Ba, 2014] Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In Interna- tional Conference on Learning Representations, page 15, 2014.","title":"Adam: A Method for Stochastic Optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kolda and Bader, 2009] Tamara G. Kolda and Brett W. Bader. Tensor Decompositions and Applications. SIAM Review, 51(3):455-500, 2009.","title":"Tensor Decompositions and Applications","authors":"Tamara G Kolda, Brett W Bader","year":"2009","venue":"SIAM Review","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2021] Xiyang Liu, Huobin Tan, Qinghong Chen, and Guangyan Lin. RAGAT: Relation Aware Graph At- tention Network for Knowledge Graph Completion. IEEE Access, 9:20840-20849, 2021.","title":"RAGAT: Relation Aware Graph Attention Network for Knowledge Graph Completion","authors":"Xiyang Liu, Huobin Tan, Qinghong Chen, Guangyan Lin","year":"2021","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mahdisoltani et al., 2015] Farzaneh Mahdisoltani, Joanna Biega, and Fabian M. Suchanek. YAGO3: A Knowledge Base from Multilingual Wikipedias. In Proceedings of the Conference on Innovative Data Systems Research, 2015.","title":"YAGO3: A Knowledge Base from Multilingual Wikipedias","authors":"Farzaneh Mahdisoltani, Joanna Biega, Fabian M Suchanek","year":"2015","venue":"Proceedings of the Conference on Innovative Data Systems Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mason et al., 1999] Llew Mason, Jonathan Baxter, Peter Bartlett, and Marcus Frean. Boosting Algorithms as Gra- dient Descent. In Advances in Neural Information Pro- cessing Systems, pages 512-518, 1999.","title":"Boosting Algorithms as Gradient Descent","authors":"Llew Mason, Jonathan Baxter, Peter Bartlett, Marcus Frean","year":"1999","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nickel et al., 2011] Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. A Three-Way Model for Collective Learning on Multi-Relational Data. In International Con- ference on Machine Learning, pages 809-816, 2011.","title":"A Three-Way Model for Collective Learning on Multi-Relational Data","authors":"Maximilian Nickel, Hans-Peter Volker Tresp, Kriegel","year":"2011","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rossi et al., 2021] Andrea Rossi, Denilson Barbosa, Do- natella Firmani, Antonio Matinata, and Paolo Merialdo. Knowledge graph embedding for link prediction: A com- parative analysis. ACM Transactions on Knowledge Dis- covery from Data (TKDD), 15(2):1-49, 2021.","title":"Knowledge graph embedding for link prediction: A comparative analysis","authors":"Andrea Rossi, Denilson Barbosa, Donatella Firmani, Antonio Matinata, Paolo Merialdo","year":"2021","venue":"ACM Transactions on Knowledge Discovery from Data (TKDD)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ruffinelli et al., 2020] Daniel Ruffinelli, Samuel Broscheit, and Rainer Gemulla. You CAN Teach an Old Dog New Tricks! On Training Knowledge Graph Embeddings. In International Conference on Learning Representations, page 20, 2020. [Sokół and Park, 2020]","title":"You CAN Teach an Old Dog New Tricks! On Training Knowledge Graph Embeddings","authors":"Daniel Ruffinelli, Samuel Broscheit, Rainer Gemulla","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"Piotr Aleksander Sokół and Il Mem- ming Park. Information Geometry of Orthogonal Initial- izations and Training. In International Conference on Learning Representations, page 17, 2020.","title":"Sokół and Il Memming Park. Information Geometry of Orthogonal Initializations and Training","authors":"Piotr Aleksander","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. The Journal of Machine Learning Re- search, 15(1):1929-1958, 2014.","title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sun et al., 2019] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. In International Conference on Learning Representations, page 18, 2019.","title":"RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space","authors":"Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, Jian Tang","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sun et al., 2020] Zhiqing Sun, Shikhar Vashishth, Soumya Sanyal, Partha Talukdar, and Yiming Yang. A Re- evaluation of Knowledge Graph Completion Methods. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5516-5522, 2020.","title":"A Reevaluation of Knowledge Graph Completion Methods","authors":"Zhiqing Sun, Shikhar Vashishth, Soumya Sanyal, Partha Talukdar, Yiming Yang","year":"2020","venue":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tang et al., 2020] Yun Tang, Jing Huang, Guangtao Wang, Xiaodong He, and Bowen Zhou. Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 2713-2722, 2020.","title":"Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding","authors":"Yun Tang, Jing Huang, Guangtao Wang, Xiaodong He, Bowen Zhou","year":"2020","venue":"Proceedings of the Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Toutanova and Chen, 2015] Kristina Toutanova and Danqi Chen. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Composition- ality, pages 57-66, 2015.","title":"Observed versus latent features for knowledge base and text inference","authors":"Chen ; Toutanova, Kristina Toutanova, Danqi Chen","year":"2015","venue":"Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tran and Takasu, 2019a] Hung Nghiep Tran and Atsuhiro Takasu. Analyzing Knowledge Graph Embedding Meth- ods from a Multi-Embedding Interaction Perspective. In Proceedings of the Data Science for Industry 4.0 Work- shop at EDBT/ICDT, page 7, 2019.","title":"Analyzing Knowledge Graph Embedding Methods from a Multi-Embedding Interaction Perspective","authors":"Tran, Takasu, Hung Nghiep, Tran, Atsuhiro Takasu","year":"2019","venue":"Proceedings of the Data Science for Industry 4.0 Workshop at EDBT/ICDT","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tran and Takasu, 2019b] Hung Nghiep Tran and Atsuhiro Takasu. Exploring Scholarly Data by Semantic Query on Knowledge Graph Embedding Space. In Proceedings of the 23rd International Conference on Theory and Practice of Digital Libraries, pages 154-162, 2019.","title":"Exploring Scholarly Data by Semantic Query on Knowledge Graph Embedding Space","authors":"Tran, Takasu, Hung Nghiep, Tran, Atsuhiro Takasu","year":"2019","venue":"Proceedings of the 23rd International Conference on Theory and Practice of Digital Libraries","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tran and Takasu, 2020] Hung Nghiep Tran and Atsuhiro Takasu. Multi-Partition Embedding Interaction with Block Term Format for Knowledge Graph Completion. In Pro- ceedings of the European Conference on Artificial Intelli- gence, pages 833-840, 2020.","title":"Multi-Partition Embedding Interaction with Block Term Format for Knowledge Graph Completion","authors":"Tran, Takasu, Hung Nghiep, Tran, Atsuhiro Takasu","year":"2020","venue":"Proceedings of the European Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tran, 2020] Hung Nghiep Tran. Multi-Relational Embed- ding for Knowledge Graph Representation and Analysis. PhD thesis, The Graduate University for Advanced Stud- ies, SOKENDAI, Japan, 2020.","title":"Multi-Relational Embedding for Knowledge Graph Representation and Analysis","authors":"Hung Nghiep, Tran","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Trouillon et al., 2016] Theo Trouillon, Johannes Welbl, Se- bastian Riedel, Eric Gaussier, and Guillaume Bouchard. Complex Embeddings for Simple Link Prediction. In International Conference on Machine Learning, pages 2071-2080, 2016.","title":"Complex Embeddings for Simple Link Prediction","authors":"Trouillon","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Tucker, 1966] Ledyard R Tucker. Some Mathematical Notes on Three-Mode Factor Analysis. Psychometrika, 31(3):279-311, 1966.","title":"Some Mathematical Notes on Three-Mode Factor Analysis","authors":"R Ledyard, Tucker","year":"1966","venue":"Psychometrika","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Vashishth et al., 2020a] Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, Nilesh Agrawal, and Partha Talukdar. In- teractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions. In Pro- ceedings of the 34th AAAI Conference on Artificial Intelli- gence, pages 3009-3016, 2020.","title":"In-teractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions","authors":"Vashishth","year":"2020","venue":"Proceedings of the 34th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Vashishth et al., 2020b] Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. Composition-based Multi-Relational Graph Convolutional Networks. In International Conference on Learning Representations, page 16, 2020.","title":"Composition-based Multi-Relational Graph Convolutional Networks","authors":"Vashishth","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yang et al., 2015] Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In International Conference on Learning Representations, page 12, 2015.","title":"Embedding Entities and Relations for Learning and Inference in Knowledge Bases","authors":"Bishan Yang, Wen-Tau Yih, Xiaodong He, Jianfeng Gao, Li Deng","year":"2015","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2019] Shuai Zhang, Yi Tay, Lina Yao, and Qi Liu. Quaternion Knowledge Graph Embedding. In Ad- vances in Neural Information Processing Systems, pages 2735-2745, 2019.","title":"Quaternion Knowledge Graph Embedding","authors":"Shuai Zhang, Yi Tay, Lina Yao, Qi Liu","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}