{"type":"ScholarlyArticle","title":"Continual Multimodal Knowledge Graph Construction","authors":[{"name":"Xiang Chen","slug":"xiang-chen","given":"Xiang","family":"Chen","affiliation":"College of Computer Science and Technology , Zhejiang University; Zhejiang University -Ant Group Joint Research Center for Knowledge Graphs"},{"name":"Jingtian Zhang","slug":"jingtian-zhang","given":"Jingtian","family":"Zhang","affiliation":"School of Software Technology , Zhejiang University; Zhejiang University -Ant Group Joint Research Center for Knowledge Graphs"},{"name":"Xiaohan Wang","slug":"xiaohan-wang","given":"Xiaohan","family":"Wang","affiliation":"School of Software Technology , Zhejiang University; Zhejiang University -Ant Group Joint Research Center for Knowledge Graphs"},{"name":"Ningyu Zhang","slug":"ningyu-zhang","given":"Ningyu","family":"Zhang","affiliation":"School of Software Technology , Zhejiang University; Zhejiang University -Ant Group Joint Research Center for Knowledge Graphs"},{"name":"Tongtong Wu","slug":"tongtong-wu","given":"Tongtong","family":"Wu","affiliation":"Monash University ,"},{"name":"Yuxiang Wang","slug":"yuxiang-wang","given":"Yuxiang","family":"Wang","affiliation":"Hangzhou Dianzi University ,"},{"name":"Yongheng Wang","slug":"yongheng-wang","given":"Yongheng","family":"Wang","affiliation":"Zhejiang Lab"},{"name":"Huajun Chen","slug":"huajun-chen","given":"Huajun","family":"Chen","affiliation":"College of Computer Science and Technology , Zhejiang University; Zhejiang University -Ant Group Joint Research Center for Knowledge Graphs"}],"abstract":"Current Multimodal Knowledge Graph Construction (MKGC) models struggle with the real-world dynamism of continuously emerging entities and relations, often succumbing to catastrophic forgetting—loss of previously acquired knowledge. This study introduces benchmarks aimed at fostering the development of the continual MKGC domain. We further introduce the MSPT framework, designed to surmount the shortcomings of existing MKGC approaches during multimedia data processing. MSPT harmonizes the retention of learned knowledge (stability) and the integration of new data (plasticity), outperforming current continual learning and multimodal methods. Our results confirm MSPT's superior performance in evolving knowledge environments, showcasing its capacity to navigate the balance between stability and plasticity.","doi":"10.24963/ijcai.2024/688","url":"https://ijcai.org/proceedings/2024/688","pdf":"https://ijcai.org/proceedings/2024/0688.pdf","markdown":"https://ijcai.org/proceedings/2024/688.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/688","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Natural Language Processing","track":"Main Track","number":688,"pages":{"first":6225,"last":6233},"keywords":["Natural Language Processing: NLP: Information extraction","Natural Language Processing: NLP: Named entities","Data Mining: DM: Knowledge graphs and knowledge base completion"],"videos":[],"references":[{"position":1,"raw":"[Arshad et al., 2019] Omer Arshad, Ignazio Gallo, Shah Nawaz, and Alessandro Calefati. Aiding intra-text repre- sentations with visual context for multimodal named entity recognition. In ICDAR 2019, pages 337-342. IEEE, 2019.","title":"Aiding intra-text representations with visual context for multimodal named entity recognition","authors":"Omer Arshad, Ignazio Gallo, Shah Nawaz, Alessandro Calefati","year":"2019","venue":"ICDAR 2019","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chaudhry et al., 2019] Arslan Chaudhry, Marc'Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with A-GEM. In ICLR 2019. OpenReview.net, 2019.","title":"Efficient lifelong learning with A-GEM","authors":"Arslan Chaudhry, Marc'aurelio Ranzato, Marcus Rohrbach, Mohamed Elhoseiny","year":"2019","venue":"OpenReview.net","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and et al, 2022] Xiang Chen and et al. Good visual guidance make a better extractor: Hierarchical visual pre- fix for multimodal entity and relation extraction. In Find- ings of NAACL 2022, pages 1607-1618, July 2022.","title":"Good visual guidance make a better extractor: Hierarchical visual prefix for multimodal entity and relation extraction","authors":"Chen","year":"2022","venue":"Findings of NAACL 2022","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2022a] Xiang Chen, Lei Li, Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si, Huajun Chen, and Ningyu Zhang. Lightner: A lightweight tuning paradigm for low-resource NER via pluggable prompting. In COLING 2022, pages 2374-2387. International Com- mittee on Computational Linguistics, 2022.","title":"Lightner: A lightweight tuning paradigm for low-resource NER via pluggable prompting","authors":"Chen","year":"2022","venue":"International Committee on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2022b] Xiang Chen, Ningyu Zhang, and et al. Hybrid transformer with multi-level fusion for multimodal knowledge graph completion. In SIGIR, SIGIR '22, page 904-915, New York, NY, USA, 2022.","title":"Hybrid transformer with multi-level fusion for multimodal knowledge graph completion","authors":"Chen","year":"2022","venue":"SIGIR, SIGIR '22","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Clark et al., 2019] Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. What does BERT look at? an analysis of bert's attention. In Tal Linzen, Grze- gorz Chrupala, Yonatan Belinkov, and Dieuwke Hupkes, editors, Proceedings of the 2019 ACL Workshop Black- boxNLP: Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@ACL 2019, Florence, Italy, August 1, 2019, pages 276-286. Association for Computational Lin- guistics, 2019.","title":"What does BERT look at? an analysis of bert's attention","authors":"Kevin Clark, Urvashi Khandelwal, Omer Levy, Christopher D Manning","year":"2019","venue":"Proceedings of the 2019 ACL Workshop Black-boxNLP: Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@ACL 2019","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cui et al., 2021] Li Cui, Deqing Yang, Jiaxin Yu, Chengwei Hu, Jiayang Cheng, Jingjie Yi, and Yanghua Xiao. Re- fining sample embeddings with relation prototypes to en- hance continual relation extraction. In ACL, pages 232- 243, Online, August 2021.","title":"Refining sample embeddings with relation prototypes to enhance continual relation extraction","authors":"Li Cui, Deqing Yang, Jiaxin Yu, Chengwei Hu, Jiayang Cheng, Jingjie Yi, Yanghua Xiao","year":"2021","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dosovitskiy et al., 2021] Alexey Dosovitskiy, Lucas Beyer, and Alexander Kolesnikov. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR 2021. OpenReview.net, 2021.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov","year":"2021","venue":"ICLR 2021","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Douillard et al., 2020] Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Pod- net: Pooled outputs distillation for small-tasks incremental learning. In ECCV 2020, volume 12365, pages 86-102. Springer, 2020.","title":"Podnet: Pooled outputs distillation for small-tasks incremental learning","authors":"Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, Eduardo Valle","year":"2020","venue":"ECCV 2020","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Han et al., 2020] Xu Han, Yi Dai, Tianyu Gao, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. Con- tinual relation learning via episodic memory activation and reconsolidation. In ACL, pages 6429-6440, Online, July 2020.","title":"Continual relation learning via episodic memory activation and reconsolidation","authors":"Xu Han, Yi Dai, Tianyu Gao, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2023] Xuming Hu, Junzhe Chen, Aiwei Liu, Shiao Meng, Lijie Wen, and Philip S. Yu. Prompt me up: Unleashing the power of alignments for multimodal entity and relation extraction. In Proceedings of the 31st ACM International Conference on Multimedia, MM 2023, Ot- tawa, ON, Canada, 29 October 2023-3 November 2023, pages 5185-5194. ACM, 2023.","title":"Prompt me up: Unleashing the power of alignments for multimodal entity and relation extraction","authors":"Hu","year":"2023","venue":"Proceedings of the 31st ACM International Conference on Multimedia, MM 2023","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kirkpatrick et al., 2016] James Kirkpatrick, Razvan Pas- canu, Neil C. Rabinowitz, Joel Veness, Guillaume Des- jardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hass- abis, Claudia Clopath, Dharshan Kumaran, and Raia Had- sell. Overcoming catastrophic forgetting in neural net- works. CoRR, abs/1612.00796, 2016.","title":"Overcoming catastrophic forgetting in neural networks","authors":"James Kirkpatrick, Razvan Pascanu, Neil C Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, Raia Hadsell","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lample et al., 2016] Guillaume Lample, Miguel Balles- teros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. Neural architectures for named entity recogni- tion. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 260- 270, San Diego, California, 2016. Association for Compu- tational Linguistics.","title":"Neural architectures for named entity recognition","authors":"Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, Chris Dyer","year":"2016","venue":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liang et al., 2023] Ke Liang, Lingyuan Meng, and et al. Learn from relational correlations and periodic events for temporal knowledge graph reasoning. In SIGIR, page 1559-1568, New York, NY, USA, 2023.","title":"Learn from relational correlations and periodic events for temporal knowledge graph reasoning","authors":"Ke Liang, Lingyuan Meng","year":"2023","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liang et al., 2024] Ke Liang, Yue Liu, Sihang Zhou, Wenx- uan Tu, Yi Wen, Xihong Yang, Xiangjun Dong, and Xin- wang Liu. Knowledge graph contrastive learning based on relation-symmetrical structure. IEEE Trans. Knowl. Data Eng., 36(1):226-238, 2024.","title":"Knowledge graph contrastive learning based on relation-symmetrical structure","authors":"Ke Liang, Yue Liu, Sihang Zhou, Wenxuan Tu, Yi Wen, Xihong Yang, Xiangjun Dong, Xinwang Liu","year":"2024","venue":"IEEE Trans. Knowl. Data Eng","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu and et al, 2019] Ye Liu and Hui Li et al. MMKG: multi- modal knowledge graphs. In ESWC 2019, volume 11503, pages 459-474. Springer, 2019.","title":"MMKG: multimodal knowledge graphs","authors":"Liu","year":"2019","venue":"ESWC 2019","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2023] Meng Liu, Ke Liang, Dayu Hu, Hao Yu, Yue Liu, Lingyuan Meng, Wenxuan Tu, Sihang Zhou, and Xinwang Liu. Tmac: Temporal multi-modal graph learn- ing for acoustic event classification. In Proceedings of the 31st ACM International Conference on Multimedia, MM '23, page 3365-3374, New York, NY, USA, 2023.","title":"Tmac: Temporal multi-modal graph learning for acoustic event classification","authors":"Meng Liu, Ke Liang, Dayu Hu, Hao Yu, Yue Liu, Lingyuan Meng, Wenxuan Tu, Sihang Zhou, Xinwang Liu","year":"2023","venue":"Proceedings of the 31st ACM International Conference on Multimedia, MM '23","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lu and et al, 2018] Di Lu and et al. Visual attention model for name tagging in multimodal social media. In ACL, pages 1990-1999, Melbourne, Australia, 2018.","title":"Visual attention model for name tagging in multimodal social media","authors":"Lu","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lu et al., 2022] Junyu Lu, Dixiang Zhang, Jiaxing Zhang, and Pingjian Zhang. Flat multi-modal interaction trans- former for named entity recognition. In COLING 2022, pages 2055-2064. International Committee on Computa- tional Linguistics, 2022.","title":"Flat multi-modal interaction transformer for named entity recognition","authors":"Lu","year":"2022","venue":"COLING 2022","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Masana et al., 2023] Marc Masana, Xialei Liu, Bartlomiej Twardowski, Mikel Menta, Andrew D. Bagdanov, and Joost van de Weijer. Class-incremental learning: Sur- vey and performance evaluation on image classification. IEEE Trans. Pattern Anal. Mach. Intell., 45(5):5513-5533, 2023.","title":"Class-incremental learning: Survey and performance evaluation on image classification","authors":"Marc Masana, Xialei Liu, Bartlomiej Twardowski, Mikel Menta, Andrew D Bagdanov, Joost Van De Weijer","year":"2023","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Monaikul et al., 2021] Natawut Monaikul, Giuseppe Castellucci, Simone Filice, and Oleg Rokhlenko. Contin- ual learning for named entity recognition. In AAAI 2021, pages 13570-13577. AAAI Press, 2021.","title":"Continual learning for named entity recognition","authors":"Monaikul","year":"2021","venue":"AAAI 2021","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Moon et al., 2018] Seungwhan Moon, Leonardo Neves, and Vitor Carvalho. Multimodal named entity recognition for short social media posts. In NAACL (Long Papers), pages 852-860, New Orleans, Louisiana, 2018.","title":"Multimodal named entity recognition for short social media posts","authors":"Moon","year":"2018","venue":"NAACL (Long Papers)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nair and Hinton, 2010] Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted boltzmann ma- chines. In (ICML-10), pages 807-814. Omnipress, 2010.","title":"Rectified linear units improve restricted boltzmann machines","authors":"Vinod Nair, Geoffrey E Hinton","year":"2010","venue":"ICML-10)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Pelosin et al., 2022] Francesco Pelosin, Saurav Jha, Andrea Torsello, Bogdan Raducanu, and Joost van de Weijer. To- wards exemplar-free continual learning in vision trans- formers: An account of attention, functional and weight regularization. In (CVPR) Workshops, pages 3820-3829, June 2022.","title":"Towards exemplar-free continual learning in vision transformers: An account of attention, functional and weight regularization","authors":"Francesco Pelosin, Saurav Jha, Andrea Torsello, Bogdan Raducanu, Joost Van De Weijer","year":"2022","venue":"CVPR) Workshops","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Peng and et al, 2022] Xiaokang Peng and et al. Balanced multimodal learning via on-the-fly gradient modulation. In CVPR, 2022.","title":"Balanced multimodal learning via on-the-fly gradient modulation","authors":"Peng","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Rusu et al., 2016] Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Had- sell. Progressive neural networks. CoRR, abs/1606.04671, 2016.","title":"Progressive neural networks","authors":"Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, Raia Hadsell","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sprechmann et al., 2018] Pablo Sprechmann, Siddhant M. Jayakumar, Jack W. Rae, Alexander Pritzel, Adrià Puig- domènech Badia, Benigno Uria, Oriol Vinyals, Demis Hassabis, Razvan Pascanu, and Charles Blundell. Memory-based parameter adaptation. In ICLR 2018. OpenReview.net, 2018.","title":"Memory-based parameter adaptation","authors":"Pablo Sprechmann, M Siddhant, Jack W Jayakumar, Alexander Rae, Adrià Pritzel, Benigno Puigdomènech Badia, Oriol Uria, Demis Vinyals, Razvan Hassabis, Charles Pascanu, Blundell","year":"2018","venue":"ICLR 2018","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Vaswani and et al, 2017] Ashish Vaswani and et al. Atten- tion is all you need. In Neurips, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani","year":"2017","venue":"Neurips","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wan and et al, 2021] Hai Wan and et al. FL-MSRE: A few- shot learning based approach to multimodal social relation extraction. In AAAI 2021, 2021.","title":"FL-MSRE: A fewshot learning based approach to multimodal social relation extraction","authors":"Wan","year":"2021","venue":"AAAI 2021","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2019] Hong Wang, Wenhan Xiong, Mo Yu, Xi- aoxiao Guo, Shiyu Chang, and William Yang Wang. Sen- tence embedding alignment for lifelong relation extrac- tion. In NAACL (Long and Short Papers), pages 796-806, Minneapolis, Minnesota, June 2019.","title":"Sentence embedding alignment for lifelong relation extraction","authors":"Wang","year":"2019","venue":"NAACL (Long and Short Papers)","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2020] Weiyao Wang, Du Tran, and Matt Feis- zli. What makes training multi-modal classification net- works hard? In CVPR 2020, 2020.","title":"What makes training multi-modal classification networks hard?","authors":"Wang","year":"2020","venue":"CVPR 2020","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2022a] Rui Wang, Tong Yu, Handong Zhao, Sungchul Kim, Subrata Mitra, Ruiyi Zhang, and Ricardo Henao. Few-shot class-incremental learning for named en- tity recognition. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 571-582, Dublin, Ireland, May 2022. Association for Computational Linguistics.","title":"Few-shot class-incremental learning for named entity recognition","authors":"Wang","year":"2022","venue":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wang et al., 2022b] Xinyu Wang, Min Gui, Yong Jiang, Zixia Jia, Nguyen Bach, Tao Wang, Zhongqiang Huang, and Kewei Tu. ITA: image-text alignments for multi- modal named entity recognition. In NAACL 2022, pages 3176-3189, 2022.","title":"ITA: image-text alignments for multimodal named entity recognition","authors":"Wang","year":"2022","venue":"NAACL 2022","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xia et al., 2022] Yu Xia, Quan Wang, Yajuan Lyu, Yong Zhu, Wenhao Wu, Sujian Li, and Dai Dai. Learn and re- view: Enhancing continual named entity recognition via reviewing synthetic samples. In Findings of the Asso- ciation for Computational Linguistics: ACL 2022, pages 2291-2300, Dublin, Ireland, May 2022. Association for Computational Linguistics.","title":"Learn and review: Enhancing continual named entity recognition via reviewing synthetic samples","authors":"Yu Xia, Quan Wang, Yajuan Lyu, Yong Zhu, Wenhao Wu, Sujian Li, Dai Dai","year":"2022","venue":"Findings of the Association for Computational Linguistics: ACL 2022","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yu et al., 2020] Jianfei Yu, Jing Jiang, Li Yang, and Rui Xia. Improving multimodal named entity recognition via entity span detection with unified multimodal transformer. In ACL, pages 3342-3352, Online, 2020. Association for Computational Linguistics.","title":"Improving multimodal named entity recognition via entity span detection with unified multimodal transformer","authors":"Jianfei Yu, Jing Jiang, Li Yang, Rui Xia","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zenke et al., 2017] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic in- telligence. In ICML 2017, volume 70, pages 3987-3995.","title":"Continual learning through synaptic intelligence","authors":"Friedemann Zenke, Ben Poole, Surya Ganguli","year":null,"venue":"ICML 2017","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"PMLR, 2017.","title":null,"authors":"Pmlr","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhang et al., 2018] Qi Zhang, Jinlan Fu, Xiaoyu Liu, and Xuanjing Huang. Adaptive co-attention network for named entity recognition in tweets. In (AAAI-18), pages 5674-5681. AAAI Press, 2018.","title":"Adaptive co-attention network for named entity recognition in tweets","authors":"Zhang","year":"2018","venue":"AAAI-18)","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhang et al., 2021] Dong Zhang, Suzhong Wei, Shoushan Li, Hanqian Wu, Qiaoming Zhu, and Guodong Zhou. Multi-modal graph fusion for named entity recognition with targeted visual guidance. In AAAI 2021,, pages 14347-14355. AAAI Press, 2021.","title":"Multi-modal graph fusion for named entity recognition with targeted visual guidance","authors":"Dong Zhang, Suzhong Wei, Shoushan Li, Hanqian Wu, Qiaoming Zhu, Guodong Zhou","year":"2021","venue":"AAAI 2021","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhang et al., 2023] Ningyu Zhang, Lei Li, Xiang Chen, Xi- aozhuan Liang, Shumin Deng, and Huajun Chen. Mul- timodal analogical reasoning over knowledge graphs. In ICLR, 2023.","title":"Multimodal analogical reasoning over knowledge graphs","authors":"Ningyu Zhang, Lei Li, Xiang Chen, Xiaozhuan Liang, Shumin Deng, Huajun Chen","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zheng et al., 2021a] Changmeng Zheng, Junhao Feng, Ze Fu, Yi Cai, Qing Li, and Tao Wang. Multimodal re- lation extraction with efficient graph alignment. In ACM Multimedia 2021, pages 5298-5306. ACM, 2021.","title":"Multimodal relation extraction with efficient graph alignment","authors":"Zheng","year":"2021","venue":"ACM Multimedia 2021","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zheng et al., 2021b] Changmeng Zheng, Zhiwei Wu, Jun- hao Feng, Ze Fu, and Yi Cai. Mnre: A challenge mul- timodal dataset for neural relation extraction with visual evidence in social media posts. In ICME, pages 1-6, 2021.","title":"Mnre: A challenge multimodal dataset for neural relation extraction with visual evidence in social media posts","authors":"Zheng","year":"2021","venue":"ICME","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zheng et al., 2023] Shangfei Zheng, Weiqing Wang, Jian- feng Qu, Hongzhi Yin, Wei Chen, and Lei Zhao. MMKGR: multi-hop multi-modal knowledge graph rea- soning. In ICDE 2023, pages 96-109. IEEE, 2023.","title":"MMKGR: multi-hop multi-modal knowledge graph reasoning","authors":"Zheng","year":"2023","venue":"ICDE 2023","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhu et al., 2022] Rui Zhu, Yiming Zhao, Wei Qu, Zhongyi Liu, and Chenliang Li. Cross-domain product search with knowledge graph. In Mohammad Al Hasan and Li Xiong, editors, Proceedings of the 31st ACM International Con- ference on Information & Knowledge Management, At- lanta, GA, USA, October 17-21, 2022, 2022.","title":"Cross-domain product search with knowledge graph","authors":"Rui Zhu, Yiming Zhao, Wei Qu, Zhongyi Liu, Chenliang Li","year":"2022","venue":"Proceedings of the 31st ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}