{"type":"ScholarlyArticle","title":"MFAN: Multi-modal Feature-enhanced Attention Networks for Rumor Detection","authors":[{"name":"Jiaqi Zheng","slug":"jiaqi-zheng","given":"Jiaqi","family":"Zheng","affiliation":"Key Laboratory of Trustworthy Distributed Computing and Service (MoE) , Beijing University of Posts and Telecommunications , China"},{"name":"Xi Zhang","slug":"xi-zhang","given":"Xi","family":"Zhang","affiliation":"Key Laboratory of Trustworthy Distributed Computing and Service (MoE) , Beijing University of Posts and Telecommunications , China"},{"name":"Sanchuan Guo","slug":"sanchuan-guo","given":"Sanchuan","family":"Guo","affiliation":"Key Laboratory of Trustworthy Distributed Computing and Service (MoE) , Beijing University of Posts and Telecommunications , China"},{"name":"Quan Wang","slug":"quan-wang","given":"Quan","family":"Wang","affiliation":"Key Laboratory of Trustworthy Distributed Computing and Service (MoE) , Beijing University of Posts and Telecommunications , China"},{"name":"Wenyu Zang","slug":"wenyu-zang","given":"Wenyu","family":"Zang","affiliation":"China Electronics Corporation"},{"name":"Yongdong Zhang","slug":"yongdong-zhang","given":"Yongdong","family":"Zhang","affiliation":"University of Science and Technology of China; Key Laboratory of Trustworthy Distributed Computing and Service (MoE) , Beijing University of Posts and Telecommunications , China"}],"abstract":"Rumor spreaders are increasingly taking advantage of multimedia content to attract and mislead news consumers on social media. Although recent multimedia rumor detection models have exploited both textual and visual features for classification, they do not integrate the social structure features simultaneously, which have shown promising performance for rumor identification. It is challenging to combine the heterogeneous multi-modal data in consideration of their complex relationships. In this work, we propose a novel Multi-modal Feature-enhanced Attention Networks (MFAN) for rumor detection, which makes the first attempt to integrate textual, visual, and social graph features in one unified framework. Specifically, it considers both the complement and alignment relationships between different modalities to achieve better fusion. Moreover, it takes into account the incomplete links in the social network data due to data collection constraints and proposes to infer hidden links to learn better social graph features. The experimental results show that MFAN can detect rumors effectively and outperform state-of-the-art methods.","doi":"10.24963/ijcai.2022/335","url":"https://ijcai.org/proceedings/2022/335","pdf":"https://ijcai.org/proceedings/2022/0335.pdf","markdown":"https://ijcai.org/proceedings/2022/335.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/335","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Data Mining","track":"Main Track","number":335,"pages":{"first":2413,"last":2419},"keywords":["Machine Learning: Multi-modal learning","Data Mining: Mining Text, Web, Social Media","Machine Learning: Attention Models","Natural Language Processing: Text Classification","Natural Language Processing: Information Retrieval and Text Mining"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986907","length_sec":83,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985050","length_sec":324,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Castillo et al., 2011] Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. Information credibility on twitter. In WWW, pages 675-684, 2011.","title":"Information credibility on twitter","authors":"Carlos Castillo, Marcelo Mendoza, Barbara Poblete","year":"2011","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Jin et al., 2017] Zhiwei Jin, Juan Cao, Han Guo, Yongdong Zhang, and Jiebo Luo. Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In ACM Multimedia, pages 795-816, 2017.","title":"Multimodal fusion with recurrent neural networks for rumor detection on microblogs","authors":"Zhiwei Jin, Juan Cao, Han Guo, Yongdong Zhang, Jiebo Luo","year":"2017","venue":"ACM Multimedia","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Karimi et al., 2018] Hamid Karimi, Proteek Roy, Sari Saba- Sadiya, and Jiliang Tang. Multi-source multi-class fake news detection. In COLING, pages 1546-1557, 2018.","title":"Multi-source multi-class fake news detection","authors":"Hamid Karimi, Proteek Roy, Sari Saba-Sadiya, Jiliang Tang","year":"2018","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Khattar et al., 2019] Dhruv Khattar, Jaipal Singh Goud, Manish Gupta, and Vasudeva Varma. Mvae: Multimodal variational autoencoder for fake news detection. In WWW, pages 2915-2921, 2019.","title":"Mvae: Multimodal variational autoencoder for fake news detection","authors":"Dhruv Khattar, Jaipal Singh Goud, Manish Gupta, Vasudeva Varma","year":"2019","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lu and Li, 2020] Yi-Ju Lu and Cheng-Te Li. Gcan: Graph- aware co-attention networks for explainable fake news de- tection on social media. arXiv preprint arXiv:2004.11648, 2020.","title":"Gcan: Graphaware co-attention networks for explainable fake news detection on social media","authors":"Li ; Yi-Ju Lu, Cheng-Te Lu, Li","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lu et al., 2019] Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguis- tic representations for vision-and-language tasks. arXiv preprint arXiv:1908.02265, 2019.","title":"Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks","authors":"Lu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ma et al., 2016] Jing Ma, Wei Gao, Prasenjit Mitra, Sejeong Kwon, Bernard J Jansen, Kam-Fai Wong, and Meeyoung Cha. Detecting rumors from microblogs with recurrent neural networks. 2016.","title":"Detecting rumors from microblogs with recurrent neural networks","authors":"Jing Ma, Wei Gao, Prasenjit Mitra, Sejeong Kwon, Bernard J Jansen, Kam-Fai Wong, Meeyoung Cha","year":"2016","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2016/537"},{"position":11,"raw":"[Ma et al., 2018] Jing Ma, Wei Gao, and Kam-Fai Wong. Rumor detection on twitter with tree-structured recursive neural networks. ACL, 2018.","title":"Rumor detection on twitter with tree-structured recursive neural networks","authors":"Jing Ma, Wei Gao, Kam-Fai Wong","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Madry et al., 2017] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017.","title":"Towards deep learning models resistant to adversarial attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Popat, 2017] Kashyap Popat. Assessing the credibility of claims on the web. In WWW Companion, pages 735-739, 2017.","title":"Assessing the credibility of claims on the web","authors":"Kashyap Popat","year":"2017","venue":"WWW Companion","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Qian et al., 2021] Shengsheng Qian, Jinguang Wang, Jun Hu, Quan Fang, and Changsheng Xu. Hierarchical multi- modal contextual attention network for fake news detec- tion. In SIGIR, pages 153-162, 2021.","title":"Hierarchical multimodal contextual attention network for fake news detection","authors":"Qian","year":"2021","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Song et al., 2019] Changhe Song, Cheng Yang, Huimin Chen, Cunchao Tu, Zhiyuan Liu, and Maosong Sun. Ced: Credible early detection of social media rumors. IEEE Transactions on Knowledge and Data Engineering, 33(8):3035-3047, 2019.","title":"Ced: Credible early detection of social media rumors","authors":"Changhe Song, Cheng Yang, Huimin Chen, Cunchao Tu, Zhiyuan Liu, Maosong Sun","year":"2019","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tian et al., 2020] Tian Tian, Yudong Liu, Xiaoyu Yang, Yuefei Lyu, Xi Zhang, and Binxing Fang. Qsan: A quantum-probability based signed attention network for explainable false information detection. In CIKM, pages 1445-1454, 2020.","title":"Qsan: A quantum-probability based signed attention network for explainable false information detection","authors":"Tian Tian, Yudong Liu, Xiaoyu Yang, Yuefei Lyu, Xi Zhang, Binxing Fang","year":"2020","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Veličković et al., 2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2018] Yaqing Wang, Fenglong Ma, Zhiwei Jin, Ye Yuan, Guangxu Xun, Kishlay Jha, Lu Su, and Jing Gao. Eann: Event adversarial neural networks for multi-modal fake news detection. In KDD, pages 849-857, 2018.","title":"Eann: Event adversarial neural networks for multi-modal fake news detection","authors":"Wang","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wei et al., 2021] Lingwei Wei, Dou Hu, Wei Zhou, Zhao- juan Yue, and Songlin Hu. Towards propagation uncertainty: Edge-enhanced bayesian graph convolu- tional networks for rumor detection. arXiv preprint arXiv:2107.11934, 2021.","title":"Towards propagation uncertainty: Edge-enhanced bayesian graph convolutional networks for rumor detection","authors":"Lingwei Wei, Dou Hu, Wei Zhou, Zhaojuan Yue, Songlin Hu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wu et al., 2021] Yang Wu, Pengwei Zhan, Yunjian Zhang, Liming Wang, and Zhen Xu. Multimodal fusion with co- attention networks for fake news detection. In Findings of the Association for Computational Linguistics: ACL- IJCNLP 2021, pages 2560-2569, 2021.","title":"Multimodal fusion with coattention networks for fake news detection","authors":"Yang Wu, Pengwei Zhan, Yunjian Zhang, Liming Wang, Zhen Xu","year":"2021","venue":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang et al., 2021] Xiaoyu Yang, Yuefei Lyu, Tian Tian, Yifei Liu, Yudong Liu, and Xi Zhang. Rumor detection on social media with graph structured adversarial learning. In IJCAI, pages 1417-1423, 2021.","title":"Rumor detection on social media with graph structured adversarial learning","authors":"Xiaoyu Yang, Yuefei Lyu, Tian Tian, Yifei Liu, Yudong Liu, Xi Zhang","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/197"},{"position":23,"raw":"[Yu et al., 2017] Feng Yu, Qiang Liu, Shu Wu, Liang Wang, Tieniu Tan, et al. A convolutional approach for misinfor- mation identification. In IJCAI, pages 3901-3907, 2017.","title":"A convolutional approach for misinformation identification","authors":"Feng Yu, Qiang Liu, Shu Wu, Liang Wang, Tieniu Tan","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/545"},{"position":24,"raw":"[Yuan et al., 2019] Chunyuan Yuan, Qianwen Ma, Wei Zhou, Jizhong Han, and Songlin Hu. Jointly embedding the local and global relations of heterogeneous graph for rumor detection. In ICDM, pages 796-805. IEEE, 2019.","title":"Jointly embedding the local and global relations of heterogeneous graph for rumor detection","authors":"Chunyuan Yuan, Qianwen Ma, Wei Zhou, Jizhong Han, Songlin Hu","year":"2019","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou et al., 2020] Xinyi Zhou, Jindi Wu, and Reza Za- farani. Safe: Similarity-aware multi-modal fake news de- tection. arXiv preprint arXiv:2003.04981, 2020.","title":"Safe: Similarity-aware multi-modal fake news detection","authors":"Xinyi Zhou, Jindi Wu, Reza Zafarani","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zubiaga et al., 2017] Arkaitz Zubiaga, Maria Liakata, and Rob Procter. Exploiting context for rumour detection in social media. In International Conference on Social Infor- matics, pages 109-123. Springer, 2017.","title":"Exploiting context for rumour detection in social media","authors":"Zubiaga","year":"2017","venue":"International Conference on Social Informatics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}