{"type":"ScholarlyArticle","title":"Clickbait Detection via Contrastive Variational Modelling of Text and Label","authors":[{"name":"Xiaoyuan Yi","slug":"xiaoyuan-yi","given":"Xiaoyuan","family":"Yi","affiliation":"Microsoft Research Asia"},{"name":"Jiarui Zhang","slug":"jiarui-zhang","given":"Jiarui","family":"Zhang","affiliation":"Tsinghua University"},{"name":"Wenhao Li","slug":"wenhao-li","given":"Wenhao","family":"Li","affiliation":"Tsinghua University"},{"name":"Xiting Wang","slug":"xiting-wang","given":"Xiting","family":"Wang","affiliation":"Microsoft Research Asia"},{"name":"Xing Xie","slug":"xing-xie","given":"Xing","family":"Xie","affiliation":"Microsoft Research Asia"}],"abstract":"Clickbait refers to deliberately created sensational or deceptive text for tricking readers into clicking, which severely hurts the web ecosystem. With a growing number of clickbaits on social media, developing automatic detection methods becomes essential. Nonetheless, the performance of existing neural classifiers is limited due to the underutilization of small labelled datasets. Inspired by related pedagogy theories that learning to write can promote comprehension ability, we propose a novel Contrastive Variational Modelling (CVM) framework to exploit the labelled data better. CVM models the conditional distributions of text and clickbait labels by predicting labels from text and generating text from labels simultaneously with Variational AutoEncoder and further differentiates the learned spaces under each label by a mixed contrastive learning loss. In this way, CVM can capture more underlying textual properties and hence utilize label information to its full potential, boosting detection performance. We theoretically demonstrate CVM as learning a joint distribution of text, clickbait label, and latent variable. Experiments on three clickbait detection datasets show our method's robustness to inadequate and biased labels, outperforming several recent strong baselines.","doi":"10.24963/ijcai.2022/621","url":"https://ijcai.org/proceedings/2022/621","pdf":"https://ijcai.org/proceedings/2022/0621.pdf","markdown":"https://ijcai.org/proceedings/2022/621.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/621","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":621,"pages":{"first":4475,"last":4481},"keywords":["Natural Language Processing: Text Classification"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987037","length_sec":91,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985180","length_sec":702,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Anand et al., 2017] Ankesh Anand, Tanmoy Chakraborty, and Noseong Park. We used neural networks to detect clickbaits: You won't believe what happened next! In ECIR, pages 541-547, 2017.","title":"We used neural networks to detect clickbaits: You won't believe what happened next!","authors":"Ankesh Anand, Tanmoy Chakraborty, Noseong Park","year":"2017","venue":"ECIR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bishop and Lasserre, 2007] Christopher M. Bishop and Ju- lia Lasserre. Generative or discriminative? getting the best of both worlds. Bayesian statistics, 8(3):3-24, 2007.","title":"Generative or discriminative? getting the best of both worlds","authors":"Christopher M Bishop, Julia Lasserre","year":"2007","venue":"Bayesian statistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Biyani et al., 2016] Prakhar Biyani, Kostas Tsioutsiouliklis, and John Blackmer. \"8 amazing secrets for getting more clicks\": detecting clickbaits in news streams using article informality. In AAAI, pages 94-100, 2016.","title":"8 amazing secrets for getting more clicks\": detecting clickbaits in news streams using article informality","authors":"Prakhar Biyani, Kostas Tsioutsiouliklis, John Blackmer","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Caccamise, 2011] Donna Caccamise. Improved reading comprehension by writing. Perspectives on Language Learning and Education, 18(1):27-31, 2011.","title":"Improved reading comprehension by writing","authors":"Donna Caccamise","year":"2011","venue":"Perspectives on Language Learning and Education","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chakraborty et al., 2016] Abhijnan Chakraborty, Bhargavi Paranjape, Sourya Kakarla, and Niloy Ganguly. Stop clickbait: Detecting and preventing clickbaits in online news media. In ASONAM, pages 9-16. IEEE, 2016.","title":"Stop clickbait: Detecting and preventing clickbaits in online news media","authors":"Chakraborty","year":"2016","venue":"ASONAM","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cheng et al., 2019] Xingyi Cheng, Weidi Xu, Taifeng Wang, Wei Chu, Weipeng Huang, Kunlong Chen, et al. Variational semi-supervised aspect-term sentiment analy- sis via transformer. In CoNLL, pages 961-969, 2019.","title":"Variational semi-supervised aspect-term sentiment analysis via transformer","authors":"Cheng","year":"2019","venue":"CoNLL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understand- ing. In NAACL-HLT, pages 4171-4186, 2019.","title":"BERT: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ding and Gimpel, 2019] Xiaoan Ding and Kevin Gimpel. Latent-variable generative models for data-efficient text classification. arXiv preprint arXiv:1910.00382, 2019.","title":"Xiaoan Ding and Kevin Gimpel. Latent-variable generative models for data-efficient text classification","authors":"Gimpel Ding","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dong et al., 2019] Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. Unified language model pretrain- ing for natural language understanding and generation. In NeurIPS, 2019.","title":"Unified language model pretraining for natural language understanding and generation","authors":"Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, Hsiao-Wuen Hon","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ecker et al., 2014] Ullrich K.H. Ecker, Stephan Lewandowsky, Ee Pin Chang, and Rekha Pillai. The effects of subtle misinformation in news headlines. Journal of experimental psychology: applied, 20(4):323, 2014.","title":"The effects of subtle misinformation in news headlines","authors":"K H Ullrich, Stephan Ecker, Ee Pin Lewandowsky, Rekha Chang, Pillai","year":"2014","venue":"Journal of experimental psychology: applied","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fu et al., 2019] Hao Fu, Chunyuan Li, Xiaodong Liu, Jian- feng Gao, Asli Celikyilmaz, and Lawrence Carin. Cycli- cal annealing schedule: A simple approach to mitigating kl vanishing. In NAACL, pages 240-250, 2019.","title":"Cyclical annealing schedule: A simple approach to mitigating kl vanishing","authors":"Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, Lawrence Carin","year":"2019","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gururangan et al., 2019] Suchin Gururangan, Tam Dang, Dallas Card, and Noah A Smith. Variational pretrain- ing for semi-supervised text classification. In ACL, pages 5880-5894, 2019.","title":"Variational pretraining for semi-supervised text classification","authors":"Suchin Gururangan, Tam Dang, Dallas Card, Noah A Smith","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ha et al., 2018] Yui Ha, Jeongmin Kim, Donghyeon Won, Meeyoung Cha, and Jungseock Joo. Characterizing click- baits on instagram. In ICWSM, pages 92-101, 2018.","title":"Characterizing clickbaits on instagram","authors":"Yui Ha, Jeongmin Kim, Donghyeon Won, Meeyoung Cha, Jungseock Joo","year":"2018","venue":"ICWSM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Indurthi et al., 2020] Vijayasaradhi Indurthi, Bakhtiyar Syed, Manish Gupta, and Vasudeva Varma. Predicting clickbait strength in online social media. In COLING, pages 4835-4846, 2020.","title":"Predicting clickbait strength in online social media","authors":"Indurthi","year":"2020","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kingma and Welling, 2014] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2020] Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun Li, Yizhe Zhang, and Jianfeng Gao. Opti- mus: Organizing sentences via pre-trained modeling of a latent space. In EMNLP, pages 4678-4699, 2020.","title":"Optimus: Organizing sentences via pre-trained modeling of a latent space","authors":"Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun Li, Yizhe Zhang, Jianfeng Gao","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[McCallum et al., 2006] Andrew McCallum, Chris Pal, Gre- gory Druck, and Xuerui Wang. Multi-conditional learning: Generative/discriminative training for clustering and clas- sification. In AAAI, pages 433-439, 2006.","title":"Multi-conditional learning: Generative/discriminative training for clustering and classification","authors":"Andrew Mccallum, Chris Pal, Gregory Druck, Xuerui Wang","year":"2006","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mishra and Zhang, 2021] Rahul Mishra and Shuo Zhang. Poshan: Cardinal pos pattern guided attention for news headline incongruence. In CIKM, pages 1294-1303, 2021.","title":"Poshan: Cardinal pos pattern guided attention for news headline incongruence","authors":"Rahul Mishra, Shuo Zhang","year":"2021","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mishra et al., 2020] Rahul Mishra, Piyush Yadav, Remi Calizzano, and Markus Leippold. Musem: Detecting in- congruent news headlines using mutual attentive semantic matching. In ICMLA, pages 709-716. IEEE, 2020.","title":"Musem: Detecting incongruent news headlines using mutual attentive semantic matching","authors":"Rahul Mishra, Piyush Yadav, Remi Calizzano, Markus Leippold","year":"2020","venue":"ICMLA","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ng and Jordan, 2002] Andrew Y Ng and Michael I Jordan. On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. In NeurIPS, pages 841-848, 2002.","title":"On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes","authors":"Y Andrew, Michael I Ng, Jordan","year":"2002","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Oord et al., 2018] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748, 2018.","title":"Representation learning with contrastive predictive coding","authors":"Aaron Van Den Oord, Yazhe Li, Oriol Vinyals","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, et al. Language models are unsupervised multitask learners. OpenAI blog, 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei","year":"2019","venue":"OpenAI blog","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rony et al., 2017] Md Main Uddin Rony, Naeemul Hassan, and Mohammad Yousuf. Diving deep into clickbaits: Who use them to what extents in which topics with what effects? In ASONAM, pages 232-239, 2017.","title":"Diving deep into clickbaits: Who use them to what extents in which topics with what effects?","authors":"Md Main, Uddin Rony, Naeemul Hassan, Mohammad Yousuf","year":"2017","venue":"ASONAM","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tseng et al., 2020] Bo-Hsiang Tseng, Jianpeng Cheng, Yi- mai Fang, and David Vandyke. A generative model for joint natural language understanding and generation. In ACL, pages 1795-1807, 2020.","title":"A generative model for joint natural language understanding and generation","authors":"Bo-Hsiang Tseng, Jianpeng Cheng, Yimai Fang, David Vandyke","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wei and Wan, 2017] Wei Wei and Xiaojun Wan. Learning to identify ambiguous and misleading news headlines. In IJCAI, pages 4172-4178, 2017.","title":"Learning to identify ambiguous and misleading news headlines","authors":"Wei Wei, Xiaojun Wan","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/583"},{"position":27,"raw":"[Xia et al., 2017] Yingce Xia, Tao Qin, Wei Chen, Jiang Bian, Nenghai Yu, and Tie-Yan Liu. Dual supervised learning. In ICML, pages 3789-3798. PMLR, 2017.","title":"Dual supervised learning","authors":"Yingce Xia, Tao Qin, Wei Chen, Jiang Bian, Nenghai Yu, Tie-Yan Liu","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yogatama et al., 2017] Dani Yogatama, Chris Dyer, Wang Ling, and Phil Blunsom. Generative and discriminative text classification with recurrent neural networks. arXiv preprint arXiv:1703.01898, 2017.","title":"Generative and discriminative text classification with recurrent neural networks","authors":"Dani Yogatama, Chris Dyer, Wang Ling, Phil Blunsom","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yoon et al., 2019] Seunghyun Yoon, Kunwoo Park, Joongbo Shin, Hongjun Lim, Seungpil Won, Meeyoung Cha, and Kyomin Jung. Detecting incongruity between news headline and body text via a deep hierarchical encoder. In AAAI, pages 791-800, 2019.","title":"Detecting incongruity between news headline and body text via a deep hierarchical encoder","authors":"Seunghyun Yoon, Kunwoo Park, Joongbo Shin, Hongjun Lim, Seungpil Won, Meeyoung Cha, Kyomin Jung","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}