{"type":"ScholarlyArticle","title":"Towards Semantics- and Domain-Aware Adversarial Attacks","authors":[{"name":"Jianping Zhang","slug":"jianping-zhang","given":"Jianping","family":"Zhang","affiliation":"Department of Computer Science and Engineering , The Chinese University of Hong Kong"},{"name":"Yung-Chieh Huang","slug":"yung-chieh-huang","given":"Yung-Chieh","family":"Huang","affiliation":"Department of Computer Science , University of Illinois Urbana-Champaign"},{"name":"Weibin Wu","slug":"weibin-wu","given":"Weibin","family":"Wu","affiliation":"School of Software Engineering , Sun Yat-sen University"},{"name":"Michael R. Lyu","slug":"michael-r-lyu","given":"Michael R.","family":"Lyu","affiliation":"Department of Computer Science and Engineering , The Chinese University of Hong Kong"}],"abstract":"Language models are known to be vulnerable to textual adversarial attacks, which add human-imperceptible perturbations to the input to mislead DNNs. It is thus imperative to devise effective attack algorithms to identify the deficiencies of DNNs before real-world deployment. However, existing word-level attacks have two major deficiencies: (1) They may change the semantics of the original sentence. (2) The generated adversarial sample can appear unnatural to humans due to the introduction of out-of-domain substitute words. In this paper, to address such drawbacks, we propose a semantics- and domain-aware word-level attack method. Specifically, we greedily replace the important words in a sentence with the ones suggested by a language model. The language model is trained to be semantics- and domain-aware via contrastive learning and in-domain pre-training. Furthermore, to balance the quality of adversarial examples and the attack success rate, we propose an iterative updating framework to optimize the contrastive learning loss and the in-domain pre-training loss in circular order. Comprehensive experimental comparisons confirm the superiority of our approach. Notably, compared with state-of-the-art benchmarks, our strategy can achieve over 3\\% improvement in attack success rates and 9.8\\% improvement in the quality of adversarial examples.","doi":"10.24963/ijcai.2023/60","url":"https://ijcai.org/proceedings/2023/60","pdf":"https://ijcai.org/proceedings/2023/0060.pdf","markdown":"https://ijcai.org/proceedings/2023/60.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/60","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI Ethics, Trust, Fairness","track":"Main Track","number":60,"pages":{"first":536,"last":544},"keywords":["AI Ethics, Trust, Fairness: ETF: Safety and robustness","Natural Language Processing: NLP: Interpretability and analysis of models for NLP"],"videos":[],"references":[{"position":1,"raw":"[Alzantot et al., 2018] Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai- Wei Chang. Generating natural language adversarial ex- amples. In Proceedings of the 2018 Conference on Em- pirical Methods in Natural Language Processing, pages 2890-2896, Brussels, Belgium, October-November 2018. Association for Computational Linguistics.","title":"Generating natural language adversarial examples","authors":"Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, Kai-Wei Chang","year":"2018","venue":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Birjali et al., 2021] Marouane Birjali, Mohammed Kasri, and Abderrahim Beni-Hssane. A comprehensive survey on sentiment analysis: Approaches, challenges and trends. Knowledge-Based Systems, 226:107134, 2021.","title":"A comprehensive survey on sentiment analysis: Approaches, challenges and trends","authors":"Birjali","year":"2021","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bowman et al., 2015] Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632-642, Lisbon, Portugal, September 2015. Association for Com- putational Linguistics.","title":"A large annotated corpus for learning natural language inference","authors":"R Samuel, Gabor Bowman, Christopher Angeli, Christopher D Potts, Manning","year":"2015","venue":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021] Yangyi Chen, Jin Su, and Wei Wei. Multi-granularity textual adversarial attack with behavior cloning. In Proceedings of the 2021 Conference on Em- pirical Methods in Natural Language Processing, pages 4511-4526, 2021.","title":"Multi-granularity textual adversarial attack with behavior cloning","authors":"Chen","year":"2021","venue":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ebrahimi et al., 2018] Javid Ebrahimi, Daniel Lowd, and Dejing Dou. On adversarial examples for character-level neural machine translation. In Proceedings of the 27th International Conference on Computational Linguistics, pages 653-663, 2018.","title":"On adversarial examples for character-level neural machine translation","authors":"Javid Ebrahimi, Daniel Lowd, Dejing Dou","year":"2018","venue":"Proceedings of the 27th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gao et al., 2021] Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simcse: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894-6910, 2021.","title":"Simcse: Simple contrastive learning of sentence embeddings","authors":"Tianyu Gao, Xingcheng Yao, Danqi Chen","year":"2021","venue":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gururangan et al., 2020] Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Belt- agy, Doug Downey, and Noah A. Smith. Don't stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342- 8360, Online, July 2020. Association for Computational Linguistics.","title":"Don't stop pretraining: Adapt language models to domains and tasks","authors":"Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A Smith","year":"2020","venue":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2022] Jen-tse Huang, Jianping Zhang, Wenx- uan Wang, Pinjia He, Yuxin Su, and Michael R. Lyu. Aeon: A method for automatic evaluation of nlp test cases. In Proceedings of the 31st ACM SIGSOFT Inter- national Symposium on Software Testing and Analysis, IS- STA 2022, page 202-214, New York, NY, USA, 2022. As- sociation for Computing Machinery. Huq et al., 2020]","title":"Aeon: A method for automatic evaluation of nlp test cases","authors":"Jen-Tse Huang, Jianping Zhang, Wenxuan Wang, Pinjia He, Yuxin Su, Michael R Lyu","year":"2020","venue":"Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, IS-STA 2022","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"Aminul Huq, Mst Pervin, et al. Adversar- ial attacks and defense on texts: A survey. arXiv preprint arXiv:2005.14108, 2020.","title":"Adversarial attacks and defense on texts: A survey","authors":"Aminul Huq, Mst Pervin","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Iyyer et al., 2018] Mohit Iyyer, John Wieting, Kevin Gim- pel, and Luke Zettlemoyer. Adversarial example genera- tion with syntactically controlled paraphrase networks. In Proceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume 1 (Long Papers), pages 1875-1885, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.","title":"Adversarial example generation with syntactically controlled paraphrase networks","authors":"Mohit Iyyer, John Wieting, Kevin Gimpel, Luke Zettlemoyer","year":"2018","venue":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jia and Liang, 2017] Robin Jia and Percy Liang. Adversar- ial examples for evaluating reading comprehension sys- tems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2021- 2031, Copenhagen, Denmark, September 2017. Associa- tion for Computational Linguistics.","title":"Adversarial examples for evaluating reading comprehension systems","authors":"Liang Jia, Robin Jia, Percy Liang","year":"2017","venue":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jin et al., 2020] Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entail- ment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8018-8025, 2020.","title":"Is bert really robust? a strong baseline for natural language attack on text classification and entailment","authors":"Di Jin, Zhijing Jin, Joey Tianyi Zhou, Peter Szolovits","year":"2020","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kenton and Toutanova, 2019] Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. In Proceedings of NAACL-HLT, pages 4171-4186, 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton, Lee Kristina, Toutanova","year":"2019","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Levenshtein and others, 1966] Vladimir I Levenshtein et al. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, volume 10, pages 707-710. Soviet Union, 1966.","title":"Binary codes capable of correcting deletions, insertions, and reversals","authors":"Vladimir I Levenshtein","year":"1966","venue":"Soviet physics doklady","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2020] Linyang Li, Ruotian Ma, Qipeng Guo, Xi- angyang Xue, and Xipeng Qiu. Bert-attack: Adversarial attack against bert using bert. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6193-6202, 2020.","title":"Bert-attack: Adversarial attack against bert using bert","authors":"Li","year":"2020","venue":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.","title":"Roberta: A robustly optimized bert pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2022a] Zihan Liu, Yun Luo, Lirong Wu, Zicheng Liu, and Stan Z Li. Towards reasonable budget allocation in untargeted graph structure attacks via gradient debias. In Advances in Neural Information Processing Systems, 2022.","title":"Towards reasonable budget allocation in untargeted graph structure attacks via gradient debias","authors":"Liu","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2022b] Zihan Liu, Yun Luo, Zelin Zang, and Stan Z Li. Surrogate representation learning with isometric mapping for gray-box graph adversarial attacks. In Pro- ceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pages 591-598, 2022.","title":"Surrogate representation learning with isometric mapping for gray-box graph adversarial attacks","authors":"Liu","year":"2022","venue":"Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Maas et al., 2011] Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Pro- ceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142-150, 2011.","title":"Learning word vectors for sentiment analysis","authors":"Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, Christopher Potts","year":"2011","venue":"Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Miller, 1998] George A Miller. WordNet: An electronic lex- ical database. MIT press, 1998.","title":"WordNet: An electronic lexical database","authors":"George A Miller","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Morris et al., 2020] John Morris, Jin Yong Yoo, and Yan- jun Qi. Textattack: Lessons learned in designing python frameworks for nlp. In Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS), pages 126- 131, 2020.","title":"Textattack: Lessons learned in designing python frameworks for nlp","authors":"John Morris, Jin Yong Yoo, Yanjun Qi","year":"2020","venue":"Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pang and Lee, 2005] Bo Pang and Lillian Lee. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting of the Association for Computational Lin- guistics (ACL'05), pages 115-124, Ann Arbor, Michigan, June 2005. Association for Computational Linguistics.","title":"Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales","authors":"Lee ; Pang, Bo Pang, Lillian Lee","year":"2005","venue":"Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Qiu et al., 2022] Shilin Qiu, Qihe Liu, Shijie Zhou, and Wen Huang. Adversarial attack and defense technologies in natural language processing: A survey. Neurocomput- ing, 492:278-307, 2022.","title":"Adversarial attack and defense technologies in natural language processing: A survey","authors":"Shilin Qiu, Qihe Liu, Shijie Zhou, Wen Huang","year":"2022","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ren et al., 2019] Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1085-1097, 2019.","title":"Generating natural language adversarial examples through probability weighted word saliency","authors":"Ren","year":"2019","venue":"Proceedings of the 57th annual meeting of the association for computational linguistics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sanh et al., 2019] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled ver- sion of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.","title":"Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter","authors":"Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schuster and Paliwal, 1997] Mike Schuster and Kuldip K Paliwal. Bidirectional recurrent neural networks. IEEE transactions on Signal Processing, 45(11):2673-2681, 1997.","title":"Bidirectional recurrent neural networks","authors":"Mike Schuster, Kuldip K Paliwal","year":"1997","venue":"IEEE transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Socher et al., 2013] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631-1642, 2013.","title":"Recursive deep models for semantic compositionality over a sentiment treebank","authors":"Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, Christopher Potts","year":"2013","venue":"Proceedings of the 2013 conference on empirical methods in natural language processing","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Stahlberg, 2020] Felix Stahlberg. Neural machine transla- tion: A review. Journal of Artificial Intelligence Research, 69:343-418, 2020.","title":"Neural machine translation: A review","authors":"Felix Stahlberg, Stahlberg","year":"2020","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Szegedy et al., 2013] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2022] Wenxuan Wang, Wenxiang Jiao, Yongchang Hao, Xing Wang, Shuming Shi, Zhaopeng Tu, and Michael Lyu. Understanding and improving sequence-to-sequence pretraining for neural machine translation. arXiv preprint arXiv:2203.08442, 2022.","title":"Understanding and improving sequence-to-sequence pretraining for neural machine translation","authors":"Wenxuan Wang, Wenxiang Jiao, Yongchang Hao, Xing Wang, Shuming Shi, Zhaopeng Tu, Michael Lyu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2023] Wenxuan Wang, Jen-tse Huang, Weibin Wu, Jianping Zhang, Yizhan Huang, Shuqing Li, Pin- jia He, and Michael Lyu. Mttm: Metamorphic testing for textual content moderation software. arXiv preprint arXiv:2302.05706, 2023.","title":"Mttm: Metamorphic testing for textual content moderation software","authors":"Wenxuan Wang, Jen-Tse Huang, Weibin Wu, Jianping Zhang, Yizhan Huang, Shuqing Li, Pinjia He, Michael Lyu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Williams et al., 2018] Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112-1122, New Orleans, Louisiana, June 2018. Associa- tion for Computational Linguistics.","title":"A broad-coverage challenge corpus for sentence understanding through inference","authors":"Williams","year":"2018","venue":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zang et al., 2020] Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. Word-level textual adversarial attacking as combinatorial optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6066-6080, Online, July 2020. Association for Compu- tational Linguistics.","title":"Word-level textual adversarial attacking as combinatorial optimization","authors":"Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, Maosong Sun","year":"2020","venue":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zeng et al., 2021] Guoyang Zeng, Fanchao Qi, Qianrui Zhou, Tingji Zhang, Zixian Ma, Bairu Hou, Yuan Zang, Zhiyuan Liu, and Maosong Sun. Openattack: An open- source textual adversarial attack toolkit. In Proceedings of the 59th Annual Meeting of the Association for Compu- tational Linguistics and the 11th International Joint Con- ference on Natural Language Processing: System Demon- strations, pages 363-371, 2021.","title":"Openattack: An opensource textual adversarial attack toolkit","authors":"Guoyang Zeng, Fanchao Qi, Qianrui Zhou, Tingji Zhang, Zixian Ma, Bairu Hou, Yuan Zang, Zhiyuan Liu, Maosong Sun","year":"2021","venue":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang et al., 2022] Jianping Zhang, Weibin Wu, Jen-tse Huang, Yizhan Huang, Wenxuan Wang, Yuxin Su, and Michael R Lyu. Improving adversarial transferability via neuron attribution-based attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14993-15002, 2022.","title":"Improving adversarial transferability via neuron attribution-based attacks","authors":"Jianping Zhang, Weibin Wu, Jen-Tse Huang, Yizhan Huang, Wenxuan Wang, Yuxin Su, Michael R Lyu","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2023a] Jianping Zhang, Jen-tse Huang, Wenx- uan Wang, Yichen Li, Weibin Wu, Xiaosen Wang, Yuxin Su, and Michael R Lyu. Improving the transferability of adversarial samples by path-augmented method. arXiv preprint arXiv:2303.15735, 2023.","title":"Improving the transferability of adversarial samples by path-augmented method","authors":"Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang et al., 2023b] Jianping Zhang, Yizhan Huang, Weibin Wu, and Michael R Lyu. Transferable adversar- ial attacks on vision transformers with token gradient regularization. arXiv preprint arXiv:2303.15754, 2023.","title":"Transferable adversarial attacks on vision transformers with token gradient regularization","authors":"Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}