{"type":"ScholarlyArticle","title":"Robust Interpretable Text Classification against Spurious Correlations Using AND-rules with Negation","authors":[{"name":"Rohan Kumar Yadav","slug":"rohan-kumar-yadav","given":"Rohan Kumar","family":"Yadav","affiliation":"Centre for Artificial Intelligence Research , University of Agder , 4879 , Grimstad , Norway"},{"name":"Lei Jiao","slug":"lei-jiao","given":"Lei","family":"Jiao","affiliation":"Centre for Artificial Intelligence Research , University of Agder , 4879 , Grimstad , Norway"},{"name":"Ole-Christoffer Granmo","slug":"ole-christoffer-granmo","given":"Ole-Christoffer","family":"Granmo","affiliation":"Centre for Artificial Intelligence Research , University of Agder , 4879 , Grimstad , Norway"},{"name":"Morten Goodwin","slug":"morten-goodwin","given":"Morten","family":"Goodwin","affiliation":"Centre for Artificial Intelligence Research , University of Agder , 4879 , Grimstad , Norway"}],"abstract":"The state-of-the-art natural language processing models have raised the bar for excellent performance on a variety of tasks in recent years. However, concerns are rising over their primitive sensitivity to distribution biases that reside in the training and testing data. This issue hugely impacts the performance of the models when exposed to out-of-distribution and counterfactual data. The root cause seems to be that many machine learning models are prone to learn the shortcuts, modelling simple correlations rather than more fundamental and general relationships. As a result, such text classifiers tend to perform poorly when a human makes minor modifications to the data, which raises questions regarding their robustness. In this paper, we employ a rule-based architecture called Tsetlin Machine (TM) that learns both simple and complex correlations by ANDing features and their negations. As such, it generates explainable AND-rules using negated and non-negated reasoning. Here, we explore how non-negated reasoning can be more prone to distribution biases than negated reasoning. We further leverage this finding by adapting the TM architecture to mainly perform negated reasoning using the specificity parameter s. As a result, the AND-rules becomes robust to spurious correlations and can also correctly predict counterfactual data. Our empirical investigation of the model's robustness uses the specificity s to control the degree of negated reasoning. Experiments on publicly available Counterfactually-Augmented Data demonstrate that the negated clauses are robust to spurious correlations and outperform Naive Bayes, SVM, and Bi-LSTM by up to 20 %, and  ELMo by almost 6 % on counterfactual test data.","doi":"10.24963/ijcai.2022/616","url":"https://ijcai.org/proceedings/2022/616","pdf":"https://ijcai.org/proceedings/2022/0616.pdf","markdown":"https://ijcai.org/proceedings/2022/616.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/616","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":616,"pages":{"first":4439,"last":4446},"keywords":["Natural Language Processing: Text Classification","Machine Learning: Robustness","Natural Language Processing: Interpretability and Analysis of Models for NLP","Natural Language Processing: Applications"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986983","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985126","length_sec":723,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Abeyrathna et al., 2021] Kuruge Darshana Abeyrathna, Bi- mal Bhattarai, Morten Goodwin, Saeed Rahimi Gorji, Ole- Christoffer Granmo, Lei Jiao, Rupsa Saha, and Rohan K. Yadav. Massively parallel and asynchronous tsetlin ma- chine architecture supporting almost constant-time scal- ing. In ICML, pages 10-20. PMLR, 2021.","title":"Massively parallel and asynchronous tsetlin ma-chine architecture supporting almost constant-time scaling","authors":"Abeyrathna","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In ICLR, California, USA, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyung, Hyun Cho, Yoshua Bengio","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhattarai et al., 2020] Bimal Bhattarai, Ole-Christoffer Granmo, and Lei Jiao. Measuring the novelty of natural language text using the conjunctive clauses of a tsetlin machine text classifier, 2020.","title":"Measuring the novelty of natural language text using the conjunctive clauses of a tsetlin machine text classifier","authors":"Bimal Bhattarai, Ole-Christoffer Granmo, Lei Jiao","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bhattarai et al., 2022] Bimal Bhattarai, Ole-Christoffer Granmo, and Lei Jiao. Word-level human interpretable scoring mechanism for novel text detection using tsetlin machines. Applied Intelligence, 2022.","title":"Word-level human interpretable scoring mechanism for novel text detection using tsetlin machines","authors":"Bimal Bhattarai, Ole-Christoffer Granmo, Lei Jiao","year":"2022","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Brunner et al., 2020] Gino Brunner, Yang Liu, Damian Pas- cual, Oliver Richter, Massimiliano Ciaramita, and Roger Wattenhofer. On identifiability in transformers. In ICLR, Addis Ababa, Ethiopia, 2020.","title":"On identifiability in transformers","authors":"Gino Brunner, Yang Liu, Damian Pascual, Oliver Richter, Massimiliano Ciaramita, Roger Wattenhofer","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Getoor and Taskar, 2007] L. Getoor and B. Taskar. Intro- duction to statistical relational learning. In MITPress, 2007.","title":"Introduction to statistical relational learning","authors":"L Getoor, B Taskar","year":"2007","venue":"MITPress","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goguen, 1973] J. Goguen. Zadeh l. a.. fuzzy sets. informa- tion and control , vol. 8 (1965), pp. 338-353. zadeh l. a.. similarity relations and fuzzy orderings. information sci- ences , vol. 3 (1971), pp. 177-200. Journal of Symbolic Logic, 38:656-657, 1973.","title":"fuzzy sets","authors":"J Goguen, Zadeh L. A","year":"1965","venue":"Journal of Symbolic Logic","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Granmo et al., 2019] Ole-Christoffer Granmo, Sondre Glimsdal, Lei Jiao, Morten Goodwin, Christian W. Omlin, and Geir Thore Berge. The Convolutional Tsetlin Machine, 2019.","title":"The Convolutional Tsetlin Machine","authors":"Ole-Christoffer Granmo, Sondre Glimsdal, Lei Jiao, Morten Goodwin, Christian W Omlin, Geir Thore, Berge","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Granmo, 2018] Ole-Christoffer Granmo. The tsetlin ma- chine -a game theoretic bandit driven approach to optimal pattern recognition with propositional logic, 2018.","title":"The tsetlin machine -a game theoretic bandit driven approach to optimal pattern recognition with propositional logic","authors":"Ole-Christoffer Granmo","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jia and Liang, 2017] Robin Jia and Percy Liang. Adversar- ial examples for evaluating reading comprehension sys- tems. In EMNLP, pages 2021-2031, Copenhagen, Den- mark, 2017. ACL.","title":"Adversarial examples for evaluating reading comprehension systems","authors":"Liang Jia, Robin Jia, Percy Liang","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kaufman et al., 2012] Shachar Kaufman, Saharon Rosset, Claudia Perlich, and Ori Stitelman. Leakage in data min- ing: Formulation, detection, and avoidance. ACM Trans. Knowl. Discov. Data, 6, 2012.","title":"Leakage in data mining: Formulation, detection, and avoidance","authors":"Shachar Kaufman, Saharon Rosset, Claudia Perlich, Ori Stitelman","year":"2012","venue":"ACM Trans. Knowl. Discov. Data","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kaushik et al., 2020] Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. Learning the difference that makes a dif- ference with counterfactually-augmented data. In ICLR, Online, 2020.","title":"Learning the difference that makes a difference with counterfactually-augmented data","authors":"Kaushik","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kim et al., 2016] Byeongchang Kim, Seonghan Ryu, and G. Lee. Two-stage multi-intent detection for spoken lan- guage understanding. Multimedia Tools and Applications, 76:11377-11390, 2016.","title":"Two-stage multi-intent detection for spoken language understanding","authors":"Kim","year":"2016","venue":"Multimedia Tools and Applications","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Krizhevsky et al., 2012] A. Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep con- volutional neural networks. Communications of the ACM, 60:84 -90, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"A Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lei et al., 2021] Jie Lei, Tousif Rahman, Rishad Shafik, Adrian Wheeldon, Alex Yakovlev, Ole-Christoffer Granmo, Fahim Kawsar, and Akhil Mathur. Low-Power Audio Keyword Spotting using Tsetlin Machines, 2021.","title":"Low-Power Audio Keyword Spotting using Tsetlin Machines","authors":"Lei","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lu et al., 2020] Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. Gender Bias in Neural Natural Language Processing, pages 189-202. Springer International Publishing, 2020.","title":"Gender Bias in Neural Natural Language Processing","authors":"Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, Anupam Datta","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ni et al., 2019] Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly- labeled reviews and fine-grained aspects. In EMNLP- IJCNLP, pages 188-197, Hong Kong, China, 2019.","title":"Justifying recommendations using distantlylabeled reviews and fine-grained aspects","authors":"Ni","year":"2019","venue":"EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"ACL.","title":null,"authors":"Acl","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Peters et al., 2018] Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In NAACL: Human Language Technolo- gies, pages 2227-2237, New Orleans, Louisiana, 2018.","title":"Deep contextualized word representations","authors":"Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer","year":"2018","venue":"NAACL: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[ACL. Raedt and Kimmig, 2015] L. D. Raedt and Angelika Kim- mig. Probabilistic (logic) programming concepts. Ma- chine Learning, 100:5-47, 2015.","title":"Probabilistic (logic) programming concepts","authors":"L D Raedt, Angelika Kimmig","year":"2015","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rosenthal et al., 2017] Sara Rosenthal, Noura Farra, and Preslav Nakov. SemEval-2017 task 4: Sentiment analy- sis in Twitter. In Proceedings of the 11th SemEval-2017, pages 502-518, Vancouver, Canada, 2017. ACL.","title":"SemEval-2017 task 4: Sentiment analysis in Twitter","authors":"Sara Rosenthal, Noura Farra, Preslav Nakov","year":"2017","venue":"Proceedings of the 11th SemEval-2017","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rudin, 2018] C. Rudin. Stop explaining black box machine learning models for high stakes decisions and use inter- pretable models instead. Nature Machine Intelligence, 1:206-215, 2018.","title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","authors":"C Rudin","year":"2018","venue":"Nature Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sauer and Geiger, 2021] Axel Sauer and Andreas Geiger. Counterfactual generative networks. In ICLR, Online, 2021.","title":"Counterfactual generative networks","authors":"Geiger Sauer, Axel Sauer, Andreas Geiger","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Serrano and Smith, 2019] Sofia Serrano and Noah A. Smith. Is attention interpretable? In Proceedings of ACL, pages 2931-2951, Florence, Italy, 2019. ACL.","title":"Is attention interpretable?","authors":"Smith ; Serrano, Sofia Serrano, Noah A Smith","year":"2019","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang and Culotta, 2020] Zhao Wang and Aron Culotta. Identifying spurious correlations for robust text classifica- tion. In Findings of the EMNLP 2020, pages 3431-3440, Online, 2020. ACL.","title":"Identifying spurious correlations for robust text classification","authors":"Wang, Culotta ; ] Zhao Wang, Aron Culotta","year":"2020","venue":"Findings of the EMNLP 2020","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wulczyn et al., 2017] Ellery Wulczyn, Nithum Thain, and Lucas Dixon. Ex machina: Personal attacks seen at scale. In International Conference on World Wide Web, page 1391-1399, Perth, Australia, 2017. WWW.","title":"Ex machina: Personal attacks seen at scale","authors":"Wulczyn","year":"2017","venue":"International Conference on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yadav et al., 2021a] Rohan Kumar Yadav, Lei Jiao, Ole- Christoffer Granmo, and Morten Goodwin. Enhancing interpretable clauses semantically using pretrained word representation. In BlackboxNLP, pages 265-274, Punta Cana, Dominican Republic, 2021. ACL.","title":"Enhancing interpretable clauses semantically using pretrained word representation","authors":"Yadav","year":"2021","venue":"BlackboxNLP","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yadav et al., 2021b] Rohan Kumar Yadav, Lei Jiao, Ole- Christoffer Granmo, and Morten Goodwin. Human-Level Interpretable Learning for Aspect-Based Sentiment Anal- ysis. In Proceedings of AAAI, Vancouver, Canada. AAAI, 2021.","title":"Human-Level Interpretable Learning for Aspect-Based Sentiment Analysis","authors":"Yadav","year":"2021","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}