{"type":"ScholarlyArticle","title":"NELLIE: A Neuro-Symbolic Inference Engine for Grounded, Compositional, and Explainable Reasoning","authors":[{"name":"Nathaniel Weir","slug":"nathaniel-weir","given":"Nathaniel","family":"Weir","affiliation":"Johns Hopkins University , Baltimore , MD , USA"},{"name":"Peter Clark","slug":"peter-clark","given":"Peter","family":"Clark","affiliation":"Allen Institute for AI , Seattle , WA , USA"},{"name":"Benjamin Van Durme","slug":"benjamin-van-durme","given":"Benjamin","family":"Van Durme","affiliation":"Johns Hopkins University , Baltimore , MD , USA"}],"abstract":"Our goal is to develop a modern approach to answering questions via systematic reasoning where answers are supported by human interpretable proof trees grounded in an NL corpus of facts. Such a system would help alleviate the challenges of interpretability and hallucination with modern LMs, and the lack of grounding of current explanation methods (e.g., Chain-of-Thought). This paper proposes a new take on Prolog-based inference engines, where we replace handcrafted rules with a combination of neural language modeling, guided generation, and semiparametric dense retrieval. Our implementation, NELLIE, is the first system to demonstrate fully interpretable, end-to-end grounded QA as entailment tree proof search, going beyond earlier work explaining known-to-be-true facts from text. In experiments, NELLIE outperforms a similar-sized state-of-the-art reasoner while producing knowledge-grounded explanations. We also find NELLIE can exploit both semi-structured and NL text corpora to guide reasoning. Together these suggest a new way to jointly reap the benefits of both modern neural methods and traditional symbolic reasoning.","doi":"10.24963/ijcai.2024/399","url":"https://ijcai.org/proceedings/2024/399","pdf":"https://ijcai.org/proceedings/2024/0399.pdf","markdown":"https://ijcai.org/proceedings/2024/399.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/399","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Knowledge Representation and Reasoning","track":"Main Track","number":399,"pages":{"first":3602,"last":3612},"keywords":["Natural Language Processing: NLP: Question answering","Knowledge Representation and Reasoning: KRR: Reasoning about knowledge and belief","Knowledge Representation and Reasoning: KRR: Automated reasoning and theorem proving","Search: S: Other"],"videos":[],"references":[{"position":1,"raw":"[Andreas et al., 2016] Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Neural module networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.","title":"Neural module networks","authors":"Andreas","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arabshahi et al., 2021a] Forough Arabshahi, Jennifer Lee, Antoine Bosselut, Yejin Choi, and Tom Mitchell. Con- versational multi-hop reasoning with neural commonsense knowledge and symbolic logic rules. In Proceedings of EMNLP, 2021.","title":"Conversational multi-hop reasoning with neural commonsense knowledge and symbolic logic rules","authors":"Arabshahi","year":"2021","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Arabshahi et al., 2021b] Forough Arabshahi, Jennifer Lee, Mikayla Gawarecki, Kathryn Mazaitis, Amos Azaria, and Tom Mitchell. Conversational neuro-symbolic common- sense reasoning. In Proceedings of AAAI, 2021.","title":"Conversational neuro-symbolic commonsense reasoning","authors":"Arabshahi","year":"2021","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bos and Markert, 2005] Johan Bos and Katja Markert. Recognising textual entailment with logical inference. In Proceedings of HLT and EMNLP, 2005.","title":"Recognising textual entailment with logical inference","authors":"Markert Bos, Johan Bos, Katja Markert","year":"2005","venue":"Proceedings of HLT and EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bosselut et al., 2021] Antoine Bosselut, Ronan Le Bras, and Yejin Choi. Dynamic neuro-symbolic knowledge graph construction for zero-shot commonsense question answer- ing. AAAI, 2021.","title":"Dynamic neuro-symbolic knowledge graph construction for zero-shot commonsense question answering","authors":"Bosselut","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bostrom et al., 2022] Kaj Bostrom, Zayne Sprague, Swarat Chaudhuri, and Greg Durrett. Natural language deduction through search over statement compositions. In Findings of ACL: EMNLP 2022, 2022.","title":"Natural language deduction through search over statement compositions","authors":"Bostrom","year":"2022","venue":"Findings of ACL: EMNLP 2022","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Clark et al., 2018] Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answer- ing? try arc, the ai2 reasoning challenge. arXiv:1803.05457, 2018.","title":"Think you have solved question answering? try arc, the ai2 reasoning challenge","authors":"Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, Oyvind Tafjord","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Clark et al., 2021] Peter Clark, Oyvind Tafjord, and Kyle Richardson. Transformers as soft reasoners over language. In Proceedings of IJCAI, 2021.","title":"Transformers as soft reasoners over language","authors":"Clark","year":"2021","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/537"},{"position":9,"raw":"[Creswell and Shanahan, 2022] Antonia Creswell and Mur- ray Shanahan. Faithful reasoning using large language models. arXiv preprint arXiv:2208.14271, 2022.","title":"Faithful reasoning using large language models","authors":"Antonia Creswell, Murray Shanahan","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dalvi et al., 2021] Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pi- patanangkura, and Peter Clark. Explaining answers with entailment trees. In Proceedings of EMNLP, 2021.","title":"Explaining answers with entailment trees","authors":"Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, Peter Clark","year":"2021","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dalvi Mishra et al., 2022] Bhavana Dalvi Mishra, Oyvind Tafjord, and Peter Clark. Towards teachable reasoning systems: Using a dynamic memory of user feedback for continual system improvement. In Proceedings of EMNLP, 2022.","title":"Towards teachable reasoning systems: Using a dynamic memory of user feedback for continual system improvement","authors":"Dalvi Mishra","year":"2022","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Das et al., 2021] Rajarshi Das, Manzil Zaheer, Dung Thai, Ameya Godbole, Ethan Perez, Jay Yoon Lee, Lizhen Tan, Lazaros Polymenakos, and Andrew McCallum. Case-based reasoning for natural language queries over knowledge bases. In Proceedings of EMNLP, 2021.","title":"Case-based reasoning for natural language queries over knowledge bases","authors":"Rajarshi Das, Manzil Zaheer, Dung Thai, Ameya Godbole, Ethan Perez, Jay Yoon Lee, Lizhen Tan, Lazaros Polymenakos, Andrew Mccallum","year":"2021","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Demszky et al., 2018] Dorottya Demszky, Kelvin Guu, and Percy Liang. Transforming question answering datasets into natural language inference datasets. arXiv:1809.02922, 2018.","title":"Transforming question answering datasets into natural language inference datasets","authors":"Dorottya Demszky, Kelvin Guu, Percy Liang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Green et al., 1961] Bert F. Green, Alice K. Wolf, Carol Chomsky, and Kenneth Laughery. Baseball: An automatic question-answerer. 1961.","title":"Baseball: An automatic question-answerer","authors":"Bert F Green, Alice K Wolf, Carol Chomsky, Kenneth Laughery","year":"1961","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gupta et al., 2020] Nitish Gupta, Kevin Lin, Dan Roth, Sameer Singh, and Matt Gardner. Neural module networks for reasoning over text. In International Conference on Learning Representations, 2020.","title":"Neural module networks for reasoning over text","authors":"Nitish Gupta, Kevin Lin, Dan Roth, Sameer Singh, Matt Gardner","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[He et al., 2022] Hangfeng He, Hongming Zhang, and Dan Roth. Rethinking with retrieval: Faithful large language model inference. arXiv preprint arXiv:2301.00303, 2022.","title":"Rethinking with retrieval: Faithful large language model inference","authors":"He","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Holtzman et al., 2019] Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In ICLR, 2019.","title":"The curious case of neural text degeneration","authors":"Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hong et al., 2022] Ruixin Hong, Hongming Zhang, Xintong Yu, and Changshui Zhang. METGEN: A module-based en- tailment tree generation framework for answer explanation. In Findings of ACL: NAACL 2022, 2022.","title":"METGEN: A module-based entailment tree generation framework for answer explanation","authors":"Ruixin Hong, Hongming Zhang, Xintong Yu, Changshui Zhang","year":"2022","venue":"Findings of ACL: NAACL 2022","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jackson, 1986] P Jackson. Introduction to expert systems. 1986.","title":"Introduction to expert systems","authors":"Jackson","year":"1986","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Jansen and Ustalov, 2019] Peter Jansen and Dmitry Ustalov. TextGraphs 2019 shared task on multi-hop inference for explanation regeneration. In Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13), 2019.","title":"TextGraphs 2019 shared task on multi-hop inference for explanation regeneration","authors":"Peter Jansen, Dmitry Ustalov","year":"2019","venue":"Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Jhamtani and Clark, 2020] Harsh Jhamtani and Peter Clark. Learning to explain: Datasets and models for identifying valid reasoning chains in multihop question-answering. In Proceedings of EMNLP (EMNLP), 2020.","title":"Harsh Jhamtani and Peter Clark. Learning to explain: Datasets and models for identifying valid reasoning chains in multihop question-answering","authors":"Clark Jhamtani","year":"2020","venue":"Proceedings of EMNLP (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ji et al., 2022] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 2022.","title":"Survey of hallucination in natural language generation","authors":"Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, Pascale Fung","year":"2022","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Johnson et al., 2019] Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 2019.","title":"Billion-scale similarity search with GPUs","authors":"Jeff Johnson, Matthijs Douze, Hervé Jégou","year":"2019","venue":"IEEE Transactions on Big Data","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Jones et al., 2000] Karen Sparck Jones, Steve Walker, and Stephen E. Robertson. A probabilistic model of information retrieval: development and comparative experiments -part 1. Inf. Process. Manag., 2000.","title":"A probabilistic model of information retrieval: development and comparative experiments -part 1","authors":"Karen Sparck Jones, Steve Walker, Stephen E Robertson","year":"2000","venue":"Inf. Process. Manag","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Jung et al., 2022] Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, and Yejin Choi. Maieutic prompting: Logically consistent reasoning with recursive explanations. In Proceedings of EMNLP, 2022.","title":"Maieutic prompting: Logically consistent reasoning with recursive explanations","authors":"Jung","year":"2022","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Kalyanpur et al., 2021] Aditya Kalyanpur, Tom Breloff, David Ferrucci, Adam Lally, and John Jantos. Braid: Weav- ing symbolic and neural knowledge into coherent logical explanations. AAAI, 2021.","title":"Braid: Weaving symbolic and neural knowledge into coherent logical explanations","authors":"Aditya Kalyanpur, Tom Breloff, David Ferrucci, Adam Lally, John Jantos","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Kautz and Selman, 1999] Henry Kautz and Bart Selman. Unifying sat-based and graph-based planning. In IJCAI, 1999.","title":"Unifying sat-based and graph-based planning","authors":"Henry Kautz, Bart Selman","year":"1999","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/1999/47"},{"position":28,"raw":"[Kautz et al., 1992] Henry A Kautz, Bart Selman, et al. Plan- ning as satisfiability. In ECAI, 1992.","title":"Planning as satisfiability","authors":"Bart Henry A Kautz, Selman","year":"1992","venue":"ECAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Kazemi et al., 2023] Mehran Kazemi, Najoung Kim, Deepti Bhatia, Xin Xu, and Deepak Ramachandran. LAMBADA: Backward chaining for automated reasoning in natural lan- guage. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023.","title":"LAMBADA: Backward chaining for automated reasoning in natural language","authors":"Mehran Kazemi, Najoung Kim, Deepti Bhatia, Xin Xu, Deepak Ramachandran","year":"2023","venue":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Khashabi et al., 2019] Daniel Khashabi, Erfan Sadeqi Azer, Tushar Khot, Ashish Sabharwal, and Dan Roth. On the possibilities and limitations of multi-hop reasoning under linguistic imperfections. arXiv:1901.02522, 2019.","title":"On the possibilities and limitations of multi-hop reasoning under linguistic imperfections","authors":"Daniel Khashabi, Erfan Sadeqi Azer, Tushar Khot, Ashish Sabharwal, Dan Roth","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Khot et al., 2017] Tushar Khot, Ashish Sabharwal, and Peter Clark. Answering complex questions using open informa- tion extraction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2017.","title":"Answering complex questions using open information extraction","authors":"Tushar Khot, Ashish Sabharwal, Peter Clark","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Khot et al., 2020] Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. Qasc: A dataset for question answering via sentence composition. In Proceedings of AAAI, 2020.","title":"Qasc: A dataset for question answering via sentence composition","authors":"Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, Ashish Sabharwal","year":"2020","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Khot et al., 2021] Tushar Khot, Daniel Khashabi, Kyle Richardson, Peter Clark, and Ashish Sabharwal. Text mod- ular networks: Learning to decompose tasks in the language of existing models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, 2021.","title":"Text modular networks: Learning to decompose tasks in the language of existing models","authors":"Tushar Khot, Daniel Khashabi, Kyle Richardson, Peter Clark, Ashish Sabharwal","year":"2021","venue":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Khot et al., 2022] Tushar Khot, Harsh Trivedi, Matthew Fin- layson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. Decomposed prompting: A modular approach for solving complex tasks. In The Eleventh International Conference on Learning Representations, 2022.","title":"Decomposed prompting: A modular approach for solving complex tasks","authors":"Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, Ashish Sabharwal","year":"2022","venue":"The Eleventh International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Kojima et al., 2022] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwa- sawa. Large language models are zero-shot reasoners. arXiv:2205.11916, 2022.","title":"Large language models are zero-shot reasoners","authors":"Takeshi Kojima, Shane Shixiang, Machel Gu, Yutaka Reid, Yusuke Matsuo, Iwasawa","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Kundu et al., 2019] Souvik Kundu, Tushar Khot, Ashish Sabharwal, and Peter Clark. Exploiting explicit paths for multi-hop reading comprehension. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.","title":"Exploiting explicit paths for multi-hop reading comprehension","authors":"Souvik Kundu, Tushar Khot, Ashish Sabharwal, Peter Clark","year":"2019","venue":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Lewis et al., 2019] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natu- ral language generation, translation, and comprehension. arXiv:1910.13461, 2019.","title":"Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","authors":"Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, Luke Zettlemoyer","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Lyu et al., 2023] Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. Faithful chain-of-thought reasoning. In ACL. Association for Computational Linguistics, 2023.","title":"Faithful chain-of-thought reasoning","authors":"Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, Chris Callison-Burch","year":"2023","venue":"ACL. Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Lyu et al., 2024] Qing Lyu, Marianna Apidianaki, and Chris Callison-Burch. Towards Faithful Model Explanation in NLP: A Survey. Computational Linguistics, 2024.","title":"Towards Faithful Model Explanation in NLP: A Survey","authors":"Qing Lyu, Marianna Apidianaki, Chris Callison-Burch","year":"2024","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Madaan et al., 2021] Aman Madaan, Niket Tandon, Dheeraj Rajagopal, Peter Clark, Yiming Yang, and Eduard Hovy. Think about it! improving defeasible reasoning by first modeling the question scenario. In Proceedings of EMNLP, 2021.","title":"Think about it! improving defeasible reasoning by first modeling the question scenario","authors":"Aman Madaan, Niket Tandon, Dheeraj Rajagopal, Peter Clark, Yiming Yang, Eduard Hovy","year":"2021","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Manhaeve et al., 2018] Robin Manhaeve, Sebastijan Duman- cic, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. Deepproblog: Neural probabilistic logic pro- gramming. NIPS, 2018.","title":"Deepproblog: Neural probabilistic logic programming","authors":"Manhaeve","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[McCarthy, 1959] John McCarthy. Programs with common sense, 1959.","title":"Programs with common sense","authors":"John Mccarthy","year":"1959","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Metaxiotis et al., 2002] Kostas S Metaxiotis, Dimitris Ask- ounis, and John Psarras. Expert systems in production planning and scheduling: A state-of-the-art survey. Journal of Intelligent Manufacturing, 2002.","title":"Expert systems in production planning and scheduling: A state-of-the-art survey","authors":"Dimitris Kostas S Metaxiotis, John Askounis, Psarras","year":"2002","venue":"Journal of Intelligent Manufacturing","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Mihaylov et al., 2018] Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answer- ing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018.","title":"Can a suit of armor conduct electricity? a new dataset for open book question answering","authors":"Todor Mihaylov, Peter Clark, Tushar Khot, Ashish Sabharwal","year":"2018","venue":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Musen and Van der Lei, 1988] Mark A Musen and Johan Van der Lei. Of brittleness and bottlenecks: Challenges in the creation of pattern-recognition and expert-system models. In Machine Intelligence and Pattern Recognition. 1988.","title":"Of brittleness and bottlenecks: Challenges in the creation of pattern-recognition and expert-system models","authors":"Van Musen, ; Der Lei, A Mark, Johan Musen, Van Der Lei","year":"1988","venue":"Machine Intelligence and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Olausson et al., 2023] Theo Olausson, Alex Gu, Ben Lipkin, Cedegao Zhang, Armando Solar-Lezama, Joshua Tenen- baum, and Roger Levy. LINC: A neurosymbolic approach for logical reasoning by combining language models with first-order logic provers. In Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Pro- cessing, 2023.","title":"LINC: A neurosymbolic approach for logical reasoning by combining language models with first-order logic provers","authors":"Theo Olausson, Alex Gu, Ben Lipkin, Cedegao Zhang, Armando Solar-Lezama, Joshua Tenenbaum, Roger Levy","year":"2023","venue":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Pan et al., 2019] Zhufeng Pan, Kun Bai, Yan Wang, Lian- qiang Zhou, and Xiaojiang Liu. Improving open-domain di- alogue systems via multi-turn incomplete utterance restora- tion. In Proceedings of EMNLP-IJCNLP, 2019.","title":"Improving open-domain dialogue systems via multi-turn incomplete utterance restoration","authors":"Pan","year":"2019","venue":"Proceedings of EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Pan et al., 2023] Liangming Pan, Alon Albalak, Xinyi Wang, and William Wang. Logic-LM: Empowering large language models with symbolic solvers for faithful logical reasoning. In Findings of ACL: EMNLP 2023, 2023.","title":"Logic-LM: Empowering large language models with symbolic solvers for faithful logical reasoning","authors":"Pan","year":"2023","venue":"Findings of ACL: EMNLP 2023","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Picco et al., 2021] Gabriele Picco, Thanh Lam Hoang, Marco Luca Sbodio, and Vanessa Lopez. Neural unification for logic reasoning over natural language. In Findings of ACL: EMNLP 2021, 2021.","title":"Neural unification for logic reasoning over natural language","authors":"Gabriele Picco, Thanh Lam Hoang, Marco Luca Sbodio, Vanessa Lopez","year":"2021","venue":"Findings of ACL: EMNLP 2021","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Polu and Sutskever, 2020] Stanislas Polu and Ilya Sutskever. Generative language modeling for automated theorem prov- ing. ArXiv, 2020.","title":"Generative language modeling for automated theorem proving","authors":"Sutskever Polu, Stanislas Polu, Ilya Sutskever","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Raffel et al., 2020] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR, 2020.","title":"Exploring the limits of transfer learning with a unified text-to-text transformer","authors":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu","year":"2020","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Riazanov and Voronkov, 2002] Alexandre Riazanov and An- drei Voronkov. The design and implementation of vampire. AI communications, 2002.","title":"Alexandre Riazanov and Andrei Voronkov. The design and implementation of vampire","authors":"Voronkov Riazanov","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Ribeiro et al., 2022] Danilo Neves Ribeiro, Shen Wang, Xi- aofei Ma, Rui Dong, Xiaokai Wei, Henry Zhu, Xinchi Chen, Zhiheng Huang, Peng Xu, Andrew O. Arnold, and Dan Roth. Entailment tree explanations via iterative retrieval- generation reasoner. In NAACL-HLT, 2022.","title":"Entailment tree explanations via iterative retrievalgeneration reasoner","authors":"Danilo Neves Ribeiro, Shen Wang, Xiaofei Ma, Rui Dong, Xiaokai Wei, Henry Zhu, Xinchi Chen, Zhiheng Huang, Peng Xu, Andrew O Arnold, Dan Roth","year":"2022","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Rocktäschel and Riedel, 2017] Tim Rocktäschel and Sebas- tian Riedel. End-to-end differentiable proving. Advances in neural information processing systems, 2017.","title":"Tim Rocktäschel and Sebastian Riedel. End-to-end differentiable proving","authors":"Riedel Rocktäschel","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Russell and Norvig, 2010] Stuart Russell and Peter Norvig. Artificial intelligence: a modern approach. 2010.","title":"Artificial intelligence: a modern approach","authors":"Stuart Russell, Peter Norvig","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Saha et al., 2023a] Swarnadeep Saha, Omer Levy, Asli Ce- likyilmaz, Mohit Bansal, Jason Weston, and Xian Li. Branch-solve-merge improves large language model evalu- ation and generation, 2023.","title":"Branch-solve-merge improves large language model evaluation and generation","authors":"Saha","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Saha et al., 2023b] Swarnadeep Saha, Xinyan Yu, Mohit Bansal, Ramakanth Pasunuru, and Asli Celikyilmaz. MUR- MUR: Modular multi-step reasoning for semi-structured data-to-text generation. In Findings of ACL: ACL 2023, 2023.","title":"MUR-MUR: Modular multi-step reasoning for semi-structured data-to-text generation","authors":"Saha","year":"2023","venue":"Findings of ACL: ACL 2023","doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Schank, 1983] Roger C Schank. Dynamic memory: A theory of reminding and learning in computers and people. 1983.","title":"Dynamic memory: A theory of reminding and learning in computers and people","authors":"C Roger, Schank","year":"1983","venue":null,"doi":null,"url":null,"internal_url":null},{"position":59,"raw":"[Schubert, 2015] Lenhart Schubert. Semantic representation. Proceedings of AAAI, 29(1), Mar. 2015.","title":"Semantic representation","authors":"Lenhart Schubert","year":"2015","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":60,"raw":"[Sessa, 2002] Maria I Sessa. Approximate reasoning by similarity-based sld resolution. Theoretical computer sci- ence, 2002.","title":"Approximate reasoning by similarity-based sld resolution","authors":"Maria I Sessa","year":"2002","venue":"Theoretical computer science","doi":null,"url":null,"internal_url":null},{"position":61,"raw":"[Sprague et al., 2022] Zayne Sprague, Kaj Bostrom, Swarat Chaudhuri, and Greg Durrett. Natural language deduction with incomplete information. In Proceedings of EMNLP, 2022.","title":"Natural language deduction with incomplete information","authors":"Zayne Sprague, Kaj Bostrom, Swarat Chaudhuri, Greg Durrett","year":"2022","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":62,"raw":"[Stern et al., 2010] Roni Stern, Tamar Kulberis, Ariel Felner, and Robert Holte. Using lookaheads with optimal best-first search. In Proceedings of AAAI, 2010.","title":"Using lookaheads with optimal best-first search","authors":"Roni Stern, Tamar Kulberis, Ariel Felner, Robert Holte","year":"2010","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":63,"raw":"[Tafjord et al., 2021] Oyvind Tafjord, Bhavana Dalvi, and Pe- ter Clark. ProofWriter: Generating implications, proofs, and abductive statements over natural language. In Findings of ACL: ACL-IJCNLP 2021, 2021.","title":"ProofWriter: Generating implications, proofs, and abductive statements over natural language","authors":"Oyvind Tafjord, Bhavana Dalvi, Peter Clark","year":"2021","venue":"Findings of ACL: ACL-IJCNLP 2021","doi":null,"url":null,"internal_url":null},{"position":64,"raw":"[Tafjord et al., 2022] Oyvind Tafjord, Bhavana Dalvi Mishra, and Peter Clark. Entailer: Answering questions with faith- ful and truthful chains of reasoning. In Proceedings of EMNLP, 2022.","title":"Entailer: Answering questions with faithful and truthful chains of reasoning","authors":"Oyvind Tafjord, Bhavana Dalvi Mishra, Peter Clark","year":"2022","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":65,"raw":"[Thayaparan et al., 2021] Mokanarangan Thayaparan, Marco Valentino, and André Freitas. Explainable inference over grounding-abstract chains for science questions. In Find- ings of ACL: ACL-IJCNLP 2021, 2021.","title":"Explainable inference over grounding-abstract chains for science questions","authors":"Thayaparan","year":"2021","venue":"Findings of ACL: ACL-IJCNLP 2021","doi":null,"url":null,"internal_url":null},{"position":66,"raw":"[Thayaparan et al., 2022] Mokanarangan Thayaparan, Marco Valentino, Deborah Ferreira, Julia Rozanova, and André Freitas. Diff-explainer: Differentiable convex optimization for explainable multi-hop inference. TACL, 2022.","title":"Diff-explainer: Differentiable convex optimization for explainable multi-hop inference","authors":"Thayaparan","year":"2022","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":67,"raw":"[Valentino et al., 2022] Marco Valentino, Mokanarangan Thayaparan, and André Freitas. Case-based abductive natural language inference. In Proceedings of the 29th International Conference on Computational Linguistics, 2022.","title":"Case-based abductive natural language inference","authors":"Marco Valentino, Mokanarangan Thayaparan, André Freitas","year":"2022","venue":"Proceedings of the 29th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":68,"raw":"[Weber et al., 2019] Leon Weber, Pasquale Minervini, Jannes Münchmeyer, Ulf Leser, and Tim Rocktäschel. NLProlog: Reasoning with weak unification for question answering in natural language. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.","title":"NLProlog: Reasoning with weak unification for question answering in natural language","authors":"Leon Weber, Pasquale Minervini, Jannes Münchmeyer, Ulf Leser, Tim Rocktäschel","year":"2019","venue":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":69,"raw":"[Wei et al., 2022] Jason Wei, Xuezhi Wang, Dale Schuur- mans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large lan- guage models. arXiv:2201.11903, 2022.","title":"Chain of thought prompting elicits reasoning in large language models","authors":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, Denny Zhou","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":70,"raw":"[Welleck et al., 2022] Sean Welleck, Jiacheng Liu, Ximing Lu, Hannaneh Hajishirzi, and Yejin Choi. Naturalprover: Grounded mathematical proof generation with language models. arXiv:2205.12910, 2022.","title":"Naturalprover: Grounded mathematical proof generation with language models","authors":"Sean Welleck, Jiacheng Liu, Ximing Lu, Hannaneh Hajishirzi, Yejin Choi","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":71,"raw":"[Wong et al., 2023] Lionel Wong, Gabriel Grand, Alexan- der K. Lew, Noah D. Goodman, Vikash K. Mansinghka, Jacob Andreas, and Joshua B. Tenenbaum. From word models to world models: Translating from natural language to the probabilistic language of thought, 2023.","title":"From word models to world models: Translating from natural language to the probabilistic language of thought","authors":"Lionel Wong, Gabriel Grand, Alexander K Lew, Noah D Goodman, K Vikash, Jacob Mansinghka, Joshua B Andreas, Tenenbaum","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":72,"raw":"[Xia et al., 2024] Yu Xia, Rui Wang, Xu Liu, Mingyan Li, Tong Yu, Xiang Chen, Julian McAuley, and Shuai Li. Be- yond chain-of-thought: A survey of chain-of-x paradigms for llms, 2024.","title":"Beyond chain-of-thought: A survey of chain-of-x paradigms for llms","authors":"Yu Xia, Rui Wang, Xu Liu, Mingyan Li, Tong Yu, Xiang Chen, Julian Mcauley, Shuai Li","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":73,"raw":"[Xie et al., 2020] Zhengnan Xie, Sebastian Thiem, Jaycie Martin, Elizabeth Wainwright, Steven Marmorstein, and Peter Jansen. WorldTree v2: A corpus of science-domain structured explanations and inference patterns supporting multi-hop inference. In Proceedings of the Twelfth Lan- guage Resources and Evaluation Conference, 2020.","title":"WorldTree v2: A corpus of science-domain structured explanations and inference patterns supporting multi-hop inference","authors":"Xie","year":"2020","venue":"Proceedings of the Twelfth Language Resources and Evaluation Conference","doi":null,"url":null,"internal_url":null},{"position":74,"raw":"[Yadav et al., 2019] Vikas Yadav, Steven Bethard, and Mihai Surdeanu. Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering. In Proceedings of EMNLP-IJCNLP, 2019.","title":"Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering","authors":"Vikas Yadav, Steven Bethard, Mihai Surdeanu","year":"2019","venue":"Proceedings of EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":75,"raw":"[Yang et al., 2022] Kaiyu Yang, Jia Deng, and Danqi Chen. Generating natural language proofs with verifier-guided search. In Proceedings of EMNLP, 2022.","title":"Generating natural language proofs with verifier-guided search","authors":"Kaiyu Yang, Jia Deng, Danqi Chen","year":"2022","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":76,"raw":"[Ye et al., 2023] Xi Ye, Qiaochu Chen, Isil Dillig, and Greg Durrett. SatLM: Satisfiability-aided language models using declarative prompting. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.","title":"SatLM: Satisfiability-aided language models using declarative prompting","authors":"Xi Ye, Qiaochu Chen, Isil Dillig, Greg Durrett","year":"2023","venue":"Thirty-seventh Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":77,"raw":"[Zelle and Mooney, 1996] John M. Zelle and Raymond J. Mooney. Learning to parse database queries using inductive logic programming. In Proceedings of AAAI, 1996.","title":"Learning to parse database queries using inductive logic programming","authors":"John M Zelle, Raymond J Mooney","year":"1996","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}