{"type":"ScholarlyArticle","title":"Solving Probability Problems in Natural Language","authors":[{"name":"Anton Dries","slug":"anton-dries","given":"Anton","family":"Dries","affiliation":"University of Edinburgh , UK"},{"name":"Angelika Kimmig","slug":"angelika-kimmig","given":"Angelika","family":"Kimmig","affiliation":"University of Edinburgh , UK"},{"name":"Jesse Davis","slug":"jesse-davis","given":"Jesse","family":"Davis","affiliation":"University of Edinburgh , UK"},{"name":"Vaishak Belle","slug":"vaishak-belle","given":"Vaishak","family":"Belle","affiliation":"‡ Department of Computer Science , KU Leuven , Belgium"},{"name":"Luc de Raedt","slug":"luc-de-raedt","given":"Luc","family":"de Raedt","affiliation":"University of Edinburgh , UK"}],"abstract":"The ability to solve probability word problems such as those found in introductory discrete mathematics textbooks, is an important cognitive and intellectual skill. In this paper, we develop a two-step end-to-end fully automated approach for solving such questions that is able to automatically provide answers to exercises about probability formulated in natural language.In the first step, a question formulated in natural language is analysed and transformed into a high-level model specified in a declarative language. In the second step, a solution to the high-level model is computed using a probabilistic programming system. On a dataset of 2160 probability problems,  our solver is able to correctly answer 97.5% of the questions given a correct model.  On the end-to-end evaluation, we are able to answer 12.5% of the questions (or 31.1% if we exclude examples not supported by design).","doi":"10.24963/ijcai.2017/556","url":"https://ijcai.org/proceedings/2017/556","pdf":"https://ijcai.org/proceedings/2017/0556.pdf","markdown":"https://ijcai.org/proceedings/2017/556.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/556","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":556,"pages":{"first":3981,"last":3987},"keywords":["Constraints and Satisfiability: Solvers and Tools","Natural Language Processing: Question Answering"],"videos":[],"references":[{"position":1,"raw":"[Alvin et al., 2014] Chris Alvin, Sumit Gulwani, Rupak Ma- jumdar, and Supratik Mukhopadhyay. Synthesis of geom- etry proof problems. In AAAI, pages 245-252, 2014.","title":"Synthesis of geometry proof problems","authors":"Chris Alvin, Sumit Gulwani, Rupak Majumdar, Supratik Mukhopadhyay","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[De Raedt and Kimmig, 2015] Luc De Raedt and Angelika Kimmig. Probabilistic (logic) programming concepts. Ma- chine Learning, pages 1-43, 2015.","title":"Probabilistic (logic) programming concepts","authors":"De Raedt, ; Kimmig, De Luc, Angelika Raedt, Kimmig","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[De Raedt et al., 2007] Luc De Raedt, Angelika Kimmig, and Hannu Toivonen. ProbLog: A probabilistic Prolog and its application in link discovery. In IJCAI, pages 2462- 2467, 2007.","title":"ProbLog: A probabilistic Prolog and its application in link discovery","authors":"De Raedt","year":"2007","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2007/396"},{"position":4,"raw":"[De Raedt et al., 2016] Luc De Raedt, Kristian Kersting, Sri- raam Natarajan, and David Poole. Statistical relational ar- tificial intelligence: Logic, probability, and computation. Synthesis Lectures on Artificial Intelligence and Machine Learning, 10(2):1-189, 2016.","title":"Statistical relational artificial intelligence: Logic, probability, and computation","authors":"De Raedt","year":"2016","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gelb, 1971] Jack P Gelb. Experiments with a natural lan- guage problem-solving system. In IJCAI, pages 455-462, 1971.","title":"Experiments with a natural language problem-solving system","authors":"P Jack, Gelb","year":"1971","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/1971/47"},{"position":6,"raw":"[Getoor and Taskar, 2007] Lise Getoor and Ben Taskar. In- troduction to statistical relational learning. MIT press, 2007.","title":"Introduction to statistical relational learning","authors":"Lise Getoor, Ben Taskar","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gordon et al., 2014] Andrew D Gordon, Thomas A Hen- zinger, Aditya V Nori, and Sriram K Rajamani. Proba- bilistic programming. In Proceedings of the on Future of Software Engineering, pages 167-181. ACM, 2014.","title":"Probabilistic programming","authors":"Thomas A Andrew D Gordon, Aditya V Henzinger, Sriram K Nori, Rajamani","year":"2014","venue":"Proceedings of the on Future of Software Engineering","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hernández-Orallo et al., 2016] José Hernández-Orallo, Fer- nando Martínez-Plumed, Ute Schmid, Michael Siebers, and David L Dowe. Computer models solving intelligence test problems: Progress and implications. Artificial Intel- ligence, 230:74-107, 2016.","title":"Computer models solving intelligence test problems: Progress and implications","authors":"Hernández-Orallo","year":"2016","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hosseini et al., 2014] Mohammad Javad Hosseini, Han- naneh Hajishirzi, Oren Etzioni, and Nate Kushman. Learn- ing to solve arithmetic word problems with verb catego- rization. In EMNLP 2014, pages 523-533, 2014.","title":"Learning to solve arithmetic word problems with verb categorization","authors":"Mohammad Javad Hosseini, Hannaneh Hajishirzi, Oren Etzioni, Nate Kushman","year":"2014","venue":"EMNLP 2014","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kushman et al., 2014] Nate Kushman, Yoav Artzi, Luke Zettlemoyer, and Regina Barzilay. Learning to automat- ically solve algebra word problems. ACL (1), pages 271- 281, 2014.","title":"Learning to automatically solve algebra word problems","authors":"Kushman","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Mukherjee and Garain, 2008] Anirban Mukherjee and Ut- pal Garain. A review of methods for automatic understand- ing of natural language mathematical problems. Artificial Intelligence Review, 29(2):93-122, 2008.","title":"Anirban Mukherjee and Utpal Garain. A review of methods for automatic understanding of natural language mathematical problems","authors":"Garain Mukherjee","year":"2008","venue":"Artificial Intelligence Review","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Niepert and Van den Broeck, 2014] Mathias Niepert and Guy Van den Broeck. Tractability through exchangeabil- ity: A new perspective on efficient probabilistic inference. In AAAI, 2014.","title":"Tractability through exchangeability: A new perspective on efficient probabilistic inference","authors":"Van Niepert, Broeck Den, Mathias Niepert, Guy Van Den Broeck","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Poole, 2003] David Poole. First-order probabilistic infer- ence. IJCAI, 3:985-991, 2003.","title":"First-order probabilistic inference","authors":"David Poole","year":"2003","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2003/141"},{"position":14,"raw":"[Seo et al., 2014] Min Joon Seo, Hannaneh Hajishirzi, Ali Farhadi, and Oren Etzioni. Diagram understanding in ge- ometry questions. In AAAI, 2014.","title":"Diagram understanding in geometry questions","authors":"Seo","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Seo et al., 2015] Minjoon Seo, Hannaneh Hajishirzi, Ali Farhadi, Oren Etzioni, and Clint Malcolm. Solving geom- etry problems: Combining text and diagram interpretation. EMNLP, 2015.","title":"Solving geometry problems: Combining text and diagram interpretation","authors":"Seo","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}