{"type":"ScholarlyArticle","title":"Symbolic Priors for RNN-based Semantic Parsing","authors":[{"name":"Chunyang Xiao","slug":"chunyang-xiao","given":"Chunyang","family":"Xiao","affiliation":"Xerox Research Centre Europe"},{"name":"Marc Dymetman","slug":"marc-dymetman","given":"Marc","family":"Dymetman","affiliation":"Xerox Research Centre Europe"},{"name":"Claire Gardent","slug":"claire-gardent","given":"Claire","family":"Gardent","affiliation":"CNRS , LORIA , UMR 7503"}],"abstract":"Seq2seq models based on Recurrent Neural Networks\r\n(RNNs) have recently received a lot of attention\r\nin the domain of Semantic Parsing. While\r\nin principle they can be trained directly on pairs\r\n(natural language utterances, logical forms), their\r\nperformance is limited by the amount of available\r\ndata. To alleviate this problem, we propose to\r\nexploit various sources of prior knowledge: the\r\nwell-formedness of the logical forms is modeled\r\nby a weighted context-free grammar; the likelihood\r\nthat certain entities present in the input utterance\r\nare also present in the logical form is modeled by\r\nweighted finite-state automata. The grammar and\r\nautomata are combined together through an efficient\r\nintersection algorithm to form a soft guide\r\n(“background”) to the RNN.We test our method on\r\nan extension of the Overnight dataset and show that\r\nit not only strongly improves over an RNN baseline,\r\nbut also outperforms non-RNN models based\r\non rich sets of hand-crafted features.","doi":"10.24963/ijcai.2017/585","url":"https://ijcai.org/proceedings/2017/585","pdf":"https://ijcai.org/proceedings/2017/0585.pdf","markdown":"https://ijcai.org/proceedings/2017/585.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/585","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":585,"pages":{"first":4186,"last":4192},"keywords":["Natural Language Processing: Question Answering","Machine Learning: Deep Learning","Machine Learning: Knowledge-based Learning"],"videos":[],"references":[{"position":1,"raw":"[Bar-Hillel et al., 1961] Yehoshua Bar-Hillel, Micha A. Per- les, and Eli Shamir. On formal properties of simple phrase structure grammars. Zeitschrift für Phonetik, Sprach- wissenschaft und Kommunicationsforschung, 14:143-172, 1961.","title":"Perles, and Eli Shamir. On formal properties of simple phrase structure grammars","authors":"Yehoshua Bar-Hillel, Micha A","year":"1961","venue":"Zeitschrift für Phonetik, Sprachwissenschaft und Kommunicationsforschung","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Berant and Liang, 2014] Jonathan Berant and Percy Liang. Semantic parsing via paraphrasing. In Annual Meeting for the Association for Computational Linguistics (ACL), 2014.","title":"Semantic parsing via paraphrasing","authors":"Liang Berant, Jonathan Berant, Percy Liang","year":"2014","venue":"Annual Meeting for the Association for Computational Linguistics (ACL)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Berant et al., 2013] Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. Semantic parsing on freebase from question-answer pairs. In Empirical Methods in Nat- ural Language Processing (EMNLP), 2013.","title":"Semantic parsing on freebase from question-answer pairs","authors":"Jonathan Berant, Andrew Chou, Roy Frostig, Percy Liang","year":"2013","venue":"Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Clark and Curran, 2007] Stephen Clark and James R. Cur- ran. Wide-coverage efficient statistical parsing with CCG and log-linear models. Computational Linguistics, 33(4):493-552, 2007.","title":"Wide-coverage efficient statistical parsing with CCG and log-linear models","authors":"Stephen Clark, James R Curran","year":"2007","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong and Lapata, 2016] Li Dong and Mirella Lapata. Lan- guage to logical form with neural attention. In Proceedings of the 54th Annual Meeting of the Association for Compu- tational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers, 2016.","title":"Language to logical form with neural attention","authors":"Li Dong, Mirella Lapata","year":"2016","venue":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dyer, 2010] Christopher Dyer. A Formal Model of Ambi- guity and its Applications in Machine Translation. PhD thesis, University of Maryland, 2010.","title":"A Formal Model of Ambiguity and its Applications in Machine Translation","authors":"Christopher Dyer","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dymetman and Xiao, 2016] Marc Dymetman and Chun- yang Xiao. Log-linear rnns: Towards recurrent neu- ral networks with flexible prior knowledge. CoRR, abs/1607.02467, 2016.","title":"Marc Dymetman and Chunyang Xiao. Log-linear rnns: Towards recurrent neural networks with flexible prior knowledge","authors":"Xiao Dymetman","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goyal et al., 2016] Raghav Goyal, Marc Dymetman, and Éric Gaussier. Natural language generation through character-based rnns with finite-state prior knowledge. In COLING, pages 1083-1092, 2016.","title":"Natural language generation through character-based rnns with finite-state prior knowledge","authors":"Raghav Goyal, Marc Dymetman, Éric Gaussier","year":"2016","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hu et al., 2016] Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard H. Hovy, and Eric P. Xing. Harnessing deep neural networks with logic rules. In ACL (1), 2016.","title":"Harnessing deep neural networks with logic rules","authors":"Hu","year":"2016","venue":"ACL (1)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jia and Liang, 2016] Robin Jia and Percy Liang. Data re- combination for neural semantic parsing. In Proceedings of the 54th Annual Meeting of the Association for Compu- tational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers, 2016.","title":"Data recombination for neural semantic parsing","authors":"Liang ; ] Jia, Robin Jia, Percy Liang","year":"2016","venue":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kwiatkowski et al., 2013] Tom Kwiatkowski, Eunsol Choi, Yoav Artzi, and Luke S. Zettlemoyer. Scaling semantic parsers with on-the-fly ontology matching. In Empiri- cal Methods in Natural Language Processing, (EMNLP), pages 1545-1556, 2013.","title":"Scaling semantic parsers with on-the-fly ontology matching","authors":"Tom Kwiatkowski, Eunsol Choi, Yoav Artzi, Luke S Zettlemoyer","year":"2013","venue":"Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liang et al., 2016] Chen Liang, Jonathan Berant, Quoc Le, Kenneth D. Forbus, and Ni Lao. Neural symbolic ma- chines: Learning semantic parsers on freebase with weak supervision. CoRR, abs/1611.00020, 2016.","title":"Neural symbolic machines: Learning semantic parsers on freebase with weak supervision","authors":"Chen Liang, Jonathan Berant, Quoc Le, Kenneth D Forbus, Ni Lao","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mou et al., 2016] Lili Mou, Zhengdong Lu, Hang Li, and Zhi Jin. Coupling distributed and symbolic execution for natural language queries. CoRR, abs/1612.02741, 2016.","title":"Coupling distributed and symbolic execution for natural language queries","authors":"Lili Mou, Zhengdong Lu, Hang Li, Zhi Jin","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nederhof and Satta, 2003] M.J. Nederhof and G. Satta. Probabilistic parsing as intersection. In Proc. 8th Inter- national Workshop on Parsing Technologies, 2003.","title":"Probabilistic parsing as intersection","authors":"M J Nederhof, G Satta","year":"2003","venue":"Proc. 8th International Workshop on Parsing Technologies","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Neelakantan et al., 2016] Arvind Neelakantan, Quoc V. Le, Martín Abadi, Andrew McCallum, and Dario Amodei. Learning a natural language interface with neural pro- grammer. CoRR, abs/1611.08945, 2016.","title":"Learning a natural language interface with neural programmer","authors":"Arvind Neelakantan, Quoc V Le, Martín Abadi, Andrew Mccallum, Dario Amodei","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pasupat and Liang, 2015] Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. In ACL(1), 2015.","title":"Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables","authors":"Liang Pasupat","year":"2015","venue":"ACL(1)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pereira and Warren, 1980] Fernando C.N. Pereira and David H.D. Warren. Definite clause grammars for language analysis a survey of the formalism and a com- parison with augmented transition networks. Artificial Intelligence, 13:231 -278, 1980.","title":"Definite clause grammars for language analysis a survey of the formalism and a comparison with augmented transition networks","authors":"C N Fernando, David H D Pereira, Warren","year":"1980","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Reddy et al., 2014] Siva Reddy, Mirella Lapata, and Mark Steedman. Large-scale semantic parsing without question- answer pairs. Transactions of the Association for Compu- tational Linguistics, 2:377-392, 2014.","title":"Large-scale semantic parsing without questionanswer pairs","authors":"Siva Reddy, Mirella Lapata, Mark Steedman","year":"2014","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Russell and Norvig, 2003] Stuart J. Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Pear- son Education, 2 edition, 2003.","title":"Artificial Intelligence: A Modern Approach","authors":"Stuart J Russell, Peter Norvig","year":"2003","venue":"Pearson Education","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Salakhutdinov et al., 2013] Ruslan Salakhutdinov, Joshua B. Tenenbaum, and Antonio Torralba. Learning with hierarchical-deep models. IEEE Trans. Pattern Analysis and Machine Intelligence, 35(8):1958-1971, 2013.","title":"Learning with hierarchical-deep models","authors":"Ruslan Salakhutdinov, Joshua B Tenenbaum, Antonio Torralba","year":"2013","venue":"IEEE Trans. Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research (JMLR), 15(1):1929-1958, 2014.","title":"Dropout: A simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"Journal of Machine Learning Research (JMLR)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Stolcke, 1994] Andreas Stolcke. An Efficient Probabilis- tic Context-Free Parsing Algorithm that Computes Prefix Probabilities. Computational Linguistics, 21(2):165-201, 1994.","title":"An Efficient Probabilistic Context-Free Parsing Algorithm that Computes Prefix Probabilities","authors":"Andreas Stolcke","year":"1994","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2015] Yushi Wang, Jonathan Berant, and Percy Liang. Building a semantic parser overnight. In Annual Meeting of the Association for Computational Linguis- tics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL Volume 1: Long Papers, pages 1332-1342, 2015.","title":"Building a semantic parser overnight","authors":"Yushi Wang, Jonathan Berant, Percy Liang","year":"2015","venue":"Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xiao et al., 2016] Chunyang Xiao, Marc Dymetman, and Claire Gardent. Sequence-based structured prediction for semantic parsing. In ACL(1), 2016.","title":"Sequence-based structured prediction for semantic parsing","authors":"Chunyang Xiao, Marc Dymetman, Claire Gardent","year":"2016","venue":"ACL(1)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}