{"type":"ScholarlyArticle","title":"Effective Search of Logical Forms for Weakly Supervised Knowledge-Based Question Answering","authors":[{"name":"Tao Shen","slug":"tao-shen","given":"Tao","family":"Shen","affiliation":"Centre for Artificial Intelligence , School of Computer Science , University of Technology Sydney"},{"name":"Xiubo Geng","slug":"xiubo-geng","given":"Xiubo","family":"Geng","affiliation":"STCA NLP Group , Microsoft"},{"name":"Guodong Long","slug":"guodong-long","given":"Guodong","family":"Long","affiliation":"Centre for Artificial Intelligence , School of Computer Science , University of Technology Sydney"},{"name":"Jing Jiang","slug":"jing-jiang","given":"Jing","family":"Jiang","affiliation":"Centre for Artificial Intelligence , School of Computer Science , University of Technology Sydney"},{"name":"Chengqi Zhang","slug":"chengqi-zhang","given":"Chengqi","family":"Zhang","affiliation":"Centre for Artificial Intelligence , School of Computer Science , University of Technology Sydney"},{"name":"Daxin Jiang","slug":"daxin-jiang","given":"Daxin","family":"Jiang","affiliation":"STCA NLP Group , Microsoft"}],"abstract":"Many algorithms for Knowledge-Based Question Answering (KBQA) depend on semantic parsing, which translates a question to its logical form. When only weak supervision is provided, it is usually necessary to search valid logical forms for model training. However, a complex question typically involves a huge search space, which creates two main problems: 1) the solutions limited by computation time and memory usually reduce the success rate of the search, and 2) spurious logical forms in the search results degrade the quality of training data. These two problems lead to a poorly-trained semantic parsing model.  In this work, we propose an effective search method for weakly supervised KBQA based on operator prediction for questions. With search space constrained by predicted operators, sufficient search paths can be explored,  more valid logical forms can be derived, and operators possibly causing spurious logical forms can be avoided. As a result, a larger proportion of questions in a weakly supervised training set are equipped with logical forms, and fewer spurious logical forms are generated. Such high-quality training data directly contributes to a better semantic parsing model. Experimental results on one of the largest KBQA datasets (i.e., CSQA) verify the effectiveness of our approach and deliver a new state-of-the-art performance.","doi":"10.24963/ijcai.2020/308","url":"https://ijcai.org/proceedings/2020/308","pdf":"https://ijcai.org/proceedings/2020/0308.pdf","markdown":"https://ijcai.org/proceedings/2020/308.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/308","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":308,"pages":{"first":2227,"last":2233},"keywords":["Machine Learning: Knowledge-based Learning","Natural Language Processing: Natural Language Semantics","Data Mining: Mining Graphs, Semi Structured Data, Complex Data"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25388/file_25388.m3u8","length_sec":339,"provider":"hls"}],"references":[{"position":1,"raw":"[Agarwal et al., 2019] Rishabh Agarwal, Chen Liang, Dale Schu- urmans, and Mohammad Norouzi. Learning to generalize from sparse and underspecified rewards. In ICML, 2019.","title":"Learning to generalize from sparse and underspecified rewards","authors":"Rishabh Agarwal, Chen Liang, Dale Schuurmans, Mohammad Norouzi","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andreas et al., 2013] Jacob Andreas, Andreas Vlachos, and Stephen Clark. Semantic parsing as machine translation. In ACL, volume 2, pages 47-52, 2013.","title":"Semantic parsing as machine translation","authors":"Andreas","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Artzi and Zettlemoyer, 2013] Yoav Artzi and Luke Zettlemoyer. Weakly supervised learning of semantic parsers for mapping in- structions to actions. TACL, 1:49-62, 2013.","title":"Weakly supervised learning of semantic parsers for mapping instructions to actions","authors":"Zettlemoyer Artzi, Yoav Artzi, Luke Zettlemoyer","year":"2013","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In ICLR, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bao et al., 2014] Junwei Bao, Nan Duan, Ming Zhou, and Tiejun Zhao. Knowledge-based question answering as machine transla- tion. In ACL, volume 1, pages 967-976, 2014.","title":"Knowledge-based question answering as machine translation","authors":"Junwei Bao, Nan Duan, Ming Zhou, Tiejun Zhao","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Berant et al., 2013] Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. Semantic parsing on freebase from question- answer pairs. In ACL, pages 1533-1544, 2013.","title":"Semantic parsing on freebase from questionanswer pairs","authors":"Jonathan Berant, Andrew Chou, Roy Frostig, Percy Liang","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dasigi et al., 2019] Pradeep Dasigi, Matt Gardner, Shikhar Murty, Luke Zettlemoyer, and Eduard Hovy. Iterative search for weakly supervised semantic parsing. In NAACL, pages 2669-2680, 2019.","title":"Iterative search for weakly supervised semantic parsing","authors":"Pradeep Dasigi, Matt Gardner, Shikhar Murty, Luke Zettlemoyer, Eduard Hovy","year":"2019","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dong and Lapata, 2018] Li Dong and Mirella Lapata. Coarse- to-fine decoding for neural semantic parsing. arXiv preprint arXiv:1805.04793, 2018.","title":"Coarseto-fine decoding for neural semantic parsing","authors":"Li Dong, Mirella Lapata","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guo et al., 2018] Daya Guo, Duyu Tang, Nan Duan, Ming Zhou, and Jian Yin. Dialog-to-action: Conversational question answer- ing over a large-scale knowledge base. In NurIPS, 2018.","title":"Dialog-to-action: Conversational question answering over a large-scale knowledge base","authors":"Duyu Daya Guo, Nan Tang, Ming Duan, Jian Zhou, Yin","year":"2018","venue":"NurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2019] Daya Guo, Duyu Tang, Nan Duan, Ming Zhou, and Jian Yin. Coupling retrieval and meta-learning for context- dependent semantic parsing. In ACL, pages 855-866, 2019.","title":"Coupling retrieval and meta-learning for contextdependent semantic parsing","authors":"Duyu Daya Guo, Nan Tang, Ming Duan, Jian Zhou, Yin","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guu et al., 2017] Kelvin Guu, Panupong Pasupat, Evan Zheran Liu, and Percy Liang. From language to programs: Bridging reinforcement learning and maximum marginal likelihood. arXiv preprint arXiv:1704.07926, 2017.","title":"From language to programs: Bridging reinforcement learning and maximum marginal likelihood","authors":"Kelvin Guu, Panupong Pasupat, Evan Zheran Liu, Percy Liang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Iyyer et al., 2017] Mohit Iyyer, Wen-tau Yih, and Ming-Wei Chang. Search-based neural structured learning for sequential question answering. In ACL, volume 1, pages 1821-1831, 2017.","title":"Search-based neural structured learning for sequential question answering","authors":"Mohit Iyyer, Wen-Tau Yih, Ming-Wei Chang","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jia and Liang, 2016] R. Jia and P. Liang. Data recombination for neural semantic parsing. In ACL, 2016.","title":"Data recombination for neural semantic parsing","authors":"R Jia, P Liang","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kwiatkowski et al., 2011] Tom Kwiatkowski, Luke Zettlemoyer, Sharon Goldwater, and Mark Steedman. Lexical generalization in ccg grammar induction for semantic parsing. In EMNLP, 2011.","title":"Lexical generalization in ccg grammar induction for semantic parsing","authors":"Kwiatkowski","year":"2011","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liang et al., 2018] Chen Liang, Mohammad Norouzi, Jonathan Berant, Quoc Le, and Ni Lao. Memory augmented policy op- timization for program synthesis with generalization. arXiv preprint arXiv:1807.02322, 2018.","title":"Memory augmented policy optimization for program synthesis with generalization","authors":"Chen Liang, Mohammad Norouzi, Jonathan Berant, Quoc Le, Ni Lao","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ling et al., 2016] Wang Ling, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiskỳ, Andrew Senior, Fumin Wang, and Phil Blunsom. Latent predictor networks for code generation. arXiv preprint arXiv:1603.06744, 2016.","title":"Latent predictor networks for code generation","authors":"Wang Ling, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiskỳ, Andrew Senior, Fumin Wang, Phil Blunsom","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Long et al., 2016] Reginald Long, Panupong Pasupat, and Percy Liang. Simpler context-dependent logical forms via model pro- jections. arXiv preprint arXiv:1606.05378, 2016.","title":"Simpler context-dependent logical forms via model projections","authors":"Reginald Long, Panupong Pasupat, Percy Liang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In NIPS, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Miller et al., 2016] Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. Key- value memory networks for directly reading documents. arXiv preprint arXiv:1606.03126, 2016.","title":"Keyvalue memory networks for directly reading documents","authors":"Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein, Antoine Karimi, Jason Bordes, Weston","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Min et al., 2019] Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. A discrete hard EM approach for weakly supervised question answering. In EMNLP, 2019.","title":"A discrete hard EM approach for weakly supervised question answering","authors":"Sewon Min, Danqi Chen, Hannaneh Hajishirzi, Luke Zettlemoyer","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pasupat and Liang, 2015] Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. arXiv preprint arXiv:1508.00305, 2015.","title":"Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables","authors":"Liang Pasupat","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pasupat and Liang, 2016] Panupong Pasupat and Percy Liang. Inferring logical forms from denotations. arXiv preprint arXiv:1606.06900, 2016.","title":"Panupong Pasupat and Percy Liang. Inferring logical forms from denotations","authors":"Liang Pasupat","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Saha et al., 2018] Amrita Saha, Vardaan Pahuja, Mitesh M Khapra, Karthik Sankaranarayanan, and Sarath Chandar. Com- plex sequential question answering: Towards learning to con- verse over linked question answer pairs with a knowledge graph. In AAAI, 2018.","title":"Complex sequential question answering: Towards learning to converse over linked question answer pairs with a knowledge graph","authors":"Amrita Saha, Vardaan Pahuja, M Mitesh, Karthik Khapra, Sarath Sankaranarayanan, Chandar","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Serban et al., 2016] Iulian V Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. Building end-to- end dialogue systems using generative hierarchical neural net- work models. In AAAI, 2016.","title":"Building end-toend dialogue systems using generative hierarchical neural network models","authors":"Alessandro Iulian V Serban, Yoshua Sordoni, Aaron Bengio, Joelle Courville, Pineau","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Shen et al., 2019] Tao Shen, Xiubo Geng, Tao Qin, Daya Guo, Duyu Tang, Nan Duan, Guodong Long, and Daxin Jiang. Multi- task learning for conversational question answering over a large- scale knowledge base. In EMNLP, pages 2442-2451, 2019.","title":"Multitask learning for conversational question answering over a largescale knowledge base","authors":"Tao Shen, Xiubo Geng, Tao Qin, Daya Guo, Duyu Tang, Nan Duan, Guodong Long, Daxin Jiang","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Suhr et al., 2018] Alane Suhr, Srinivasan Iyer, and Yoav Artzi. Learning to map context-dependent sentences to executable for- mal queries. arXiv preprint arXiv:1804.06868, 2018.","title":"Learning to map context-dependent sentences to executable formal queries","authors":"Alane Suhr, Srinivasan Iyer, Yoav Artzi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Shazeer, Noam, Niki Par- mar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Vlachos and Clark, 2014] Andreas Vlachos and Stephen Clark. A new corpus and imitation learning framework for context- dependent semantic parsing. TACL, 2:547-560, 2014.","title":"A new corpus and imitation learning framework for contextdependent semantic parsing","authors":"Clark ; Vlachos, Andreas Vlachos, Stephen Clark","year":"2014","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wong and Mooney, 2007] Yuk Wah Wong and Raymond Mooney. Learning synchronous grammars for semantic parsing with lambda calculus. In ACL, pages 960-967, 2007.","title":"Learning synchronous grammars for semantic parsing with lambda calculus","authors":"Mooney Wong, Yuk Wah Wong, Raymond Mooney","year":"2007","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xiao et al., 2016] Chunyang Xiao, Marc Dymetman, and Claire Gardent. Sequence-based structured prediction for semantic pars- ing. In ACL, volume 1, pages 1341-1350, 2016.","title":"Sequence-based structured prediction for semantic parsing","authors":"Chunyang Xiao, Marc Dymetman, Claire Gardent","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yih et al., 2015] Scott Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. semantic parsing via staged query graph generation: question answering with knowledge base. In ACL, 2015.","title":"semantic parsing via staged query graph generation: question answering with knowledge base","authors":"Wen-Tau Scott, Ming-Wei Yih, Xiaodong Chang, Jianfeng He, Gao","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zettlemoyer and Collins, 2009] Luke S Zettlemoyer and Michael Collins. Learning context-dependent mappings from sentences to logical form. In ACL, pages 976-984, 2009.","title":"Learning context-dependent mappings from sentences to logical form","authors":"Collins ; Zettlemoyer, S Luke, Michael Zettlemoyer, Collins","year":"2009","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang et al., 2017] Yuchen Zhang, Panupong Pasupat, and Percy Liang. Macro grammars and holistic triggering for efficient se- mantic parsing. arXiv preprint arXiv:1707.07806, 2017.","title":"Macro grammars and holistic triggering for efficient semantic parsing","authors":"Yuchen Zhang, Panupong Pasupat, Percy Liang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhao and Huang, 2014] Kai Zhao and Liang Huang. Type-driven incremental semantic parsing with polymorphism. arXiv preprint arXiv:1411.5379, 2014.","title":"Type-driven incremental semantic parsing with polymorphism","authors":"Huang ; ] Kai Zhao, Liang Zhao, Huang","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}