{"type":"ScholarlyArticle","title":"An Encoder-Decoder Framework Translating Natural Language to Database Queries","authors":[{"name":"Ruichu Cai","slug":"ruichu-cai","given":"Ruichu","family":"Cai","affiliation":"Faculty of Computer , Guangdong University of Technology , China"},{"name":"Boyan Xu","slug":"boyan-xu","given":"Boyan","family":"Xu","affiliation":"Faculty of Computer , Guangdong University of Technology , China"},{"name":"Zhenjie Zhang","slug":"zhenjie-zhang","given":"Zhenjie","family":"Zhang","affiliation":"Singapore R&D , Yitu Technology Ltd ."},{"name":"Xiaoyan Yang","slug":"xiaoyan-yang","given":"Xiaoyan","family":"Yang","affiliation":"Singapore R&D , Yitu Technology Ltd ."},{"name":"Zijian Li","slug":"zijian-li","given":"Zijian","family":"Li","affiliation":"Faculty of Computer , Guangdong University of Technology , China"},{"name":"Zhihao Liang","slug":"zhihao-liang","given":"Zhihao","family":"Liang","affiliation":"Faculty of Computer , Guangdong University of Technology , China"}],"abstract":"Machine translation is going through a radical revolution, driven by the explosive development of deep learning techniques using Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). In this paper, we consider a special case in machine translation problems, targeting to convert natural language into Structured Query Language (SQL) for data retrieval over relational database. Although generic CNN and RNN learn the grammar structure of SQL when trained with sufficient samples, the accuracy and training efficiency of the model could be dramatically improved, when the translation model is deeply integrated with the grammar rules of SQL. We present a new encoder-decoder framework, with a suite of new approaches, including new semantic features fed into the encoder, grammar-aware states injected into the memory of decoder, as well as recursive state management for sub-queries. These techniques help the neural network better focus on understanding semantics of operations in natural language and save the efforts on SQL grammar learning. The empirical evaluation on real world database and queries show that our approach outperform state-of-the-art solution by a significant margin.","doi":"10.24963/ijcai.2018/553","url":"https://ijcai.org/proceedings/2018/553","pdf":"https://ijcai.org/proceedings/2018/0553.pdf","markdown":"https://ijcai.org/proceedings/2018/553.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/553","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":553,"pages":{"first":3977,"last":3983},"keywords":["Natural Language Processing: NLP Applications and Tools","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Castano et al., 1997] M Asunción Castano, Francisco Casacuberta, and Enrique Vidal. Machine translation using neural networks and finite-state models. Theoretical and Methodological Issues in Machine Translation, pages 160-167, 1997.","title":"Machine translation using neural networks and finite-state models","authors":"Francisco M Asunción Castano, Enrique Casacuberta, Vidal","year":"1997","venue":"Theoretical and Methodological Issues in Machine Translation","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014.","title":"Learning phrase representations using rnn encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dong and Lapata, 2016] Li Dong and Mirella Lapata. Lan- guage to logical form with neural attention. In ACL, 2016.","title":"Language to logical form with neural attention","authors":"Li Dong, Mirella Lapata","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gehring et al., 2017] Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. Convo- lutional sequence to sequence learning. In ICML, pages 1243-1252, 2017.","title":"Convolutional sequence to sequence learning","authors":"Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N Dauphin","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Giordani and Moschitti, 2012] Alessandra Giordani and Alessandro Moschitti. Translating questions to SQL queries with generative parsers discriminatively reranked. In COLING, pages 401-410, 2012.","title":"Translating questions to SQL queries with generative parsers discriminatively reranked","authors":"Alessandra Giordani, Alessandro Moschitti","year":"2012","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Guu et al., 2017] Kelvin Guu, Panupong Pasupat, Evan Zheran Liu, and Percy Liang. From language to programs: Bridging reinforcement learning and maximum marginal likelihood. In ACL, 2017.","title":"From language to programs: Bridging reinforcement learning and maximum marginal likelihood","authors":"Kelvin Guu, Panupong Pasupat, Evan Zheran Liu, Percy Liang","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Iyer et al., 2017] Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, Jayant Krishnamurthy, and Luke Zettlemoyer. Learning a neural semantic parser from user feedback. In ACL, pages 963-973, 2017.","title":"Learning a neural semantic parser from user feedback","authors":"Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, Jayant Krishnamurthy, Luke Zettlemoyer","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jia and Liang, 2016] Robin Jia and Percy Liang. Data re- combination for neural semantic parsing. In ACL, 2016.","title":"Data recombination for neural semantic parsing","authors":"Liang Jia, Robin Jia, Percy Liang","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Koncar and Guthrie, 1997] Nenad Koncar and Gregory Guthrie. A natural language translation neural network. In New Methods in Language Processing, pages 219-228, 1997.","title":"A natural language translation neural network","authors":"Guthrie ; Koncar, Nenad Koncar, Gregory Guthrie","year":"1997","venue":"New Methods in Language Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lafferty et al., 2001] John Lafferty, Andrew McCallum, Fernando Pereira, et al. Conditional random fields: Proba- bilistic models for segmenting and labeling sequence data. In ICML, volume 1, pages 282-289, 2001.","title":"Conditional random fields: Probabilistic models for segmenting and labeling sequence data","authors":"Lafferty","year":"2001","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Jagadish, 2014] Fei Li and HV Jagadish. Construct- ing an interactive natural language interface for relational databases. Proceedings of the VLDB Endowment, 8(1):73- 84, 2014.","title":"Constructing an interactive natural language interface for relational databases","authors":"Jagadish ; ] Fei Li, H V Li, Jagadish","year":"2014","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liang et al., 2017] Chen Liang, Jonathan Berant, Quoc Le, Kenneth D. Forbus, and Ni Lao. Neural symbolic ma- chines: Learning semantic parsers on freebase with weak supervision. In ACL, pages 23-33, 2017.","title":"Neural symbolic machines: Learning semantic parsers on freebase with weak supervision","authors":"Chen Liang, Jonathan Berant, Quoc Le, Kenneth D Forbus, Ni Lao","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liang, 2016] Percy Liang. Learning executable semantic parsers for natural language understanding. Communica- tions of ACM, 59(9):68-76, 2016.","title":"Learning executable semantic parsers for natural language understanding","authors":"Percy Liang, Liang","year":"2016","venue":"Communications of ACM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Long et al., 2016] Reginald Long, Panupong Pasupat, and Percy Liang. Simpler context-dependent logical forms via model projections. In ACL, 2016.","title":"Simpler context-dependent logical forms via model projections","authors":"Reginald Long, Panupong Pasupat, Percy Liang","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed rep- resentations of words and phrases and their composition- ality. In NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mou et al., 2017] Lili Mou, Zhengdong Lu, Hang Li, and Zhi Jin. Coupling distributed and symbolic execution for natural language queries. In ICML, pages 2518-2526, 2017.","title":"Coupling distributed and symbolic execution for natural language queries","authors":"Lili Mou, Zhengdong Lu, Hang Li, Zhi Jin","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Neelakantan et al., 2016] Arvind Neelakantan, Quoc V Le, Martin Abadi, Andrew McCallum, and Dario Amodei. Learning a natural language interface with neural pro- grammer. arXiv preprint arXiv:1611.08945, 2016.","title":"Learning a natural language interface with neural programmer","authors":"Arvind Neelakantan, V Quoc, Martin Le, Andrew Abadi, Dario Mccallum, Amodei","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Popescu et al., 2003] Ana-Maria Popescu, Oren Etzioni, and Henry A. Kautz. Towards a theory of natural language interfaces to databases. In IUI, pages 149-157, 2003.","title":"Towards a theory of natural language interfaces to databases","authors":"Ana-Maria Popescu, Oren Etzioni, Henry A Kautz","year":"2003","venue":"IUI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rabinovich et al., 2017] Maxim Rabinovich, Mitchell Stern, and Dan Klein. Abstract syntax networks for code generation and semantic parsing. In ACL, pages 1139-1149, 2017.","title":"Abstract syntax networks for code generation and semantic parsing","authors":"Rabinovich","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Roy et al., 2013] Senjuti Basu Roy, Martine De Cock, Vani Mandava, Swapna Savanna, Brian Dalessandro, Claudia Perlich, William Cukierski, and Ben Hamner. The mi- crosoft academic search dataset and kdd cup 2013. In KDD Cup 2013, page 1, 2013.","title":"The microsoft academic search dataset and kdd cup 2013","authors":"Roy","year":"2013","venue":"KDD Cup 2013","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wu et al., 2016] Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google's neural machine translation sys- tem: Bridging the gap between human and machine trans- lation. arXiv preprint arXiv:1609.08144, 2016.","title":"Google's neural machine translation system: Bridging the gap between human and machine translation","authors":"Yonghui Wu, Mike Schuster, Zhifeng Chen, V Quoc, Mohammad Le, Wolfgang Norouzi, Maxim Macherey, Yuan Krikun, Qin Cao, Klaus Gao, Macherey","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xiao et al., 2016] Chunyang Xiao, Marc Dymetman, and Claire Gardent. Sequence-based structured prediction for semantic parsing. ACL, 2016.","title":"Sequence-based structured prediction for semantic parsing","authors":"Chunyang Xiao, Marc Dymetman, Claire Gardent","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yin and Neubig, 2017] Pengcheng Yin and Graham Neu- big. A syntactic neural model for general-purpose code generation. In ACL, pages 440-450, 2017.","title":"Pengcheng Yin and Graham Neubig. A syntactic neural model for general-purpose code generation","authors":"Neubig Yin","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yin et al., 2016] Pengcheng Yin, Zhengdong Lu, Hang Li, and Ben Kao. Neural enquirer: Learning to query tables in natural language. In IJCAI, pages 2308-2314, 2016.","title":"Neural enquirer: Learning to query tables in natural language","authors":"Pengcheng Yin, Zhengdong Lu, Hang Li, Ben Kao","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/329"},{"position":27,"raw":"[Zhou et al., 2016] Jie Zhou, Ying Cao, Xuguang Wang, Peng Li, and Wei Xu. Deep recurrent models with fast- forward connections for neural machine translation. TACL, 4:371-383, 2016.","title":"Deep recurrent models with fastforward connections for neural machine translation","authors":"Jie Zhou, Ying Cao, Xuguang Wang, Peng Li, Wei Xu","year":"2016","venue":"TACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}