{"type":"ScholarlyArticle","title":"Improving Few-Shot Text-to-SQL with Meta Self-Training via Column Specificity","authors":[{"name":"Xinnan Guo","slug":"xinnan-guo","given":"Xinnan","family":"Guo","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing , China"},{"name":"Yongrui Chen","slug":"yongrui-chen","given":"Yongrui","family":"Chen","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing , China"},{"name":"Guilin Qi","slug":"guilin-qi","given":"Guilin","family":"Qi","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing , China"},{"name":"Tianxing Wu","slug":"tianxing-wu","given":"Tianxing","family":"Wu","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing , China"},{"name":"Hao Xu","slug":"hao-xu","given":"Hao","family":"Xu","affiliation":"Zhejiang Lab , Zhejiang , China"}],"abstract":"The few-shot problem is an urgent challenge for single-table text-to-SQL. Existing methods ignore the potential value of unlabeled data, and merely rely on a coarse-grained Meta-Learning (ML) algorithm that neglects the differences of column contributions to the optimization object. This paper proposes a Meta Self-Training text-to-SQL (MST-SQL) method to solve the problem. Specifically, MST-SQL is based on column-wise HydraNet and adopts self-training as an effective mechanism to learn from readily available unlabeled samples. During each epoch of training, it first predicts pseudo-labels for unlabeled samples and then leverages them to update the parameters. A fine-grained ML algorithm is used in updating, which weighs the contribution of columns by their specificity, in order to further improve the generalizability. Extensive experimental results on both open-domain and domain-specific benchmarks reveal that our MST-SQL has significant advantages in few-shot scenarios,  and is also competitive in standard supervised settings.","doi":"10.24963/ijcai.2022/576","url":"https://ijcai.org/proceedings/2022/576","pdf":"https://ijcai.org/proceedings/2022/0576.pdf","markdown":"https://ijcai.org/proceedings/2022/576.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/576","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":576,"pages":{"first":4150,"last":4156},"keywords":["Data Mining: Information Retrieval","Natural Language Processing: Question Answering"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986723","length_sec":80,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984866","length_sec":332,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Chang et al., 2020] Shuaichen Chang, Pengfei Liu, Yun Tang, Jing Huang, Xiaodong He, and Bowen Zhou. Zero- shot text-to-sql learning with auxiliary task. In AAAI, pages 7488-7495, 2020.","title":"Zeroshot text-to-sql learning with auxiliary task","authors":"Chang","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2021] Yongrui Chen, Xinnan Guo, Chaojie Wang, Jian Qiu, Guilin Qi, Meng Wang, and Huiying Li. Leveraging table content for zero-shot text-to-sql with meta- learning. In AAAI, pages 3992-4000, 2021.","title":"Leveraging table content for zero-shot text-to-sql with metalearning","authors":"Yongrui Chen, Xinnan Guo, Chaojie Wang, Jian Qiu, Guilin Qi, Meng Wang, Huiying Li","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dong and Lapata, 2018] Li Dong and Mirella Lapata. Coarse-to-fine decoding for neural semantic parsing. In ACL, pages 731-742, 2018.","title":"Coarse-to-fine decoding for neural semantic parsing","authors":"Li Dong, Mirella Lapata","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[He et al., 2019] Pengcheng He, Yi Mao, Kaushik Chakrabarti, and Weizhu Chen. X-SQL: reinforce schema representation with context. CoRR, abs/1908.08113, 2019.","title":"X-SQL: reinforce schema representation with context","authors":"Pengcheng He, Yi Mao, Kaushik Chakrabarti, Weizhu Chen","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Herzig et al., 2020] Jonathan Herzig, Pawel Krzysztof Nowak, Thomas Müller, Francesco Piccinno, and Ju- lian Martin Eisenschlos. Tapas: Weakly supervised table parsing via pre-training. In ACL, pages 4320-4333, 2020.","title":"Tapas: Weakly supervised table parsing via pre-training","authors":"Jonathan Herzig, Krzysztof Pawel, Thomas Nowak, Francesco Müller, Julian Piccinno, Eisenschlos Martin","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hu et al., 2021] Xuming Hu, Chenwei Zhang, Fukun Ma, Chenyao Liu, Lijie Wen, and Philip S. Yu. Semi-supervised relation extraction via incremental meta self-training. In Findings of EMNLP, pages 487-496, 2021.","title":"Semi-supervised relation extraction via incremental meta self-training","authors":"Hu","year":"2021","venue":"Findings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hui et al., 2021] Binyuan Hui, Xiang Shi, Ruiying Geng, Binhua Li, Yongbin Li, Jian Sun, and Xiaodan Zhu. Improv- ing text-to-sql with schema dependency learning. CoRR, abs/2103.04399, 2021.","title":"Improving text-to-sql with schema dependency learning","authors":"Binyuan Hui, Xiang Shi, Ruiying Geng, Binhua Li, Yongbin Li, Jian Sun, Xiaodan Zhu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hwang et al., 2019] Wonseok Hwang, Jinyeung Yim, Se- unghyun Park, and Minjoon Seo. A comprehensive explo- ration on wikisql with table-aware word contextualization. CoRR, abs/1902.01069, 2019.","title":"A comprehensive exploration on wikisql with table-aware word contextualization","authors":"Hwang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2019] Xinzhe Li, Qianru Sun, Yaoyao Liu, Qin Zhou, Shibao Zheng, Tat-Seng Chua, and Bernt Schiele. Learning to self-train for semi-supervised few-shot classifi- cation. In NeurIPS, pages 10276-10286, 2019.","title":"Learning to self-train for semi-supervised few-shot classification","authors":"Li","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lin et al., 2020] Xi Victoria Lin, Richard Socher, and Caim- ing Xiong. Bridging textual and tabular data for cross- domain text-to-sql semantic parsing. In Findings of EMNLP, volume EMNLP, pages 4870-4888, 2020.","title":"Bridging textual and tabular data for crossdomain text-to-sql semantic parsing","authors":"Lin","year":"2020","venue":"Findings of EMNLP, volume EMNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692, 2019.","title":"Roberta: A robustly optimized BERT pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lyu et al., 2020] Qin Lyu, Kaushik Chakrabarti, Shob- hit Hathi, Souvik Kundu, Jianwen Zhang, and Zheng Chen. Hybrid ranking network for text-to-sql. CoRR, abs/2008.04759, 2020.","title":"Hybrid ranking network for text-to-sql","authors":"Qin Lyu, Kaushik Chakrabarti, Shobhit Hathi, Souvik Kundu, Jianwen Zhang, Zheng Chen","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ma et al., 2020] Jianqiang Ma, Zeyu Yan, Shuai Pang, Yang Zhang, and Jianping Shen. Mention extraction and linking for SQL query generation. In EMNLP, pages 6936-6942, 2020.","title":"Mention extraction and linking for SQL query generation","authors":"Jianqiang Ma, Zeyu Yan, Shuai Pang, Yang Zhang, Jianping Shen","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Pasupat and Liang, 2015] Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. In ACL, pages 1470-1480, 2015.","title":"Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables","authors":"Liang Pasupat","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Qi et al., 2020] Qi Qi, Xiaolu Wang, Haifeng Sun, Jingyu Wang, Xiao Liang, and Jianxin Liao. A novel multi-task learning framework for semi-supervised semantic parsing. IEEE TASL, 28:2552-2560, 2020.","title":"A novel multi-task learning framework for semi-supervised semantic parsing","authors":"Qi Qi, Xiaolu Wang, Haifeng Sun, Jingyu Wang, Xiao Liang, Jianxin Liao","year":"2020","venue":"IEEE TASL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Scudder, 1965] H. J. Scudder. Probability of error of some adaptive pattern-recognition machines. IEEE Trans. Inf. Theory, 11(3):363-371, 1965.","title":"Probability of error of some adaptive pattern-recognition machines","authors":"H J Scudder","year":"1965","venue":"IEEE Trans. Inf. Theory","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sun et al., 2020] Ningyuan Sun, Xuefeng Yang, and Yun- feng Liu. Tableqa: a large-scale chinese text-to-sql dataset for table-aware SQL generation. CoRR, abs/2006.06434, 2020.","title":"Tableqa: a large-scale chinese text-to-sql dataset for table-aware SQL generation","authors":"Sun","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2018] Chenglong Wang, Po-Sen Huang, Alex Polozov, Marc Brockschmidt, and Rishabh Singh. Execution-guided neural program decoding. CoRR, abs/1807.03100, 2018.","title":"Execution-guided neural program decoding","authors":"Chenglong Wang, Po-Sen Huang, Alex Polozov, Marc Brockschmidt, Rishabh Singh","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2021a] Bailin Wang, Mirella Lapata, and Ivan Titov. Meta-learning for domain generalization in semantic parsing. In NAACL-HLT, pages 366-379, 2021.","title":"Meta-learning for domain generalization in semantic parsing","authors":"Wang","year":"2021","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2021b] Yaqing Wang, Subhabrata Mukherjee, Haoda Chu, Yuancheng Tu, Ming Wu, Jing Gao, and Ahmed Hassan Awadallah. Meta self-training for few-shot neural sequence labeling. In KDD, pages 1737-1747, 2021.","title":"Meta self-training for few-shot neural sequence labeling","authors":"Wang","year":"2021","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wei et al., 2021] Colin Wei, Kendrick Shen, Yining Chen, and Tengyu Ma. Theoretical analysis of self-training with deep networks on unlabeled data. In ICLR, 2021.","title":"Theoretical analysis of self-training with deep networks on unlabeled data","authors":"Colin Wei, Kendrick Shen, Yining Chen, Tengyu Ma","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xu et al., 2017] Xiaojun Xu, Chang Liu, and Dawn Song. Sqlnet: Generating structured queries from nat- ural language without reinforcement learning. CoRR, abs/1711.04436, 2017.","title":"Sqlnet: Generating structured queries from natural language without reinforcement learning","authors":"Xiaojun Xu, Chang Liu, Dawn Song","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xuan et al., 2021] Kuan Xuan, Yongbo Wang, Yongliang Wang, Zujie Wen, and Yang Dong. Sead: End-to-end text- to-sql generation with schema-aware denoising. CoRR, abs/2105.07911, 2021.","title":"Sead: End-to-end textto-sql generation with schema-aware denoising","authors":"Yongbo Kuan Xuan, Yongliang Wang, Zujie Wang, Yang Wen, Dong","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yin et al., 2020] Pengcheng Yin, Graham Neubig, Wen-tau Yih, and Sebastian Riedel. Tabert: Pretraining for joint understanding of textual and tabular data. In ACL, pages 8413-8426, 2020.","title":"Tabert: Pretraining for joint understanding of textual and tabular data","authors":"Pengcheng Yin, Graham Neubig, Wen-Tau Yih, Sebastian Riedel","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yu et al., 2021] Tao Yu, Chien-Sheng Wu, Xi Victoria Lin, Bailin Wang, Yi Chern Tan, Xinyi Yang, Dragomir R. Radev, Richard Socher, and Caiming Xiong. Grappa: Grammar-augmented pre-training for table semantic pars- ing. In ICLR. OpenReview.net, 2021.","title":"Grappa: Grammar-augmented pre-training for table semantic parsing","authors":"Tao Yu, Chien-Sheng Wu, Xi Victoria Lin, Bailin Wang, Yi Chern Tan, Xinyi Yang, Dragomir R Radev, Richard Socher, Caiming Xiong","year":"2021","venue":"ICLR. OpenReview.net","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhong et al., 2017] Victor Zhong, Caiming Xiong, and Richard Socher. Seq2sql: Generating structured queries from natural language using reinforcement learning. CoRR, abs/1709.00103, 2017.","title":"Seq2sql: Generating structured queries from natural language using reinforcement learning","authors":"Victor Zhong, Caiming Xiong, Richard Socher","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}