{"type":"ScholarlyArticle","title":"Genetic Prompt Search via Exploiting Language Model Probabilities","authors":[{"name":"Jiangjiang Zhao","slug":"jiangjiang-zhao","given":"Jiangjiang","family":"Zhao","affiliation":"Beijing University of Posts and Telecommunications , P.R. China; China Mobile Online Services Co., Ltd. Beijing , P.R. China"},{"name":"Zhuoran Wang","slug":"zhuoran-wang","given":"Zhuoran","family":"Wang","affiliation":"Clouchie Limited , London , United Kingdom"},{"name":"Fangchun Yang","slug":"fangchun-yang","given":"Fangchun","family":"Yang","affiliation":"Beijing University of Posts and Telecommunications , P.R. China"}],"abstract":"Prompt tuning for large-scale pretrained language models (PLMs) has shown remarkable potential, especially in low-resource scenarios such as few-shot learning. Moreover, derivative-free optimisation (DFO) techniques make it possible to tune prompts for a black-box PLM to better fit downstream tasks. However, there are usually preconditions to apply existing DFO-based prompt tuning methods, e.g. the backbone PLM needs to provide extra APIs so that hidden states (and/or embedding vectors) can be injected into it as continuous prompts, or carefully designed (discrete) manual prompts need to be available beforehand, serving as the initial states of the tuning algorithm. To waive such preconditions and make DFO-based prompt tuning ready for general use, this paper introduces a novel genetic algorithm (GA) that evolves from empty prompts, and uses the predictive probabilities derived from the backbone PLM(s) on the basis of a (few-shot) training set to guide the token selection process during prompt mutations. Experimental results on diverse benchmark datasets show that the proposed precondition-free method significantly outperforms the existing DFO-style counterparts that require preconditions, including black-box tuning, genetic prompt search and gradient-free instructional prompt search.","doi":"10.24963/ijcai.2023/588","url":"https://ijcai.org/proceedings/2023/588","pdf":"https://ijcai.org/proceedings/2023/0588.pdf","markdown":"https://ijcai.org/proceedings/2023/588.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/588","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Natural Language Processing","track":"Main Track","number":588,"pages":{"first":5296,"last":5305},"keywords":["Natural Language Processing: NLP: Language models","Natural Language Processing: NLP: Other","Machine Learning: ML: Few-shot learning"],"videos":[],"references":[{"position":1,"raw":"[Ben-David et al., 2022] Eyal Ben-David, Nadav Oved, and Roi Reichart. PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains. Transactions of ACL, 10:414-433, 2022.","title":"PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains","authors":"Eyal Ben-David, Nadav Oved, Roi Reichart","year":"2022","venue":"Transactions of ACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bowman et al., 2015] Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of EMNLP, 2015.","title":"A large annotated corpus for learning natural language inference","authors":"R Samuel, Gabor Bowman, Christopher Angeli, Christopher D Potts, Manning","year":"2015","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brown et al., 2020] Tom Brown, Benjamin Mann, Nick Ry- der, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learn- ers. In Advances in Neural Information Processing Sys- tems, volume 33, 2020.","title":"Language models are few-shot learners","authors":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam Mccandlish, Alec Radford, Ilya Sutskever, Dario Amodei","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2022] Guanzheng Chen, Fangyu Liu, Zaiqiao Meng, and Shangsong Liang. Revisiting parameter- efficient tuning: Are we really there yet? CoRR, abs/2202.07962, 2022.","title":"Revisiting parameterefficient tuning: Are we really there yet?","authors":"Chen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Clark et al., 2020] Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. ELECTRA: Pre-training text encoders as discriminators rather than generators. In Proceedings of ICLR, 2020.","title":"ELECTRA: Pre-training text encoders as discriminators rather than generators","authors":"Kevin Clark, Minh-Thang Luong, Quoc V Le, Christopher D Manning","year":"2020","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Deng et al., 2022] Mingkai Deng, Jianyu Wang, Cheng- Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. RLPrompt: Optimiz- ing discrete text prompts with reinforcement learning. In Proceedings of EMNLP, 2022.","title":"RLPrompt: Optimizing discrete text prompts with reinforcement learning","authors":"Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, Zhiting Hu","year":"2022","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understand- ing. In Proceedings of NAACL-HLT, 2019.","title":"BERT: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Diao et al., 2023] Shizhe Diao, Zhichao Huang, Ruijia Xu, Xuechun Li, Yong Lin, Xiao Zhou, and Tong Zhang. Black-box prompt learning for pre-trained language mod- els. Transactions on Machine Learning Research, 2023.","title":"Black-box prompt learning for pre-trained language models","authors":"Diao","year":"2023","venue":"Transactions on Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gao et al., 2021] Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few- shot learners. In Proceedings of ACL-IJCNLP, 2021.","title":"Making pre-trained language models better fewshot learners","authors":"Tianyu Gao, Adam Fisch, Danqi Chen","year":"2021","venue":"Proceedings of ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hansen et al., 2003] Nikolaus Hansen, Sibylle D. Müller, and Petros Koumoutsakos. Reducing the time complex- ity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evolutionary Computation, 11(1):1-18, 2003.","title":"Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES)","authors":"Nikolaus Hansen, Sibylle D Müller, Petros Koumoutsakos","year":"2003","venue":"Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hou et al., 2022] Bairu Hou, Joe O'Connor, Jacob Andreas, Shiyu Chang, and Yang Zhang. PromptBoosting: Black- box text classification with ten forward passes. CoRR, abs/2212.09257, 2022.","title":"PromptBoosting: Blackbox text classification with ten forward passes","authors":"Bairu Hou, O' Joe, Jacob Connor, Shiyu Andreas, Yang Chang, Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Houlsby et al., 2019] Neil Houlsby, Andrei Giurgiu, Stanis- law Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In Proceed- ings of the ICML, 2019.","title":"Parameter-efficient transfer learning for NLP","authors":"Houlsby","year":"2019","venue":"Proceedings of the ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jiang et al., 2021] Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. How can we know when language models know? on the calibration of language models for question answering. Transactions of ACL, 9, 2021.","title":"How can we know when language models know? on the calibration of language models for question answering","authors":"Jiang","year":"2021","venue":"Transactions of ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceed- ings of ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kolda et al., 2003] Tamara G. Kolda, Robert Michael Lewis, and Virginia Torczon. Optimization by direct search: New perspectives on some classical and modern methods. SIAM Review, 45(3):385-482, 2003.","title":"Optimization by direct search: New perspectives on some classical and modern methods","authors":"Tamara G Kolda, Robert Michael Lewis, Virginia Torczon","year":"2003","venue":"SIAM Review","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lester et al., 2021] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Proceedings of EMNLP, 2021.","title":"The power of scale for parameter-efficient prompt tuning","authors":"Brian Lester, Rami Al-Rfou, Noah Constant","year":"2021","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lewis et al., 2020] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of ACL, 2020.","title":"BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","authors":"Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, Luke Zettlemoyer","year":"2020","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li and Liang, 2021] Xiang Lisa Li and Percy Liang. Prefix- tuning: Optimizing continuous prompts for generation. In Proceedings of ACL-IJCNLP, 2021.","title":"Prefixtuning: Optimizing continuous prompts for generation","authors":"Lisa Xiang, Percy Li, Liang","year":"2021","venue":"Proceedings of ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized BERT pretraining ap- proach. CoRR, abs/1907.11692, 2019.","title":"RoBERTa: A robustly optimized BERT pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2021a] Xiao Liu, Kaixuan Ji, Yicheng Fu, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-Tuning v2: Prompt tuning can be comparable to fine-tuning univer- sally across scales and tasks. CoRR, abs/2110.07602, 2021.","title":"P-Tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks","authors":"Liu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2021b] Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. GPT understands, too. CoRR, abs/2103.10385, 2021.","title":"GPT understands, too","authors":"Liu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2022] Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-Tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of ACL (Volume 2: Short Papers), 2022.","title":"P-Tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks","authors":"Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, Jie Tang","year":"2022","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2023] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):no.195:1-35, 2023.","title":"Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing","authors":"Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig","year":"2023","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Loshchilov and Hutter, 2019] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In Pro- ceedings of ICLR, 2019.","title":"Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization","authors":"Hutter Loshchilov","year":"2019","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mishra et al., 2022] Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task gener- alization via natural language crowdsourcing instructions. In Proceedings of ACL, 2022.","title":"Cross-task generalization via natural language crowdsourcing instructions","authors":"Swaroop Mishra, Daniel Khashabi, Chitta Baral, Hannaneh Hajishirzi","year":"2022","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Mitchell, 1998] Melanie Mitchell. An Introduction to Ge- netic Algorithms. The MIT Press, 1998.","title":"An Introduction to Genetic Algorithms","authors":"Melanie Mitchell","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Ouyang et al., 2022] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training lan- guage models to follow instructions with human feedback. CoRR, abs/2203.02155, 2022.","title":"Training language models to follow instructions with human feedback","authors":"Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, Ryan Lowe","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Petroni et al., 2019] Fabio Petroni, Tim Rocktäschel, Se- bastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowl- edge bases? In Proceedings of EMNLP-IJCNLP, 2019.","title":"Language models as knowledge bases?","authors":"Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander Miller","year":"2019","venue":"Proceedings of EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Pfeiffer et al., 2020] Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebas- tian Ruder, Kyunghyun Cho, and Iryna Gurevych. AdapterHub: A framework for adapting transformers. In Proceedings of EMNLP: System Demonstrations, 2020.","title":"AdapterHub: A framework for adapting transformers","authors":"Pfeiffer","year":"2020","venue":"Proceedings of EMNLP: System Demonstrations","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Prasad et al., 2022] Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. GrIPS: Gradient-free, edit-based instruction search for prompting large language models. CoRR, abs/2203.07281, 2022.","title":"GrIPS: Gradient-free, edit-based instruction search for prompting large language models","authors":"Prasad","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Qin and Eisner, 2021] Guanghui Qin and Jason Eisner. Learning how to ask: Querying LMs with mixtures of soft prompts. In Proceedings of NAACL-HLT, June 2021.","title":"Learning how to ask: Querying LMs with mixtures of soft prompts","authors":"Eisner Qin, Guanghui Qin, Jason Eisner","year":"2021","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Radford et al., 2018] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language un- derstanding by generative pre-training. Technical report, OpenAI, 2018.","title":"Improving language understanding by generative pre-training","authors":"Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Radford et al., 2019] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. Technical re- port, OpenAI, 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Raffel et al., 2020] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020.","title":"Exploring the limits of transfer learning with a unified text-to-text transformer","authors":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu","year":"2020","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Rios and Sahinidis, 2013] Luis Miguel Rios and Niko- laos V. Sahinidis. Derivative-free optimization: a review of algorithms and comparison of software implementations. Journal of Global Optimization, 56(3):1247--1293, 2013.","title":"Derivative-free optimization: a review of algorithms and comparison of software implementations","authors":"Miguel Luis, Nikolaos V Rios, Sahinidis","year":"2013","venue":"Journal of Global Optimization","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Sanh et al., 2022] Victor Sanh, Albert Webson, Colin Raf- fel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Févry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Bi- derman, Leo Gao, Thomas Wolf, and Alexander M. Rush. Multitask prompted training enables zero-shot task gener- alization. In Proceedings of ICLR, 2022.","title":"Multitask prompted training enables zero-shot task generalization","authors":"Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, V Nihal, Debajyoti Nayak, Jonathan Datta, Mike Chang, Tian-Jian, Han Jiang, Matteo Wang, Sheng Manica, Zheng Xin Shen, Harshit Yong, Rachel Pandey, Thomas Bawden, Trishala Wang, Jos Neeraj, Abheesht Rozen, Andrea Sharma, Thibault Santilli, Jason Févry, Alan Fries, Ryan Teehan, Le Teven, Stella Scao, Leo Biderman, Thomas Gao, Alexander M Wolf, Rush","year":"2022","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Schick and Schütze, 2021a] Timo Schick and Hinrich Schütze. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceed- ings of EACL, 2021.","title":"Exploiting cloze-questions for few-shot text classification and natural language inference","authors":"Schütze Schick, Timo Schick, Hinrich Schütze","year":"2021","venue":"Proceedings of EACL","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Schick and Schütze, 2021b] Timo Schick and Hinrich Schütze. Few-shot text generation with natural language instructions. In Proceedings of EMNLP, 2021.","title":"Few-shot text generation with natural language instructions","authors":"Schütze Schick, Timo Schick, Hinrich Schütze","year":"2021","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Schick and Schütze, 2022] Timo Schick and Hinrich Schütze. True few-shot learning with Prompts-A real-world perspective. Transactions of ACL, 10:716-731, 2022.","title":"True few-shot learning with Prompts-A real-world perspective","authors":"Schütze Schick, Timo Schick, Hinrich Schütze","year":"2022","venue":"Transactions of ACL","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Shin et al., 2020] Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. AutoPrompt: Eliciting Knowledge from Language Models with Auto- matically Generated Prompts. In Proceedings of EMNLP, 2020.","title":"AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts","authors":"Taylor Shin, Yasaman Razeghi, Robert L Logan, I V, Eric Wallace, Sameer Singh","year":"2020","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Sun et al., 2022a] Tianxiang Sun, Zhengfu He, Hong Qian, Yunhua Zhou, Xuanjing Huang, and Xipeng Qiu. BBTv2: Towards a gradient-free future with large language models. In Proceedings of EMNLP, 2022.","title":"BBTv2: Towards a gradient-free future with large language models","authors":"Sun","year":"2022","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Sun et al., 2022b] Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, and Xipeng Qiu. Black-box tuning for language-model-as-a-service. In Proceedings of ICML, 2022.","title":"Black-box tuning for language-model-as-a-service","authors":"Sun","year":"2022","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wang et al., 2018] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and In- terpreting Neural Networks for NLP, 2018.","title":"GLUE: A multi-task benchmark and analysis platform for natural language understanding","authors":"Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, Samuel Bowman","year":"2018","venue":"Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Xu et al., 2022] Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yanggang Wang, Haiyu Li, and Zhilin Yang. GPS: Genetic prompt search for efficient few-shot learning. CoRR, abs/2210.17041, 2022.","title":"GPS: Genetic prompt search for efficient few-shot learning","authors":"Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yanggang Wang, Haiyu Li, Zhilin Yang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Yu and Gen, 2010] Xinjie Yu and Mitsuo Gen. Introduction to Evolutionary Algorithms. Springer, 2010.","title":"Introduction to Evolutionary Algorithms","authors":"Gen ; Yu, Xinjie Yu, Mitsuo Gen","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Zhang et al., 2015] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In Advances in Neural Information Process- ing Systems, volume 28, 2015.","title":"Character-level convolutional networks for text classification","authors":"Xiang Zhang, Junbo Zhao, Yann Lecun","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Zhang et al., 2020] Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. PEGASUS: Pre-training with ex- tracted gap-sentences for abstractive summarization. In Proceedings of ICML, 2020.","title":"PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization","authors":"Zhang","year":"2020","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}