{"type":"ScholarlyArticle","title":"On Conditional and Compositional Language Model Differentiable Prompting","authors":[{"name":"Jonathan Pilault","slug":"jonathan-pilault","given":"Jonathan","family":"Pilault","affiliation":"Mila -Québec AI Institute , Polytechnique Montréal"},{"name":"Can Liu","slug":"can-liu","given":"Can","family":"Liu","affiliation":"Amazon Alexa ,"},{"name":"Mohit Bansal","slug":"mohit-bansal","given":"Mohit","family":"Bansal","affiliation":"University of North Carolina at Chapel Hill"},{"name":"Markus Dreyer","slug":"markus-dreyer","given":"Markus","family":"Dreyer","affiliation":"Amazon Alexa ,"}],"abstract":"Prompts have been shown to be an effective method to adapt a frozen Pretrained Language Model (PLM) to perform well on downstream tasks. Prompts can be represented by a human-engineered word sequence or by a learned continuous embedding. \r\nIn this work, we investigate conditional and compositional differentiable prompting.\r\nWe propose a new model, Prompt Production System (ProPS), which learns to transform task instructions or input metadata, into continuous prompts that elicit task-specific outputs from the PLM. \r\nOur model uses a modular network structure based on our neural formulation of Production Systems, which allows the model to learn discrete rules -- neural functions that learn to specialize in transforming particular prompt input patterns, making it suitable for compositional transfer learning and few-shot learning. \r\nWe present extensive empirical and theoretical analysis and show that ProPS consistently surpasses other PLM adaptation techniques, and often improves upon fully fine-tuned models, on compositional generalization tasks, controllable summarization and multilingual translation, while needing fewer trainable parameters.","doi":"10.24963/ijcai.2023/460","url":"https://ijcai.org/proceedings/2023/460","pdf":"https://ijcai.org/proceedings/2023/0460.pdf","markdown":"https://ijcai.org/proceedings/2023/460.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/460","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":460,"pages":{"first":4136,"last":4144},"keywords":["Machine Learning: ML: Multi-task and transfer learning","Machine Learning: ML: Neuro-symbolic methods"],"videos":[],"references":[{"position":1,"raw":"[Ando and Zhang, 2005] Rie Kubota Ando and Tong Zhang. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6(61):1817-1853, 2005.","title":"A framework for learning predictive structures from multiple tasks and unlabeled data","authors":"Zhang ; ] Ando, Kubota Rie, Tong Ando, Zhang","year":"2005","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bapna and Firat, 2019] Ankur Bapna and Orhan Firat. Simple, scal- able adaptation for neural machine translation. In Proceedings of the 2019 Conference on EMNLP and the 9th International Joint Conference on Natural Language Processing, pages 1538-1548, Hong Kong, China, November 2019. Association for Computa- tional Linguistics.","title":"Simple, scalable adaptation for neural machine translation","authors":"Ankur Bapna, Orhan Firat","year":"2019","venue":"Proceedings of the 2019 Conference on EMNLP and the 9th International Joint Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cheng and Lapata, 2016] Jianpeng Cheng and Mirella Lapata. Neu- ral summarization by extracting sentences and words. In Proceed- ings of the 54th Annual Meeting of the ACL (Volume 1: Long Papers), pages 484-494, Berlin, Germany, August 2016. Associa- tion for Computational Linguistics.","title":"Neural summarization by extracting sentences and words","authors":"Lapata Cheng, Jianpeng Cheng, Mirella Lapata","year":"2016","venue":"Proceedings of the 54th Annual Meeting of the ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Efrat and Levy, 2020] Avia Efrat and Omer Levy. The turking test: Can language models understand instructions? CoRR, abs/2010.11982, 2020.","title":"The turking test: Can language models understand instructions?","authors":"Levy ; Efrat, Avia Efrat, Omer Levy","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Furrer et al., 2020] Daniel Furrer, Marc van Zee, Nathan Scales, and Nathanael Scharli. Compositional generalization in seman- tic parsing: Pre-training vs. specialized architectures. ArXiv, abs/2007.08970, 2020.","title":"Compositional generalization in semantic parsing: Pre-training vs. specialized architectures","authors":"Daniel Furrer, Nathan Marc Van Zee, Nathanael Scales, Scharli","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goyal and Bengio, 2021] Anirudh Goyal and Yoshua Bengio. Neu- ral production systems. In Advances in Neural Information Pro- cessing Systems, 2021.","title":"Neural production systems","authors":"Anirudh Goyal, Yoshua Bengio","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goyal et al., 2021] Anirudh Goyal, A. Lamb, J. Hoffmann, Shagun Sodhani, S. Levine, Yoshua Bengio, and Bernhard Schölkopf. Recurrent independent mechanisms. In International Conference on Learning Representations, 2021.","title":"Recurrent independent mechanisms","authors":"Anirudh Goyal, A Lamb, J Hoffmann, Shagun Sodhani, S Levine, Yoshua Bengio, Bernhard Schölkopf","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2021] Demi Guo, Alexander M Rush, and Yoon Kim. Parameter-efficient transfer learning with diff pruning. In Pro- ceedings of ACL, 2021.","title":"Parameter-efficient transfer learning with diff pruning","authors":"Demi Guo, Alexander M Rush, Yoon Kim","year":"2021","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Han et al., 2021] Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun. PTR: prompt tuning with rules for text classifi- cation. CoRR, abs/2105.11259, 2021.","title":"PTR: prompt tuning with rules for text classification","authors":"Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, Maosong Sun","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Houlsby et al., 2019] Neil Houlsby, Andrei Giurgiu, Stanislaw Jas- trzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Ges- mundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Con- ference on Machine Learning, volume 97 of Proceedings of Ma- chine Learning Research, pages 2790-2799. PMLR, 09-15 Jun 2019.","title":"Parameter-efficient transfer learning for NLP","authors":"Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, Sylvain Gelly","year":"2019","venue":"Proceedings of the 36th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2021] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.","title":"LoRA: Low-rank adaptation of large language models","authors":"J Edward, Yelong Hu, Phillip Shen, Zeyuan Wallis, Yuanzhi Allen-Zhu, Shean Li, Weizhu Wang, Chen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jiang et al., 2020] Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of the Association for Computational Linguistics (ACL), 8:423-438, 2020.","title":"How can we know what language models know","authors":"Zhengbao Jiang, Frank F Xu, Jun Araki, Graham Neubig","year":"2020","venue":"Transactions of the Association for Computational Linguistics (ACL)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Keysers et al., 2020] Daniel Keysers, Nathanael Schärli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, Dmitry Tsarkov, Xiao Wang, Marc van Zee, and Olivier Bous- quet. Measuring compositional generalization: A comprehensive method on realistic data. In International Conference on Learning Representations, 2020.","title":"Measuring compositional generalization: A comprehensive method on realistic data","authors":"Daniel Keysers, Nathanael Schärli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, Dmitry Tsarkov, Xiao Wang, Marc Van Zee, Olivier Bousquet","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kirsch et al., 2018] Louis Kirsch, Julius Kunze, and David Barber. Modular networks: Learning to decompose neural computation. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31.","title":"Modular networks: Learning to decompose neural computation","authors":"Kirsch","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"Curran Associates, Inc., 2018.","title":null,"authors":"Curran Associates, Inc","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Koehn, 2004] Philipp Koehn. Statistical significance tests for ma- chine translation evaluation. In Proceedings of the 2004 Con- ference on Empirical Methods in Natural Language Processing, pages 388-395, Barcelona, Spain, July 2004. Association for Computational Linguistics.","title":"Statistical significance tests for machine translation evaluation","authors":"Philipp Koehn, Koehn","year":"2004","venue":"Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Koehn, 2005] Philipp Koehn. Europarl: A parallel corpus for statis- tical machine translation. In Proceedings of Machine Translation Summit X: Papers, pages 79-86, Phuket, Thailand, September 13-15 2005.","title":"Europarl: A parallel corpus for statistical machine translation","authors":"Philipp Koehn","year":"2005","venue":"Proceedings of Machine Translation Summit X: Papers","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lake and Baroni, 2018] Brenden M. Lake and Marco Baroni. Gen- eralization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In ICML, 2018.","title":"Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks","authors":"M Brenden, Marco Lake, Baroni","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lester et al., 2021] Brian Lester, Rami Al-Rfou, and Noah Con- stant. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045-3059, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.","title":"The power of scale for parameter-efficient prompt tuning","authors":"Brian Lester, Rami Al-Rfou, Noah Constant","year":"2021","venue":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lewis et al., 2020] Mike Lewis, Yinhan Liu, Naman Goyal, Mar- jan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to- sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the ACL, pages 7871-7880, Online, July 2020. Association for Computational Linguistics.","title":"BART: Denoising sequence-tosequence pre-training for natural language generation, translation, and comprehension","authors":"Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, Luke Zettlemoyer","year":"2020","venue":"Proceedings of the 58th Annual Meeting of the ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Li and Liang, 2021] Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the ACL and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582-4597, Online, August 2021. Associa- tion for Computational Linguistics.","title":"Prefix-tuning: Optimizing continuous prompts for generation","authors":"Lisa Xiang, Percy Li, Liang","year":"2021","venue":"Proceedings of the 59th Annual Meeting of the ACL and the 11th International Joint Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Li et al., 2022] Junyi Li, Tianyi Tang, Jian-Yun Nie, Ji-Rong Wen, and Xin Zhao. Learning to transfer prompts for text generation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3506-3518, Seattle, United States, July 2022. Association for Computational Linguistics.","title":"Learning to transfer prompts for text generation","authors":"Junyi Li, Tianyi Tang, Jian-Yun Nie, Ji-Rong Wen, Xin Zhao","year":"2022","venue":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lin et al., 2020] Zhaojiang Lin, Andrea Madotto, and Pascale Fung. Exploring versatile generative language model via parameter- efficient transfer learning. In Findings of EMNLP 2020, pages 441-459, Online, November 2020. Association for Computational Linguistics.","title":"Exploring versatile generative language model via parameterefficient transfer learning","authors":"Lin","year":"2020","venue":"Findings of EMNLP 2020","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liu et al., 2020] Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. Multilingual denoising pre-training for neural ma- chine translation. Transactions of the Association for Computa- tional Linguistics, 8:726-742, 2020.","title":"Multilingual denoising pre-training for neural machine translation","authors":"Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer","year":"2020","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Liu et al., 2021] Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. GPT understands, too. CoRR, abs/2103.10385, 2021.","title":"GPT understands, too","authors":"Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, Jie Tang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Mishra et al., 2021] Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Natural instructions: Benchmark- ing generalization to new tasks from natural language instructions. CoRR, abs/2104.08773, 2021.","title":"Natural instructions: Benchmarking generalization to new tasks from natural language instructions","authors":"Swaroop Mishra, Daniel Khashabi, Chitta Baral, Hannaneh Hajishirzi","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Mrini et al., 2021a] Khalil Mrini, Can Liu, and Markus Dreyer. Re- wards with negative examples for reinforced topic-focused abstrac- tive summarization. In Proceedings of the Third Workshop on New Frontiers in Summarization, pages 33-38, Online and in Domini- can Republic, November 2021. Association for Computational Linguistics.","title":"Rewards with negative examples for reinforced topic-focused abstractive summarization","authors":"Mrini","year":"2021","venue":"Proceedings of the Third Workshop on New Frontiers in Summarization","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Mrini et al., 2021b] Khalil Mrini, Can Liu, and Markus Dreyer. Re- wards with negative examples for reinforced topic-focused abstrac- tive summarization. In Proceedings of the Third Workshop on New Frontiers in Summarization, pages 33-38, Online and in Domini- can Republic, November 2021. Association for Computational Linguistics.","title":"Rewards with negative examples for reinforced topic-focused abstractive summarization","authors":"Mrini","year":"2021","venue":"Proceedings of the Third Workshop on New Frontiers in Summarization","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Narayan et al., 2018] Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't give me the details, just the summary! Topic-aware convolutional neural networks for extreme summarization. In Proceedings of EMNLP, 2018.","title":"Don't give me the details, just the summary! Topic-aware convolutional neural networks for extreme summarization","authors":"Shashi Narayan, Shay B Cohen, Mirella Lapata","year":"2018","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Newell, 1972] Allen. Newell. Human problem solving. Prentice- Hall, Englewood Cliffs, N. J, 1972.","title":"Human problem solving","authors":"Allen Newell","year":"1972","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Nilsson, 2010] Nils J. Nilsson. The Quest for Artificial Intelligence: A History of Ideas and Achievements. Cambridge University Press, Cambridge, UK, 2010.","title":"The Quest for Artificial Intelligence: A History of Ideas and Achievements","authors":"Nils J Nilsson","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Novikova et al., 2017] Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. The E2E dataset: New challenges for end-to-end generation. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 201-206, Saarbrücken, Germany, August 2017.","title":"The E2E dataset: New challenges for end-to-end generation","authors":"Jekaterina Novikova, Ondřej Dušek, Verena Rieser","year":"2017","venue":"Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ostapenko and Charlin, 2021] Oleksiy Ostapenko and Laurent Charlin. Continual learning via local module composition. In Advances in Neural Information Processing Systems, 2021.","title":"Continual learning via local module composition","authors":"Oleksiy Ostapenko, Laurent Charlin","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Petroni et al., 2019] Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexan- der Miller. Language models as knowledge bases? In Proceedings of the 2019 Conference on EMNLP and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463-2473, Hong Kong, China, November 2019. Associa- tion for Computational Linguistics.","title":"Language models as knowledge bases?","authors":"Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander Miller","year":"2019","venue":"Proceedings of the 2019 Conference on EMNLP and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Pilault et al., 2020] Jonathan Pilault, Raymond Li, Sandeep Sub- ramanian, and Chris Pal. On extractive and abstractive neural document summarization with transformer language models. In Proceedings of the 2020 Conference on EMNLP, pages 9308- 9319, Online, November 2020. Association for Computational Linguistics.","title":"On extractive and abstractive neural document summarization with transformer language models","authors":"Jonathan Pilault, Raymond Li, Sandeep Subramanian, Chris Pal","year":"2020","venue":"Proceedings of the 2020 Conference on EMNLP","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Pilault et al., 2021] Jonathan Pilault, Amine El hattami, and Christopher Pal. Conditionally adaptive multi-task learning: Im- proving transfer learning in nlp using fewer parameters & less data. In International Conference on Learning Representations, 2021.","title":"Conditionally adaptive multi-task learning: Improving transfer learning in nlp using fewer parameters & less data","authors":"Jonathan Pilault, Amine El, Christopher Pal","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Ponti et al., 2023] Edoardo Maria Ponti, Alessandro Sordoni, Yoshua Bengio, and Siva Reddy. Combining parameter-efficient modules for task-level generalisation. In Proceedings of the 17th Conference of the European Chapter of the Association for Com- putational Linguistics, pages 687-702, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics.","title":"Combining parameter-efficient modules for task-level generalisation","authors":"Maria Edoardo, Alessandro Ponti, Yoshua Sordoni, Siva Bengio, Reddy","year":"2023","venue":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Raffel et al., 2020] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020.","title":"Exploring the limits of transfer learning with a unified text-to-text transformer","authors":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu","year":"2020","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Schick and Schütze, 2021] Timo Schick and Hinrich Schütze. Ex- ploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the ACL: Main Volume, pages 255-269, Online, April 2021. Association for Computational Linguistics.","title":"Exploiting cloze-questions for few-shot text classification and natural language inference","authors":"Schütze Schick, Timo Schick, Hinrich Schütze","year":"2021","venue":"Proceedings of the 16th Conference of the European Chapter of the ACL: Main","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Touretzky and Hinton, 1988] David S. Touretzky and Geoffrey E. Hinton. A distributed connectionist production system. Cognitive Science, 12(3):423-466, 1988.","title":"A distributed connectionist production system","authors":"David S Touretzky, Geoffrey E Hinton","year":"1988","venue":"Cognitive Science","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Trischler et al., 2017] Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. NewsQA: A machine comprehension dataset. In Pro- ceedings of the 2nd Workshop on Representation Learning for NLP, pages 191-200, Vancouver, Canada, August 2017. Associa- tion for Computational Linguistics.","title":"NewsQA: A machine comprehension dataset","authors":"Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, Kaheer Suleman","year":"2017","venue":"Proceedings of the 2nd Workshop on Representation Learning for NLP","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Tsimpoukelli et al., 2021] Maria Tsimpoukelli, Jacob Menick, Serkan Cabi, S. M. Ali Eslami, Oriol Vinyals, and Felix Hill. Mul- timodal few-shot learning with frozen language models. CoRR, abs/2106.13884, 2021.","title":"Multimodal few-shot learning with frozen language models","authors":"Maria Tsimpoukelli, Jacob Menick, Serkan Cabi, S M Ali Eslami, Oriol Vinyals, Felix Hill","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł Ukasz Kaiser, Illia Polosukhin","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Wagner, 2020] C.G. Wagner. A First Course in Enumerative Com- binatorics. Pure and Applied Undergraduate Texts. American Mathematical Society, 2020.","title":"A First Course in Enumerative Combinatorics","authors":"C G Wagner","year":"2020","venue":"Pure and Applied Undergraduate Texts","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Wang et al., 2018] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understand- ing. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353- 355, Brussels, Belgium, November 2018. Association for Compu- tational Linguistics.","title":"GLUE: A multi-task benchmark and analysis platform for natural language understanding","authors":"Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, Samuel Bowman","year":"2018","venue":"Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Wu et al., 2020] Chuhan Wu, Fangzhao Wu, Tao Qi, Xiaohui Cui, and Yongfeng Huang. Attentive pooling with learnable norms for text representation. In Proceedings of the 58th Annual Meeting of the ACL, pages 2961-2970, Online, July 2020. Association for Computational Linguistics.","title":"Attentive pooling with learnable norms for text representation","authors":"Chuhan Wu, Fangzhao Wu, Tao Qi, Xiaohui Cui, Yongfeng Huang","year":"2020","venue":"Proceedings of the 58th Annual Meeting of the ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}