{"type":"ScholarlyArticle","title":"Text Transformations in Contrastive Self-Supervised Learning: A Review","authors":[{"name":"Amrita Bhattacharjee","slug":"amrita-bhattacharjee","given":"Amrita","family":"Bhattacharjee","affiliation":"Arizona State University , Tempe , AZ"},{"name":"Mansooreh Karami","slug":"mansooreh-karami","given":"Mansooreh","family":"Karami","affiliation":"Arizona State University , Tempe , AZ"},{"name":"Huan Liu","slug":"huan-liu","given":"Huan","family":"Liu","affiliation":"Arizona State University , Tempe , AZ"}],"abstract":"Contrastive self-supervised learning has become a prominent technique in representation learning. The main step in these methods is to contrast semantically similar and dissimilar pairs of samples. However, in the domain of Natural Language Processing (NLP), the augmentation methods used in creating similar pairs with regard to contrastive learning (CL) assumptions are challenging. This is because, even simply modifying a word in the input might change the semantic meaning of the sentence, and hence, would violate the distributional hypothesis. In this review paper, we formalize the contrastive learning framework, emphasize the considerations that need to be addressed in the data transformation step, and review the state-of-the-art methods and evaluations for contrastive representation learning in NLP. Finally, we describe some challenges and potential directions for learning better text representations using contrastive methods.","doi":"10.24963/ijcai.2022/757","url":"https://ijcai.org/proceedings/2022/757","pdf":"https://ijcai.org/proceedings/2022/0757.pdf","markdown":"https://ijcai.org/proceedings/2022/757.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/757","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Survey Track","track":"Survey Track","number":757,"pages":{"first":5394,"last":5401},"keywords":["Survey Track: Natural Language Processing","Survey Track: Machine Learning","Survey Track: Data Mining","Survey Track: -"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987308","length_sec":82,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985451","length_sec":667,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Arora et al., 2019] Sanjeev Arora, Hrishikesh Khande- parkar, Mikhail Khodak, Orestis Plevrakis, and Nikunj Saunshi. A theoretical analysis of contrastive unsupervised representation learning. In 36th ICML, pages 9904-9923. International Machine Learning Society (IMLS), 2019.","title":"A theoretical analysis of contrastive unsupervised representation learning","authors":"Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak, Orestis Plevrakis, Nikunj Saunshi","year":"2019","venue":"36th ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Batmanghelich et al., 2016] Kayhan Batmanghelich, Arda- van Saeedi, Karthik Narasimhan, and Sam Gershman. Nonparametric spherical topic modeling with word em- beddings. In Proc. of the ACL, page 537, 2016.","title":"Nonparametric spherical topic modeling with word embeddings","authors":"Batmanghelich","year":"2016","venue":"Proc. of the ACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bowman et al., 2015] Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. A large annotated corpus for learning natural language inference. In Proc. of the 2015 Conference on EMNLP, pages 632- 642, 2015.","title":"A large annotated corpus for learning natural language inference","authors":"Bowman","year":"2015","venue":"Proc. of the 2015 Conference on EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICLR, pages 1597-1607. PMLR, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chopra et al., 2005] Sumit Chopra, Raia Hadsell, and Yann LeCun. Learning a similarity metric discriminatively, with application to face verification. In 2005 IEEE Computer Society Conference on CVPR, pages 539-546, 2005.","title":"Learning a similarity metric discriminatively, with application to face verification","authors":"Sumit Chopra, Raia Hadsell, Yann Lecun","year":"2005","venue":"IEEE Computer Society Conference on CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chuang et al., 2020] Ching-Yao Chuang, Joshua Robinson, Yen-Chen Lin, Antonio Torralba, and Stefanie Jegelka. Debiased contrastive learning. In NeurIPS, 2020.","title":"Debiased contrastive learning","authors":"Ching-Yao Chuang, Joshua Robinson, Yen-Chen Lin, Antonio Torralba, Stefanie Jegelka","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dhingra et al., 2018] Bhuwan Dhingra, Christopher Shal- lue, Mohammad Norouzi, Andrew Dai, and George Dahl. Embedding text in hyperbolic spaces. In TextGraphs-12 Workshop, pages 59-69, 2018.","title":"Embedding text in hyperbolic spaces","authors":"Bhuwan Dhingra, Christopher Shallue, Mohammad Norouzi, Andrew Dai, George Dahl","year":"2018","venue":"TextGraphs-12 Workshop","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fang et al., 2020] Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, and Pengtao Xie. Cert: Contrastive self-supervised learning for language understanding. In ACL, page 7517-7523, 2020.","title":"Cert: Contrastive self-supervised learning for language understanding","authors":"Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, Pengtao Xie","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gao et al., 2021] Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simcse: Simple contrastive learning of sentence embeddings. In Proceedings of EMNLP, 2021.","title":"Simcse: Simple contrastive learning of sentence embeddings","authors":"Tianyu Gao, Xingcheng Yao, Danqi Chen","year":"2021","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Giorgi et al., 2021] John M Giorgi, Osvald Nitski, Gary D Bader, and Bo Wang. Declutr: Deep contrastive learning for unsupervised textual representations. In ACL-IJCNLP, page 879-895, 2021.","title":"Declutr: Deep contrastive learning for unsupervised textual representations","authors":"Osvald John M Giorgi, Gary D Nitski, Bo Bader, Wang","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gutmann and Hyvärinen, 2010] Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proc. of the thirteenth AISTATS, pages 297-304. JMLR Workshop and Conference Proceedings, 2010.","title":"Noise-contrastive estimation: A new estimation principle for unnormalized statistical models","authors":"Hyvärinen Gutmann, Michael Gutmann, Aapo Hyvärinen","year":"2010","venue":"JMLR Workshop and Conference Proceedings","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hadsell et al., 2006] Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on CVPR, pages 1735-1742, 2006.","title":"Dimensionality reduction by learning an invariant mapping","authors":"Raia Hadsell, Sumit Chopra, Yann Lecun","year":"2006","venue":"IEEE Computer Society Conference on CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hase et al., 2021] Peter Hase, Harry Xie, and Mohit Bansal. The out-of-distribution problem in explainability and search methods for feature importance explanations. Ad- vances in NeurIPS, 34, 2021.","title":"The out-of-distribution problem in explainability and search methods for feature importance explanations","authors":"Peter Hase, Harry Xie, Mohit Bansal","year":"2021","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[He et al., 2020] Kaiming He, Haoqi Fan, Yuxin Wu, Sain- ing Xie, and Ross Girshick. Momentum contrast for unsu- pervised visual representation learning. In Proceedings of the Conference on CVPR, pages 9729-9738, 2020.","title":"Momentum contrast for unsupervised visual representation learning","authors":"Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross Girshick","year":"2020","venue":"Proceedings of the Conference on CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hu and Li, 2021] Zhiting Hu and Li Erran Li. A causal lens for controllable text generation. NeurIPS, 34, 2021.","title":"A causal lens for controllable text generation","authors":"Li ; ] Zhiting Hu, Li Hu, Li Erran","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jiang et al., 2021] Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Improving contrastive learning on imbalanced data via open-world sampling. NeurIPS, 2021.","title":"Improving contrastive learning on imbalanced data via open-world sampling","authors":"Ziyu Jiang, Tianlong Chen, Ting Chen, Zhangyang Wang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jiao et al., 2020] Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. Tinybert: Distilling bert for natural language under- standing. In Findings of the ACL: EMNLP 2020, pages 4163-4174, 2020.","title":"Tinybert: Distilling bert for natural language understanding","authors":"Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun Liu","year":"2020","venue":"Findings of the ACL: EMNLP 2020","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Joshi and He, 2021] Nitish Joshi and He He. An investi- gation of the (in) effectiveness of counterfactually aug- mented data. arXiv preprint arXiv:2107.00753, 2021.","title":"An investigation of the (in) effectiveness of counterfactually augmented data","authors":"Nitish Joshi, He He","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kalantidis et al., 2020] Yannis Kalantidis, Mert Bulent Sariyildiz, Noe Pion, Philippe Weinzaepfel, and Diane Larlus. Hard negative mixing for contrastive learning. In 34th Conference on NeurIPS, 2020.","title":"Hard negative mixing for contrastive learning","authors":"Yannis Kalantidis, Bulent Mert, Noe Sariyildiz, Philippe Pion, Diane Weinzaepfel, Larlus","year":"2020","venue":"34th Conference on NeurIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kang et al., 2020] Bingyi Kang, Yu Li, Sa Xie, Zehuan Yuan, and Jiashi Feng. Exploring balanced feature spaces for representation learning. In ICLR, 2020.","title":"Exploring balanced feature spaces for representation learning","authors":"Bingyi Kang, Yu Li, Sa Xie, Zehuan Yuan, Jiashi Feng","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kaushik et al., 2019] Divyansh Kaushik, Eduard Hovy, and Z. Lipton. Learning the difference that makes a difference with counterfactually-augmented data. In ICLR, 2019.","title":"Learning the difference that makes a difference with counterfactually-augmented data","authors":"Kaushik","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Khosla et al., 2020] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised con- trastive learning. Advances in NeurIPS, 33, 2020.","title":"Supervised contrastive learning","authors":"Khosla","year":"2020","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kiros et al., 2015] Ryan Kiros, Yukun Zhu, Russ R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. In Ad- vances in NeurIPS, pages 3294-3302, 2015.","title":"Skip-thought vectors","authors":"Ryan Kiros, Yukun Zhu, Richard Russ R Salakhutdinov, Raquel Zemel, Antonio Urtasun, Sanja Torralba, Fidler","year":"2015","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lee et al., 2020] Seanie Lee, Dong Bok Lee, and Sung Ju Hwang. Contrastive learning with adversarial perturba- tions for conditional text generation. In ICLR, 2020.","title":"Contrastive learning with adversarial perturbations for conditional text generation","authors":"Seanie Lee, Dong Bok Lee, Sung Ju Hwang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Li et al., 2020] Junnan Li, Pan Zhou, Caiming Xiong, and Steven Hoi. Prototypical contrastive learning of unsuper- vised representations. In ICLR, 2020.","title":"Prototypical contrastive learning of unsupervised representations","authors":"Li","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Liao, 2021] Danqi Liao. Sentence embeddings us- ing supervised contrastive learning. arXiv preprint arXiv:2106.04791, 2021.","title":"Sentence embeddings using supervised contrastive learning","authors":"Danqi Liao, Liao","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Liu et al., 2021] Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang. Self- supervised learning: Generative or contrastive. IEEE Transactions on Knowledge and Data Engineering, 2021.","title":"Selfsupervised learning: Generative or contrastive","authors":"Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, Jie Tang","year":"2021","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Marelli et al., 2014] Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. A sick cure for the evaluation of com- positional distributional semantic models. In Proc. of LREC'14, pages 216-223, 2014.","title":"A sick cure for the evaluation of compositional distributional semantic models","authors":"Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, Roberto Zamparelli","year":"2014","venue":"Proc. of LREC'14","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Maudslay et al., 2019] Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, and Simone Teufel. It's all in the name: Mitigating gender bias with name-based counterfactual data substitution. In EMNLP-IJCNLP, 2019.","title":"It's all in the name: Mitigating gender bias with name-based counterfactual data substitution","authors":"Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, Simone Teufel","year":"2019","venue":"EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Meng et al., 2019] Yu Meng, Jiaxin Huang, Guangyuan Wang, Chao Zhang, Honglei Zhuang, Lance Kaplan, and Jiawei Han. Spherical text embedding. Advances in NeurIPS, 32:8208-8217, 2019.","title":"Spherical text embedding","authors":"Yu Meng, Jiaxin Huang, Guangyuan Wang, Chao Zhang, Honglei Zhuang, Lance Kaplan, Jiawei Han","year":"2019","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Meng et al., 2021] Yu Meng, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul Bennett, Jiawei Han, and Xia Song. Coco-lm: Correcting and contrasting text sequences for language model pretraining. NeurIPS, 2021.","title":"Coco-lm: Correcting and contrasting text sequences for language model pretraining","authors":"Yu Meng, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul Bennett, Jiawei Han, Xia Song","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Mikolov et al., 2013] Tomás Mikolov, Kai Chen, Greg Cor- rado, and Jeffrey Dean. Efficient estimation of word rep- resentations in vector space. In Yoshua Bengio and Yann LeCun, editors, ICLR, Workshop Track Proceedings, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Tomás Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean","year":"2013","venue":"ICLR, Workshop Track Proceedings","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Misra and Maaten, 2020] Ishan Misra and Laurens van der Maaten. Self-supervised learning of pretext-invariant rep- resentations. In Proc. of the IEEE/CVF Conference on CVPR, pages 6707-6717, 2020.","title":"Self-supervised learning of pretext-invariant representations","authors":"Maaten Misra, Ishan Misra, Laurens Van Der Maaten","year":"2020","venue":"Proc. of the IEEE/CVF Conference on CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Moraffah et al., 2020] Raha Moraffah, Mansooreh Karami, Ruocheng Guo, Adrienne Raglin, and Huan Liu. Causal interpretability for machine learning-problems, methods and evaluation. ACM SIGKDD Explorations Newsletter, 22(1):18-33, 2020.","title":"Causal interpretability for machine learning-problems, methods and evaluation","authors":"Raha Moraffah, Mansooreh Karami, Ruocheng Guo, Adrienne Raglin, Huan Liu","year":"2020","venue":"ACM SIGKDD Explorations Newsletter","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Morris et al., 2020] John Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. Textattack: A frame- work for adversarial attacks, data augmentation, and ad- versarial training in nlp. In Proc. of the 2020 Conference on EMNLP, pages 119-126, 2020.","title":"Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp","authors":"John Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, Yanjun Qi","year":"2020","venue":"Proc. of the 2020 Conference on EMNLP","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Nickel and Kiela, 2017] Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical rep- resentations. Advances in NeurIPS, 30:6338-6347, 2017.","title":"Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations","authors":"Kiela Nickel","year":"2017","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Oh Song et al., 2016] Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese. Deep metric learning via lifted structured feature embedding. In Proc. of the IEEE conference on CVPR, pages 4004-4012, 2016.","title":"Deep metric learning via lifted structured feature embedding","authors":"Hyun Oh Song, Yu Xiang, Stefanie Jegelka, Silvio Savarese","year":"2016","venue":"Proc. of the IEEE conference on CVPR","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"Proceedings of the 2014 conference on EMNLP","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Poole et al., 2019] Ben Poole, Sherjil Ozair, Aäron van den Oord, Alex Alemi, and George Tucker. On variational bounds of mutual information. In ICML, 2019.","title":"On variational bounds of mutual information","authors":"Ben Poole, Sherjil Ozair, Aäron Van Den Oord, Alex Alemi, George Tucker","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Ram et al., 2021] Ori Ram, Gal Shachaf, Omer Levy, Jonathan Berant, and Amir Globerson. Learning to retrieve passages without supervision. arXiv preprint arXiv:2112.07708, 2021.","title":"Learning to retrieve passages without supervision","authors":"Ram","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Robinson et al., 2020] Joshua David Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka. Contrastive learning with hard negative samples. In ICLR, 2020.","title":"Contrastive learning with hard negative samples","authors":"Joshua David Robinson, Ching-Yao Chuang, Suvrit Sra, Stefanie Jegelka","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Rozsa et al., 2016] Andras Rozsa, Ethan M Rudd, and Ter- rance E Boult. Adversarial diversity and hard positive gen- eration. In Proc. of the IEEE Conference on CCVPR Work- shops, pages 25-32, 2016.","title":"Adversarial diversity and hard positive generation","authors":"Andras Rozsa, Ethan M Rudd, Terrance E Boult","year":"2016","venue":"Proc. of the IEEE Conference on CCVPR Workshops","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proc. of the IEEE confer- ence on CVPR, pages 815-823, 2015.","title":"Facenet: A unified embedding for face recognition and clustering","authors":"Schroff","year":"2015","venue":"Proc. of the IEEE conference on CVPR","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Schultz and Joachims, 2004] Matthew Schultz and Thorsten Joachims. Learning a distance metric from relative comparisons. NeurIPS, 16:41-48, 2004.","title":"Learning a distance metric from relative comparisons","authors":"Matthew Schultz, Thorsten Joachims","year":"2004","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Sohn, 2016] Kihyuk Sohn. Improved deep metric learn- ing with multi-class n-pair loss objective. In Advances in NeurIPS, pages 1857-1865, 2016.","title":"Improved deep metric learning with multi-class n-pair loss objective","authors":"Kihyuk Sohn, Sohn","year":"2016","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Suh et al., 2019] Yumin Suh, Bohyung Han, Wonsik Kim, and Kyoung Mu Lee. Stochastic class-based hard example mining for deep metric learning. In Proc. of the IEEE/CVF Conference on CVPR, pages 7251-7259, 2019.","title":"Stochastic class-based hard example mining for deep metric learning","authors":"Yumin Suh, Bohyung Han, Wonsik Kim, Kyoung Mu, Lee","year":"2019","venue":"Proc. of the IEEE/CVF Conference on CVPR","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Teney et al., 2020] Damien Teney, Ehsan Abbasnedjad, and Anton van den Hengel. Learning what makes a difference from counterfactual examples and gradient supervision. In 16th ECCV, pages 580-599. Springer, 2020.","title":"Learning what makes a difference from counterfactual examples and gradient supervision","authors":"Damien Teney, Ehsan Abbasnedjad, Anton Van Den, Hengel","year":"2020","venue":"16th ECCV","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Wang and Isola, 2020] Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In ICML, pages 9929-9939. PMLR, 2020.","title":"Understanding contrastive representation learning through alignment and uniformity on the hypersphere","authors":"Wang, Isola ;, Tongzhou Wang, Phillip Isola","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Wang et al., 2014] Jiang Wang, Yang Song, Thomas Leung, Chuck Rosenberg, Jingbin Wang, James Philbin, Bo Chen, and Ying Wu. Learning fine-grained image similarity with deep ranking. In CVPR, pages 1386-1393, 2014.","title":"Learning fine-grained image similarity with deep ranking","authors":"Jiang Wang, Yang Song, Thomas Leung, Chuck Rosenberg, Jingbin Wang, James Philbin, Bo Chen, Ying Wu","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Wang et al., 2019] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In ICLR, 2019.","title":"GLUE: A multi-task benchmark and analysis platform for natural language understanding","authors":"Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, Samuel R Bowman","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Wei and Zou, 2019] Jason Wei and Kai Zou. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. In Proc. of the 2019 Conference on EMNLP-IJCNLP, pages 6382-6388, 2019.","title":"Eda: Easy data augmentation techniques for boosting performance on text classification tasks","authors":"Zou ; ] Wei, Jason Wei, Kai Zou","year":"2019","venue":"Proc. of the 2019 Conference on EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Wu et al., 2020a] Mike Wu, Milan Mosse, Chengxu Zhuang, Daniel Yamins, and Noah Goodman. Conditional negative sampling for contrastive learning of visual representations. In ICLR, 2020.","title":"Conditional negative sampling for contrastive learning of visual representations","authors":"Wu","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Wu et al., 2020b] Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, and Hao Ma. Clear: Contrastive learning for sentence representation. arXiv preprint arXiv:2012.15466, 2020.","title":"Clear: Contrastive learning for sentence representation","authors":"Wu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Wu et al., 2021] Tongshuang Wu, Marco Tulio Ribeiro, Jef- frey Heer, and Daniel S Weld. Polyjuice: Generating coun- terfactuals for explaining, evaluating, and improving mod- els. In Proceedings of the ACL, 2021.","title":"Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models","authors":"Wu","year":"2021","venue":"Proceedings of the ACL","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Xiao et al., 2020] Tete Xiao, Xiaolong Wang, Alexei A Efros, and Trevor Darrell. What should not be contrastive in contrastive learning. In ICLR, 2020.","title":"What should not be contrastive in contrastive learning","authors":"Tete Xiao, Xiaolong Wang, Alexei A Efros, Trevor Darrell","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Xie et al., 2020] Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data aug- mentation for consistency training. Advances in NeurIPS, 33:6256-6268, 2020.","title":"Unsupervised data augmentation for consistency training","authors":"Xie","year":"2020","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Xiong et al., 2020] Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N Bennett, Junaid Ahmed, and Arnold Overwijk. Approximate nearest neighbor negative contrastive learning for dense text re- trieval. In ICLR, 2020.","title":"Approximate nearest neighbor negative contrastive learning for dense text retrieval","authors":"Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N Bennett, Junaid Ahmed, Arnold Overwijk","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Xu et al., 2021] Shusheng Xu, Xingxing Zhang, Yi Wu, and Furu Wei. Sequence level contrastive learning for text summarization. arXiv preprint arXiv:2109.03481, 2021.","title":"Sequence level contrastive learning for text summarization","authors":"Xu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":59,"raw":"[Yan et al., 2021] Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. ConSERT: A contrastive framework for self-supervised sentence repre- sentation transfer. In ACL-IJCNLP, 2021.","title":"ConSERT: A contrastive framework for self-supervised sentence representation transfer","authors":"Yan","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}