{"type":"ScholarlyArticle","title":"Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignment","authors":[{"name":"Peng Jin","slug":"peng-jin","given":"Peng","family":"Jin","affiliation":"School of Electronic and Computer Engineering , Peking University , Shenzhen , China; AI for Science (AI4S)-Preferred Program , Peking University Shenzhen Graduate School , Shenzhen , China"},{"name":"Hao Li","slug":"hao-li","given":"Hao","family":"Li","affiliation":"School of Electronic and Computer Engineering , Peking University , Shenzhen , China; AI for Science (AI4S)-Preferred Program , Peking University Shenzhen Graduate School , Shenzhen , China"},{"name":"Zesen Cheng","slug":"zesen-cheng","given":"Zesen","family":"Cheng","affiliation":"School of Electronic and Computer Engineering , Peking University , Shenzhen , China; AI for Science (AI4S)-Preferred Program , Peking University Shenzhen Graduate School , Shenzhen , China"},{"name":"Jinfa Huang","slug":"jinfa-huang","given":"Jinfa","family":"Huang","affiliation":"School of Electronic and Computer Engineering , Peking University , Shenzhen , China; AI for Science (AI4S)-Preferred Program , Peking University Shenzhen Graduate School , Shenzhen , China"},{"name":"Zhennan Wang","slug":"zhennan-wang","given":"Zhennan","family":"Wang","affiliation":"Peng Cheng Laboratory , Shenzhen , China"},{"name":"Li Yuan","slug":"li-yuan","given":"Li","family":"Yuan","affiliation":"School of Electronic and Computer Engineering , Peking University , Shenzhen , China; AI for Science (AI4S)-Preferred Program , Peking University Shenzhen Graduate School , Shenzhen , China; Peng Cheng Laboratory , Shenzhen , China"},{"name":"Chang Liu","slug":"chang-liu","given":"Chang","family":"Liu","affiliation":"Department of Automation and BNRist , Tsinghua University , Beijing , China"},{"name":"Jie Chen","slug":"jie-chen","given":"Jie","family":"Chen","affiliation":"School of Electronic and Computer Engineering , Peking University , Shenzhen , China; AI for Science (AI4S)-Preferred Program , Peking University Shenzhen Graduate School , Shenzhen , China; Peng Cheng Laboratory , Shenzhen , China"}],"abstract":"Text-video retrieval is a challenging cross-modal task, which aims to align visual entities with natural language descriptions. Current methods either fail to leverage the local details or are computationally expensive. What's worse, they fail to leverage the heterogeneous concepts in data. In this paper, we propose the Disentangled Conceptualization and Set-to-set Alignment (DiCoSA) to simulate the conceptualizing and reasoning process of human beings. For disentangled conceptualization, we divide the coarse feature into multiple latent factors related to semantic concepts. For set-to-set alignment, where a set of visual concepts correspond to a set of textual concepts, we propose an adaptive pooling method to aggregate semantic concepts to address the partial matching. In particular, since we encode concepts independently in only a few dimensions, DiCoSA is superior at efficiency and granularity, ensuring fine-grained interactions using a similar computational complexity as coarse-grained alignment. Extensive experiments on five datasets, including MSR-VTT, LSMDC, MSVD, ActivityNet, and DiDeMo, demonstrate that our method outperforms the existing state-of-the-art methods.","doi":"10.24963/ijcai.2023/104","url":"https://ijcai.org/proceedings/2023/104","pdf":"https://ijcai.org/proceedings/2023/0104.pdf","markdown":"https://ijcai.org/proceedings/2023/104.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/104","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":104,"pages":{"first":938,"last":946},"keywords":["Computer Vision: CV: Video analysis and understanding","Computer Vision: CV: Vision and language","Computer Vision: CV: Image and video retrieval"],"videos":[],"references":[{"position":1,"raw":"[Anne Hendricks et al., 2017] Lisa Anne Hendricks, Oliver Wang, Eli Shechtman, Josef Sivic, Trevor Darrell, and Bryan Russell. Localizing moments in video with natural language. In ICCV, pages 5803-5812, 2017.","title":"Localizing moments in video with natural language","authors":"Anne Lisa, Oliver Hendricks, Eli Wang, Josef Shechtman, Trevor Sivic, Bryan Darrell, Russell","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bain et al., 2021] Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. arXiv preprint arXiv:2104.00650, 2021.","title":"Frozen in time: A joint video and image encoder for end-to-end retrieval","authors":"Bain","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bain et al., 2022] Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. A clip-hitchhiker's guide to long video retrieval. arXiv preprint arXiv:2205.08508, 2022.","title":"A clip-hitchhiker's guide to long video retrieval","authors":"Bain","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bengio and others, 2009] Yoshua Bengio et al. Learning deep architectures for ai. Foundations and trends® in Ma- chine Learning, 2(1):1-127, 2009.","title":"Learning deep architectures for ai","authors":"Yoshua Bengio","year":"2009","venue":"Foundations and trends® in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bogolin et al., 2022] Simion-Vlad Bogolin, Ioana Croitoru, Hailin Jin, Yang Liu, and Samuel Albanie. Cross modal retrieval with querybank normalisation. In CVPR, pages 5194-5205, 2022.","title":"Cross modal retrieval with querybank normalisation","authors":"Simion-Vlad Bogolin, Ioana Croitoru, Hailin Jin, Yang Liu, Samuel Albanie","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen and Dolan, 2011] David Chen and William B Dolan. Collecting highly parallel data for paraphrase evaluation. In ACL, pages 190-200, 2011.","title":"Collecting highly parallel data for paraphrase evaluation","authors":"David Chen, William B Dolan","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2016] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: In- terpretable representation learning by information maximiz- ing generative adversarial nets. NeurIPS, 29, 2016.","title":"Infogan: Interpretable representation learning by information maximizing generative adversarial nets","authors":"Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2020] Shizhe Chen, Yida Zhao, Qin Jin, and Qi Wu. Fine-grained video-text retrieval with hierarchical graph reasoning. In CVPR, pages 10638-10647, 2020.","title":"Fine-grained video-text retrieval with hierarchical graph reasoning","authors":"Chen","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cheng et al., 2023] Zesen Cheng, Kehan Li, Peng Jin, Xi- angyang Ji, Li Yuan, Chang Liu, and Jie Chen. Parallel vertex diffusion for unified visual grounding. arXiv preprint arXiv:2303.07216, 2023.","title":"Parallel vertex diffusion for unified visual grounding","authors":"Cheng","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Croitoru et al., 2021] Ioana Croitoru, Simion-Vlad Bogolin, Marius Leordeanu, Hailin Jin, Andrew Zisserman, Samuel Albanie, and Yang Liu. Teachtext: Crossmodal generalized distillation for text-video retrieval. In ICCV, pages 11583- 11593, 2021.","title":"Teachtext: Crossmodal generalized distillation for text-video retrieval","authors":"Ioana Croitoru, Simion-Vlad Bogolin, Marius Leordeanu, Hailin Jin, Andrew Zisserman, Samuel Albanie, Yang Liu","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Do and Tran, 2019] Kien Do and Truyen Tran. Theory and evaluation metrics for learning disentangled representations. arXiv preprint arXiv:1908.09961, 2019.","title":"Kien Do and Truyen Tran. Theory and evaluation metrics for learning disentangled representations","authors":"Tran Do","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Dosovitskiy et al., 2021] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transform- ers for image recognition at scale. In ICLR, 2021.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Eastwood and Williams, 2018] Cian Eastwood and Christo- pher KI Williams. A framework for the quantitative evalua- tion of disentangled representations. In ICLR, 2018.","title":"A framework for the quantitative evaluation of disentangled representations","authors":"Cian Eastwood, K I Christopher, Williams","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Fang et al., 2021] Han Fang, Pengfei Xiong, Luhui Xu, and Yu Chen. Clip2video: Mastering video-text retrieval via image clip. arXiv preprint arXiv:2106.11097, 2021.","title":"Clip2video: Mastering video-text retrieval via image clip","authors":"Han Fang, Pengfei Xiong, Luhui Xu, Yu Chen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gabeur et al., 2020] Valentin Gabeur, Chen Sun, Karteek Alahari, and Cordelia Schmid. Multi-modal transformer for video retrieval. In ECCV, volume 12349, pages 214-229, 2020.","title":"Multi-modal transformer for video retrieval","authors":"Valentin Gabeur, Chen Sun, Karteek Alahari, Cordelia Schmid","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gorti et al., 2022] Satya Krishna Gorti, Noël Vouitsis, Jun- wei Ma, Keyvan Golestan, Maksims Volkovs, Animesh Garg, and Guangwei Yu. X-pool: Cross-modal language- video attention for text-video retrieval. In CVPR, pages 5006-5015, 2022.","title":"X-pool: Cross-modal languagevideo attention for text-video retrieval","authors":"Krishna Satya, Noël Gorti, Junwei Vouitsis, Keyvan Ma, Maksims Golestan, Animesh Volkovs, Guangwei Garg, Yu","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jin et al., 2022] Peng Jin, Jinfa Huang, Fenglin Liu, Xian Wu, Shen Ge, Guoli Song, David Clifton, and Jie Chen. Expectation-maximization contrastive learning for compact video-and-language representations. NeurIPS, 35:30291- 30306, 2022.","title":"Expectation-maximization contrastive learning for compact video-and-language representations","authors":"Jin Peng, Jinfa Huang, Fenglin Liu, Xian Wu, Shen Ge, Guoli Song, David Clifton, Jie Chen","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Jin et al., 2023a] Peng Jin, Jinfa Huang, Pengfei Xiong, Shangxuan Tian, Chang Liu, Xiangyang Ji, Li Yuan, and Jie Chen. Video-text as game players: Hierarchical banzhaf in- teraction for cross-modal representation learning. In CVPR, 2023.","title":"Video-text as game players: Hierarchical banzhaf interaction for cross-modal representation learning","authors":"Jin","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jin et al., 2023b] Peng Jin, Hao Li, Zesen Cheng, Kehan Li, Xiangyang Ji, Chang Liu, Li Yuan, and Jie Chen. Diffusion- ret: Generative text-video retrieval with diffusion model. arXiv preprint arXiv:2303.09867, 2023.","title":"Diffusionret: Generative text-video retrieval with diffusion model","authors":"Jin","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Krishna et al., 2017] Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. Dense-captioning events in videos. In ICCV, pages 706-715, 2017.","title":"Dense-captioning events in videos","authors":"Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, Juan Carlos Niebles","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kuo and Sloan, 2005] Frances Y Kuo and Ian H Sloan. Lift- ing the curse of dimensionality. Notices of the AMS, 52(11):1320-1328, 2005.","title":"Lifting the curse of dimensionality","authors":"Y Frances, Ian H Kuo, Sloan","year":"2005","venue":"Notices of the AMS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lei et al., 2021] Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, and Jingjing Liu. Less is more: Clipbert for video-and-language learning via sparse sampling. In CVPR, pages 7331-7341, 2021.","title":"Less is more: Clipbert for video-and-language learning via sparse sampling","authors":"Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L Berg, Mohit Bansal, Jingjing Liu","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Li et al., 2022a] Hao Li, Jinfa Huang, Peng Jin, Guoli Song, Qi Wu, and Jie Chen. Toward 3d spatial reasoning for human-like text-based visual question answering. arXiv preprint arXiv:2209.10326, 2022.","title":"Toward 3d spatial reasoning for human-like text-based visual question answering","authors":"Li","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Li et al., 2022b] Hao Li, Xu Li, Belhal Karimi, Jie Chen, and Mingming Sun. Joint learning of object graph and relation graph for visual question answering. In ICME, pages 01-06. IEEE, 2022.","title":"Joint learning of object graph and relation graph for visual question answering","authors":"Li","year":"2022","venue":"ICME","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Li et al., 2022c] Kehan Li, Runyi Yu, Zhennan Wang, Li Yuan, Guoli Song, and Jie Chen. Locality guidance for improving vision transformers on tiny datasets. In ECCV, pages 110-127. Springer, 2022.","title":"Locality guidance for improving vision transformers on tiny datasets","authors":"Li","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Liu et al., 2019] Yang Liu, Samuel Albanie, Arsha Nagrani, and Andrew Zisserman. Use what you have: Video retrieval using representations from collaborative experts. In BMVC, page 279, 2019.","title":"Use what you have: Video retrieval using representations from collaborative experts","authors":"Yang Liu, Samuel Albanie, Arsha Nagrani, Andrew Zisserman","year":"2019","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Liu et al., 2021] Yang Liu, Qingchao Chen, and Samuel Al- banie. Adaptive cross-modal prototypes for cross-domain visual-language retrieval. In CVPR, pages 14954-14964, 2021.","title":"Adaptive cross-modal prototypes for cross-domain visual-language retrieval","authors":"Yang Liu, Qingchao Chen, Samuel Albanie","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Liu et al., 2022] Yuqi Liu, Pengfei Xiong, Luhui Xu, Sheng- ming Cao, and Qin Jin. Ts2-net: Token shift and selection transformer for text-video retrieval. In ECCV, pages 319- 335. Springer, 2022.","title":"Ts2-net: Token shift and selection transformer for text-video retrieval","authors":"Yuqi Liu, Pengfei Xiong, Luhui Xu, Shengming Cao, Qin Jin","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Locatello et al., 2019] Francesco Locatello, Michael Tschannen, Stefan Bauer, Gunnar Rätsch, Bernhard Schölkopf, and Olivier Bachem. Disentangling fac- tors of variation using few labels. arXiv preprint arXiv:1905.01258, 2019.","title":"Disentangling factors of variation using few labels","authors":"Francesco Locatello, Michael Tschannen, Stefan Bauer, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Luo et al., 2022] Huaishao Luo, Lei Ji, Ming Zhong, Yang Chen, Wen Lei, Nan Duan, and Tianrui Li. Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning. Neurocomputing, 508:293-304, 2022.","title":"Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning","authors":"Huaishao Luo, Lei Ji, Ming Zhong, Yang Chen, Wen Lei, Nan Duan, Tianrui Li","year":"2022","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Ma et al., 2019] Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. Disentangled graph convolutional networks. In ICML, pages 4212-4221, 2019.","title":"Disentangled graph convolutional networks","authors":"Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, Wenwu Zhu","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Oord et al., 2018] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.","title":"Representation learning with contrastive predictive coding","authors":"Aaron Van Den Oord, Yazhe Li, Oriol Vinyals","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Patrick et al., 2021] Mandela Patrick, Po-Yao Huang, Yuki Asano, Florian Metze, Alexander G Hauptmann, Joao F. Henriques, and Andrea Vedaldi. Support-set bottlenecks for video-text representation learning. In ICLR, 2021.","title":"Support-set bottlenecks for video-text representation learning","authors":"Patrick","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transfer- able visual models from natural language supervision. In ICML, pages 8748-8763, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Ridgeway and Mozer, 2018] Karl Ridgeway and Michael C Mozer. Learning deep disentangled embeddings with the f-statistic loss. NeurIPS, 31, 2018.","title":"Learning deep disentangled embeddings with the f-statistic loss","authors":"Mozer Ridgeway, Karl Ridgeway, Michael C Mozer","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Rohrbach et al., 2015] Anna Rohrbach, Marcus Rohrbach, Niket Tandon, and Bernt Schiele. A dataset for movie description. In CVPR, pages 3202-3212, 2015.","title":"A dataset for movie description","authors":"Anna Rohrbach, Marcus Rohrbach, Niket Tandon, Bernt Schiele","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Sreekar et al., 2021] P Aditya Sreekar, Ujjwal Tiwari, and Anoop Namboodiri. Mutual information based method for unsupervised disentanglement of video representation. In ICPR, pages 6396-6403, 2021.","title":"Mutual information based method for unsupervised disentanglement of video representation","authors":"Aditya Sreekar, Ujjwal Tiwari, Anoop Namboodiri","year":"2021","venue":"ICPR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Suter et al., 2019] Raphael Suter, Djordje Miladinovic, Bern- hard Schölkopf, and Stefan Bauer. Robustly disentangled causal mechanisms: Validating deep representations for in- terventional robustness. In ICML, pages 6056-6065, 2019.","title":"Robustly disentangled causal mechanisms: Validating deep representations for interventional robustness","authors":"Raphael Suter, Djordje Miladinovic, Bernhard Schölkopf, Stefan Bauer","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Tran et al., 2017] Luan Tran, Xi Yin, and Xiaoming Liu. Dis- entangled representation learning gan for pose-invariant face recognition. In CVPR, pages 1415-1424, 2017.","title":"Disentangled representation learning gan for pose-invariant face recognition","authors":"Luan Tran, Xi Yin, Xiaoming Liu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Van Steenkiste et al., 2019] Sjoerd Van Steenkiste, Francesco Locatello, Jürgen Schmidhuber, and Olivier Bachem. Are disentangled representations helpful for abstract visual reasoning? NeurIPS, 32, 2019.","title":"Are disentangled representations helpful for abstract visual reasoning?","authors":"Sjoerd Van Steenkiste, Francesco Locatello, Jürgen Schmidhuber, Olivier Bachem","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, volume 30, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wang et al., 2021a] Tan Wang, Zhongqi Yue, Jianqiang Huang, Qianru Sun, and Hanwang Zhang. Self-supervised learning disentangled group representation as feature. NeurIPS, 34, 2021.","title":"Self-supervised learning disentangled group representation as feature","authors":"Wang","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wang et al., 2021b] Xiaohan Wang, Linchao Zhu, and Yi Yang. T2vlad: Global-local sequence alignment for text-video retrieval. In CVPR, pages 5079-5088, 2021.","title":"T2vlad: Global-local sequence alignment for text-video retrieval","authors":"Wang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Wang et al., 2022] Qiang Wang, Yanhao Zhang, Yun Zheng, Pan Pan, and Xian-Sheng Hua. Disentangled represen- tation learning for text-video retrieval. arXiv preprint arXiv:2203.07111, 2022.","title":"Disentangled representation learning for text-video retrieval","authors":"Qiang Wang, Yanhao Zhang, Yun Zheng, Pan Pan, Xian-Sheng Hua","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Wray et al., 2019] Michael Wray, Gabriela Csurka, Diane Larlus, and Dima Damen. Fine-grained action retrieval through multiple parts-of-speech embeddings. In ICCV, pages 450-459, 2019.","title":"Fine-grained action retrieval through multiple parts-of-speech embeddings","authors":"Michael Wray, Gabriela Csurka, Diane Larlus, Dima Damen","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Xu et al., 2016] Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. In CVPR, pages 5288-5296, 2016.","title":"Msr-vtt: A large video description dataset for bridging video and language","authors":"Jun Xu, Tao Mei, Ting Yao, Yong Rui","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Xue et al., 2022] Hongwei Xue, Yuchong Sun, Bei Liu, Jianlong Fu, Ruihua Song, Houqiang Li, and Jiebo Luo. Clip-vip: Adapting pre-trained image-text model to video-language representation alignment. arXiv preprint arXiv:2209.06430, 2022.","title":"Clip-vip: Adapting pre-trained image-text model to video-language representation alignment","authors":"Yuchong Hongwei Xue, Bei Sun, Jianlong Liu, Ruihua Fu, Houqiang Song, Jiebo Li, Luo","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Ye et al., 2023] Qichen Ye, Bowen Cao, Nuo Chen, Weiyuan Xu, and Yuexian Zou. Fits: Fine-grained two-stage training for knowledge-aware question answering. arXiv preprint arXiv:2302.11799, 2023.","title":"Fits: Fine-grained two-stage training for knowledge-aware question answering","authors":"Qichen Ye, Bowen Cao, Nuo Chen, Weiyuan Xu, Yuexian Zou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Zhang et al., 2021] Shaofeng Zhang, Feng Zhu, Junchi Yan, Rui Zhao, and Xiaokang Yang. Zero-cl: Instance and fea- ture decorrelation for negative-free symmetric contrastive learning. In ICLR, 2021.","title":"Zero-cl: Instance and feature decorrelation for negative-free symmetric contrastive learning","authors":"Shaofeng Zhang, Feng Zhu, Junchi Yan, Rui Zhao, Xiaokang Yang","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Zhang et al., 2022a] Shaofeng Zhang, Meng Liu, Junchi Yan, Hengrui Zhang, Lingxiao Huang, Xiaokang Yang, and Pinyan Lu. M-mix: Generating hard negatives via multi-sample mixing for contrastive learning. In KDD, pages 2461-2470, 2022.","title":"M-mix: Generating hard negatives via multi-sample mixing for contrastive learning","authors":"Zhang","year":"2022","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Zhang et al., 2022b] Shaofeng Zhang, Lyn Qiu, Feng Zhu, Junchi Yan, Hengrui Zhang, Rui Zhao, Hongyang Li, and Xiaokang Yang. Align representations with base: A new ap- proach to self-supervised learning. In CVPR, pages 16600- 16609, 2022.","title":"Align representations with base: A new approach to self-supervised learning","authors":"Zhang","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Zhang et al., 2023] Shaofeng Zhang, Feng Zhu, Rui Zhao, and Junchi Yan. Patch-level contrasting without patch corre- spondence for accurate and dense contrastive representation learning. In ICLR, 2023.","title":"Patch-level contrasting without patch correspondence for accurate and dense contrastive representation learning","authors":"Shaofeng Zhang, Feng Zhu, Rui Zhao, Junchi Yan","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}