{"type":"ScholarlyArticle","title":"Robustifying Vision Transformer without Retraining from Scratch by Test-Time Class-Conditional Feature Alignment","authors":[{"name":"Takeshi Kojima","slug":"takeshi-kojima","given":"Takeshi","family":"Kojima","affiliation":"The University of Tokyo , Japan"},{"name":"Yutaka Matsuo","slug":"yutaka-matsuo","given":"Yutaka","family":"Matsuo","affiliation":"The University of Tokyo , Japan"},{"name":"Yusuke Iwasawa","slug":"yusuke-iwasawa","given":"Yusuke","family":"Iwasawa","affiliation":"The University of Tokyo , Japan"}],"abstract":"Vision Transformer (ViT) is becoming more popular in image processing. Specifically, we investigate the effectiveness of test-time adaptation (TTA) on ViT, a technique that has emerged to correct its prediction during test-time by itself. First, we benchmark various test-time adaptation approaches on ViT-B16 and ViT-L16. It is shown that the TTA is effective on ViT and the prior-convention (sensibly selecting modulation parameters) is not necessary when using proper loss function. Based on the observation, we propose a new test-time adaptation method called class-conditional feature alignment (CFA), which minimizes both the class-conditional distribution differences and the whole distribution differences of the hidden representation between the source and target in an online manner. Experiments of image classification tasks on common corruption (CIFAR-10-C, CIFAR-100-C, and ImageNet-C) and domain adaptation (digits datasets and ImageNet-Sketch) show that CFA stably outperforms the existing baselines on various datasets. We also verify that CFA is model agnostic by experimenting on ResNet, MLP-Mixer, and several ViT variants (ViT-AugReg, DeiT, and BeiT). Using BeiT backbone, CFA achieves 19.8% top-1 error rate on ImageNet-C, outperforming the existing test-time adaptation baseline 44.0%. This is a state-of-the-art result among TTA methods that do not need to alter training phase.","doi":"10.24963/ijcai.2022/141","url":"https://ijcai.org/proceedings/2022/141","pdf":"https://ijcai.org/proceedings/2022/0141.pdf","markdown":"https://ijcai.org/proceedings/2022/141.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/141","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":141,"pages":{"first":1009,"last":1016},"keywords":["Machine Learning: Robustness","Computer Vision: Recognition (object detection, categorization)"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986557","length_sec":88,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984700","length_sec":348,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Ba et al., 2016] Jimmy Lei Ba, Jamie Ryan Kiros, and Ge- offrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.","title":"Layer normalization","authors":"Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E Hinton","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bao et al., 2021] Hangbo Bao, Li Dong, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021.","title":"Beit: Bert pre-training of image transformers","authors":"Hangbo Bao, Li Dong, Furu Wei","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhojanapalli et al., 2021] Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Un- terthiner, and Andreas Veit. Understanding robustness of transformers for image classification. In Proceedings of the IEEE/CVF ICCV, pages 10231-10241, 2021.","title":"Understanding robustness of transformers for image classification","authors":"Bhojanapalli","year":"2021","venue":"Proceedings of the IEEE/CVF ICCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Carion et al., 2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, pages 213-229. Springer, 2020.","title":"End-to-end object detection with transformers","authors":"Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2022] Xiangning Chen, Cho-Jui Hsieh, and Bo- qing Gong. When vision transformers outperform resnets without pre-training or strong data augmentations. In ICLR, 2022.","title":"When vision transformers outperform resnets without pre-training or strong data augmentations","authors":"Xiangning Chen, Cho-Jui Hsieh, Boqing Gong","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchi- cal image database. In CVPR, pages 248-255, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Deng et al., 2019] Zhijie Deng, Yucen Luo, and Jun Zhu. Clus- ter alignment with a teacher for unsupervised domain adapta- tion. In Proceedings of ICCV, pages 9944-9953, 2019.","title":"Cluster alignment with a teacher for unsupervised domain adaptation","authors":"Zhijie Deng, Yucen Luo, Jun Zhu","year":"2019","venue":"Proceedings of ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ganin and Lempitsky, 2015] Yaroslav Ganin and Victor Lem- pitsky. Unsupervised domain adaptation by backpropagation. In ICML, pages 1180-1189. PMLR, 2015.","title":"Unsupervised domain adaptation by backpropagation","authors":"Yaroslav Ganin, Victor Lempitsky","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hendrycks and Dietterich, 2019] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to com- mon corruptions and perturbations. Proc. of the ICLR, 2019.","title":"Benchmarking neural network robustness to common corruptions and perturbations","authors":"Dietterich Hendrycks, Dan Hendrycks, Thomas Dietterich","year":"2019","venue":"Proc. of the ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hendrycks et al., 2019] Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.02781, 2019.","title":"Augmix: A simple data processing method to improve robustness and uncertainty","authors":"Dan Hendrycks, Norman Mu, D Ekin, Barret Cubuk, Justin Zoph, Balaji Gilmer, Lakshminarayanan","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hendrycks et al., 2020] Dan Hendrycks, Steven Basart, Nor- man Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of- distribution generalization. arXiv preprint arXiv:2006.16241, 2020.","title":"The many faces of robustness: A critical analysis of out-ofdistribution generalization","authors":"Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, pages 448-456.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"PMLR, 2015.","title":null,"authors":"Pmlr","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Iwasawa and Matsuo, 2021] Yusuke Iwasawa and Yutaka Mat- suo. Test-time classifier adjustment module for model- agnostic domain generalization. In Advances in NeurIPS, 2021.","title":"Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for modelagnostic domain generalization","authors":"Matsuo Iwasawa","year":"2021","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Krizhevsky and Hinton, 2009] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny im- ages. Technical report, University of Toronto, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Ben- gio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lee and others, 2013] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in repre- sentation learning, ICML, volume 3, page 896, 2013.","title":"Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks","authors":"Dong-Hyun Lee","year":"2013","venue":"Workshop on challenges in representation learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lester et al., 2021] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Proceedings of EMNLP, pages 3045-3059, 2021.","title":"The power of scale for parameter-efficient prompt tuning","authors":"Brian Lester, Rami Al-Rfou, Noah Constant","year":"2021","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liang et al., 2020] Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In ICML, pages 6028-6039. PMLR, 2020.","title":"Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation","authors":"Jian Liang, Dapeng Hu, Jiashi Feng","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2021] Yuejiang Liu, Parth Kothari, Bastien Ger- main van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. TTT++: When does self-supervised test- time training fail or thrive? In Advances in NeurIPS, 2021.","title":"TTT++: When does self-supervised testtime training fail or thrive?","authors":"Yuejiang Liu, Parth Kothari, Bastien Germain Van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, Alexandre Alahi","year":"2021","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Morrison et al., 2021] Katelyn Morrison, Benjamin Gilby, Colton Lipchak, Adam Mattioli, and Adriana Kovashka. Exploring corruption robustness: Inductive biases in vision transformers and mlp-mixers. arXiv preprint arXiv:2106.13122, 2021.","title":"Exploring corruption robustness: Inductive biases in vision transformers and mlp-mixers","authors":"Katelyn Morrison, Benjamin Gilby, Colton Lipchak, Adam Mattioli, Adriana Kovashka","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nado et al., 2020] Zachary Nado, Shreyas Padhy, D Scul- ley, Alexander D'Amour, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normaliza- tion for robustness under covariate shift. arXiv preprint arXiv:2006.10963, 2020.","title":"Evaluating prediction-time batch normalization for robustness under covariate shift","authors":"Zachary Nado, Shreyas Padhy, D Sculley, D' Alexander, Balaji Amour, Jasper Lakshminarayanan, Snoek","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Naseer et al., 2021] Muzammal Naseer, Kanchana Ranas- inghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Intriguing properties of vision trans- formers. arXiv preprint arXiv:2105.10497, 2021.","title":"Muzammal Naseer","authors":"Naseer","year":"2021","venue":"Intriguing properties of vision transformers","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Netzer et al., 2011] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop, 2011.","title":"Reading digits in natural images with unsupervised feature learning","authors":"Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y Ng","year":"2011","venue":"NIPS Workshop","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in NeurIPS, pages 8026- 8037, 2019.","title":"Pytorch: An imperative style, high-performance deep learning library","authors":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga","year":"2019","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Paul and Chen, 2021] Sayak Paul and Pin-Yu Chen. Vi- sion transformers are robust learners. arXiv preprint arXiv:2105.07581, 2(3), 2021.","title":"Vision transformers are robust learners","authors":"Chen ; Paul, Sayak Paul, Pin-Yu Chen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Rusak et al., 2020] Evgenia Rusak, Lukas Schott, Roland S Zimmermann, Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. A simple way to make neural networks robust against diverse image corruptions. In ECCV, pages 53-69. Springer, 2020.","title":"A simple way to make neural networks robust against diverse image corruptions","authors":"Evgenia Rusak, Lukas Schott, Roland S Zimmermann, Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, Wieland Brendel","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Russakovsky et al., 2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV, 115(3):211-252, 2015.","title":"Imagenet large scale visual recognition challenge","authors":"Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein","year":"2015","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Schneider et al., 2020] Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. Advances in NeurIPS, 33, 2020.","title":"Improving robustness against common corruptions by covariate shift adaptation","authors":"Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, Matthias Bethge","year":"2020","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from over- fitting. The JMLR, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"The JMLR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Steiner et al., 2021] Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lu- cas Beyer. How to train your vit? data, augmentation, and regularization in vision transformers. arXiv preprint arXiv:2106.10270, 2021.","title":"How to train your vit? data, augmentation, and regularization in vision transformers","authors":"Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, Lucas Beyer","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Tolstikhin et al., 2021] Ilya Tolstikhin, Neil Houlsby, Alexan- der Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Un- terthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, et al. Mlp-mixer: An all-mlp architecture for vision. arXiv preprint arXiv:2105.01601, 2021.","title":"Mlp-mixer: An all-mlp architecture for vision","authors":"Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Touvron et al., 2021] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distilla- tion through attention. In ICML, pages 10347-10357, 2021.","title":"Training data-efficient image transformers & distillation through attention","authors":"Touvron","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in NeurIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wang et al., 2019] Haohan Wang, Songwei Ge, Zachary Lip- ton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In Advances in NeurIPS, pages 10506-10518, 2019.","title":"Learning robust global representations by penalizing local predictive power","authors":"Wang","year":"2019","venue":"Advances in NeurIPS","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wang et al., 2020] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test- time adaptation by entropy minimization. In ICLR, 2020.","title":"Tent: Fully testtime adaptation by entropy minimization","authors":"Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, Trevor Darrell","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wightman, 2019] Ross Wightman. Pytorch image models. https://github.com/rwightman/pytorch-image-models, 2019. Accessed: 2021-12-27.","title":"Pytorch image models","authors":"Ross Wightman","year":"2021","venue":null,"doi":null,"url":"https://github.com/rwightman/pytorch-image-models,2019","internal_url":null},{"position":39,"raw":"[Xie et al., 2018] Shaoan Xie, Zibin Zheng, Liang Chen, and Chuan Chen. Learning semantic representations for unsuper- vised domain adaptation. In ICML, pages 5423-5432, 2018.","title":"Learning semantic representations for unsupervised domain adaptation","authors":"Shaoan Xie, Zibin Zheng, Liang Chen, Chuan Chen","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zellinger et al., 2017] Werner Zellinger, Thomas Grub- inger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger-Platz. Central moment discrepancy (CMD) for domain-invariant representation learning. In 5th ICLR, 2017.","title":"Central moment discrepancy (CMD) for domain-invariant representation learning","authors":"Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, Susanne Saminger-Platz","year":"2017","venue":"5th ICLR","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zeng et al., 2020] Yanhong Zeng, Jianlong Fu, and Hongyang Chao. Learning joint spatial-temporal transformations for video inpainting. In ECCV, pages 528-543. Springer, 2020.","title":"Learning joint spatial-temporal transformations for video inpainting","authors":"Yanhong Zeng, Jianlong Fu, Hongyang Chao","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhang et al., 2020] Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbabaie. Why gradient clipping accelerates train- ing: A theoretical justification for adaptivity. In ICLR, 2020.","title":"Why gradient clipping accelerates training: A theoretical justification for adaptivity","authors":"Zhang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhou et al., 2018] Luowei Zhou, Yingbo Zhou, Jason J Corso, Richard Socher, and Caiming Xiong. End-to-end dense video captioning with masked transformer. In Proceedings of the IEEE Conference on CVPR, pages 8739-8748, 2018.","title":"End-to-end dense video captioning with masked transformer","authors":"Zhou","year":"2018","venue":"Proceedings of the IEEE Conference on CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}