{"type":"ScholarlyArticle","title":"Mitigating robust overfitting via self-residual-calibration regularization (Abstract Reprint)","authors":[{"name":"Hong Liu","slug":"hong-liu","given":"Hong","family":"Liu","affiliation":"National Institute of Informatics , Tokyo"},{"name":"Zhun Zhong","slug":"zhun-zhong","given":"Zhun","family":"Zhong","affiliation":"University of Trento"},{"name":"Nicu Sebe","slug":"nicu-sebe","given":"Nicu","family":"Sebe","affiliation":"University of Trento"},{"name":"Shin'ichi Satoh","slug":"shin-ichi-satoh","given":"Shin'ichi","family":"Satoh","affiliation":"National Institute of Informatics , Tokyo; The University of Tokyo , Tokyo"}],"abstract":"Overfitting in adversarial training has attracted the interest of researchers in the community of artificial intelligence and machine learning in recent years. To address this issue, in this paper we begin by evaluating the defense performances of several calibration methods on various robust models. Our analysis and experiments reveal two intriguing properties: 1) a well-calibrated robust model is decreasing the confidence of robust model; 2) there is a trade-off between the confidences of natural and adversarial images. These new properties offer a straightforward insight into designing a simple but effective regularization, called Self-Residual-Calibration (SRC). The proposed SRC calculates the absolute residual between adversarial and natural logit features corresponding to the ground-truth labels. Furthermore, we utilize the pinball loss to minimize the quantile residual between them, resulting in more robust regularization. Extensive experiments indicate that our SRC can effectively mitigate the overfitting problem while improving the robustness of state-of-the-art models. Importantly, SRC is complementary to various regularization methods. When combined with them, we are capable of achieving the top-rank performance on the AutoAttack benchmark leaderboard.","doi":"10.24963/ijcai.2024/954","url":"https://ijcai.org/proceedings/2024/954","pdf":"https://ijcai.org/proceedings/2024/0954.pdf","markdown":"https://ijcai.org/proceedings/2024/954.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/954","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Journal Track","track":"Journal Track","number":954,"pages":{"first":8482,"last":8482},"keywords":["Journal Track: Journal Track"],"videos":[],"references":[{"position":1,"raw":"[Liu et al., 2023] Hong Liu, Zhun Zhong, Nicu Sebe, and Shin'ichi Satoh. Mitigating robust overfitting via self- residual-calibration regularization. Artificial Intelligence, 317:103877, 2023.","title":"Mitigating robust overfitting via selfresidual-calibration regularization","authors":"Liu","year":"2023","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}