{"type":"ScholarlyArticle","title":"PILO: Principal Component-based Implicit Regularization with Low-rank Optimization for Robust Transfer Learning","authors":[{"name":"Shuaihe Liu","slug":"shuaihe-liu","given":"Shuaihe","family":"Liu","affiliation":""},{"name":"Qiugang Zhan","slug":"qiugang-zhan","given":"Qiugang","family":"Zhan","affiliation":""},{"name":"Guisong Liu","slug":"guisong-liu","given":"Guisong","family":"Liu","affiliation":""},{"name":"Tai-Xiang Jiang","slug":"tai-xiang-jiang","given":"Tai-Xiang","family":"Jiang","affiliation":""}],"abstract":"Adapting large, adversarially pre-trained models to specialized domains via transfer learning is a promising path toward building secure AI systems. However, a critical challenge arises when fine-tuning on limited downstream data: models often suffer from catastrophic forgetting of robustness, where the generalizable robust features from pre-training are erased, leading to severe robust overfitting. To address this, we first establish the underlying principle that adversarial vulnerability is not diffuse, but is concentrated in a low-dimensional subspace defined by the principal components of the model's weight matrices. We then introduce Principal component-based Implicit regularization with Low-rank Optimization (PILO), a novel parameter-efficient framework that operationalizes this insight. PILO employs a principled decoupling of learning objectives: a spectrally-guided adversarial branch, initialized via Singular Value Decomposition (SVD), performs a surgical update on the identified vulnerable subspace to enhance robustness. This is complemented by a transient, standard low-rank branch that gently adapts the model to natural data, preserving clean accuracy. This dual-branch design acts as a form of implicit spectral regularization, anchoring the model to its robust pre-trained state. Extensive experiments show that PILO significantly outperforms state-of-the-art full-parameter and parameter-efficient methods in robust accuracy across multiple benchmarks, all while reducing trainable parameters by up to 71%. Our work thus establishes a new, more effective paradigm for robust transfer learning through principled and targeted parameter optimization.","doi":"10.24963/ijcai.2026/160","url":"https://ijcai.org/proceedings/2026/160","pdf":"https://ijcai.org/proceedings/2026/0160.pdf","markdown":"https://ijcai.org/proceedings/2026/160.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/160","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Computer Vision","track":"Main Track","number":160,"pages":{"first":1433,"last":1441},"keywords":["Machine Learning: Adversarial machine learning","Computer Vision: Adversarial learning, adversarial attack and defense methods"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}