{"type":"ScholarlyArticle","title":"LoCo: Low-Rank Compositional Rotation Fine-Tuning","authors":[{"name":"An Nguyen","slug":"an-nguyen","given":"An","family":"Nguyen","affiliation":""},{"name":"Jaesik Choi","slug":"jaesik-choi","given":"Jaesik","family":"Choi","affiliation":""},{"name":"Anh Tong","slug":"anh-tong","given":"Anh","family":"Tong","affiliation":""}],"abstract":"Parameter-efficient fine-tuning (PEFT) has emerged as an critical technique for adapting large-scale foundation models across natural language processing and computer vision. While existing methods such as low-rank adaptations achieve parameter efficiency via low-rank weight updates, they are limited in their ability to preserve the geometric structure of pretrained representations. We introduce Low-rank Compositional Orthogonal fine-tuning (LoCO), a novel PEFT method that constructs orthogonal transformations through low-rank skew-symmetric matrices and compositional rotation chains. We propose an approximation scheme that enables fully parallel computation of compositional rotations, making the approach practical for high-dimensional feature spaces. Our method maintains low computational complexity while maintaining orthogonality with controlled approximation error. We validate LoCO across diverse domains, including diffusion transformer fine-tuning, vision transformer adaptation, and language model adaptation. Our approach demonstrates superior or competitive performance compared to both existing orthogonal and non-orthogonal baselines.","doi":"10.24963/ijcai.2026/521","url":"https://ijcai.org/proceedings/2026/521","pdf":"https://ijcai.org/proceedings/2026/0521.pdf","markdown":"https://ijcai.org/proceedings/2026/521.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/521","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Machine Learning","track":"Main Track","number":521,"pages":{"first":4678,"last":4686},"keywords":["Machine Learning: Generative models","Machine Learning: Learnware/model reuse/transfer learning","Natural Language Processing: Language generation"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}