---
title: "LoCo: Low-Rank Compositional Rotation Fine-Tuning"
authors:
  - "An Nguyen"
  - "Jaesik Choi"
  - "Anh Tong"
doi: "10.24963/ijcai.2026/521"
url: "https://ijcai.org/proceedings/2026/521"
pdf: "https://ijcai.org/proceedings/2026/0521.pdf"
venue: "Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"
year: 2026
pages: "4678-4686"
track: "Main Track"
keywords:
  - "Machine Learning: Generative models"
  - "Machine Learning: Learnware/model reuse/transfer learning"
  - "Natural Language Processing: Language generation"
---

# LoCo: Low-Rank Compositional Rotation Fine-Tuning

An Nguyen, Jaesik Choi, Anh Tong

*Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence*, pages 4678-4686. DOI: [10.24963/ijcai.2026/521](https://doi.org/10.24963/ijcai.2026/521)

## Abstract

Parameter-efficient fine-tuning (PEFT) has emerged as an critical technique for adapting large-scale foundation models across natural language processing and computer vision. While existing methods such as low-rank adaptations achieve parameter efficiency via low-rank weight updates, they are limited in their ability to preserve the geometric structure of pretrained representations. We introduce Low-rank Compositional Orthogonal fine-tuning (LoCO), a novel PEFT method that constructs orthogonal transformations through low-rank skew-symmetric matrices and compositional rotation chains. We propose an approximation scheme that enables fully parallel computation of compositional rotations, making the approach practical for high-dimensional feature spaces. Our method maintains low computational complexity while maintaining orthogonality with controlled approximation error. We validate LoCO across diverse domains, including diffusion transformer fine-tuning, vision transformer adaptation, and language model adaptation. Our approach demonstrates superior or competitive performance compared to both existing orthogonal and non-orthogonal baselines.