---
title: "Block Circulant Adapter for Large Language Models"
authors:
  - "Xinyu Ding"
  - "Meiqi Wang"
  - "Siyu Liao"
  - "Zhongfeng Wang"
doi: "10.24963/ijcai.2025/560"
url: "https://ijcai.org/proceedings/2025/560"
pdf: "https://ijcai.org/proceedings/2025/0560.pdf"
venue: "Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence"
year: 2025
pages: "5030-5038"
track: "Main Track"
keywords:
  - "Natural Language Processing: NLP: Language models"
  - "Computer Vision: CV: Efficiency and Optimization"
  - "Machine Learning: ML: Matrix/tensor methods"
  - "Machine Learning: ML: Deep learning architectures"
---

# Block Circulant Adapter for Large Language Models

Xinyu Ding, Meiqi Wang, Siyu Liao, Zhongfeng Wang

*Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence*, pages 5030-5038. DOI: [10.24963/ijcai.2025/560](https://doi.org/10.24963/ijcai.2025/560)

## Abstract

Fine-tuning large language models (LLMs) is difficult due to their  huge model size. Recent Fourier domain-based methods show potential for reducing fine-tuning costs. 
We propose a block circulant matrix-based fine-tuning method with a stable training heuristic to leverage the properties of circulant matrices and one-dimensional Fourier transforms to reduce storage and computation costs.
Experiments show that our method uses 14× less number of parameters than VeRA, 16× smaller than LoRA and 32× less FLOPs than FourierFT,  while maintaining close or better task performance.
Our approach presents a promising way in frequency domain to fine-tune large models on downstream tasks.