Proceedings

Gradient Enhancement Task Aware Post-training Quantization

Yihua Shao, Yan Gu, Minxi Yan, Siyu Chen, Haiyang Liu, Ziyang Yan, Yongjia Li, Yan Wang, Qun Song, Hao Tang, Haotong Qin, Jingcai Guo, Nicu Sebe

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Main Track. Pages 1577-1585.

https://doi.org/10.24963/ijcai.2026/176

Keywords

  • Machine Learning: Cost-sensitive learning
  • Computer Vision: Transfer, low-shot, semi- and un- supervised learning
  • Machine Learning: Learning sparse models
  • Machine Learning: Foundation models
  • Machine Learning: Open-World/Open-Set/OOD Learning

Abstract

The huge parameters and extensive training corpora of Large Language Models (LLMs) empower them to tackle complex tasks. Yet, their massive size incurs substantial inference costs, hindering deployment on resource‑constrained edge devices. Low‑bit quantization provides a way to compress models and improve their practicality for edge-side deployment. Nonetheless, many edge applications do not require the full generalization abilities of LLMs and instead only depend on their knowledge in specific domains. This paper introduces Gradient Enhancement Task Aware Post-training Quantization, i.e., GTAQ, to address the generalization issue. Concretely, GTAQ locates task‑critical weights via gradient‑based validation and retrieval, and then amplifies these salient weights. For that purpose, GTAQ preserves and strengthens weights related to the target tasks, enabling task‑aware uniform‑bit quantization. We extensively evaluate the LLaMA family of language models on WikiText, C4, and MMLU. Our experiments show that GTAQ consistently delivers notable performance improvements across tasks, surpassing both general‑purpose and task‑aware quantization baselines. At the same time, GTAQ yields more than 3.5× speedup in model inference and substantially cuts storage requirements. https://github.com/YihuaJerry/GTAQ.git.

Yihua Shao, Yan Gu, Minxi Yan, Siyu Chen, Haiyang Liu, Ziyang Yan, Yongjia Li, Yan Wang, Qun Song, Hao Tang, Haotong Qin, Jingcai Guo, and Nicu Sebe. 2026. Gradient Enhancement Task Aware Post-training Quantization. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26. IJCAI, 1577–1585. https://doi.org/10.24963/ijcai.2026/176
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards