What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworks

Huang, Chengrui; Shi, Zhengliang; Wen, Yuntao; Chen, Xiuying; Han, Peng; Gao, Shen; Shang, Shuo

doi:10.24963/ijcai.2026/16

What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworks

Chengrui Huang, Zhengliang Shi, Yuntao Wen, Xiuying Chen, Peng Han, Shen Gao, Shuo Shang

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Main Track. Pages 136-144.

https://doi.org/10.24963/ijcai.2026/16

Keywords

Agent-based and Multi-agent Systems: Applications
Agent-based and Multi-agent Systems: Agent communication
Data Mining: Applications

Abstract

Tool learning methods have enhanced the ability of large language models (LLMs) to interact with real-world applications.

Many existing works fine-tune LLMs or design prompts to enable LLMs to select appropriate tools and correctly invoke them to meet user requirements. However, it is observed in previous works that the performance of tool learning varies from tasks, datasets, training settings, and algorithms. Without understanding the impact of these factors, it can lead to inconsistent results, inefficient model deployment, and suboptimal tool utilization, ultimately hindering the practical integration and scalability of LLMs in real-world scenarios. Therefore, in this paper, we explore the impact of both internal and external factors on the performance of tool learning frameworks. Through extensive experiments on two benchmark datasets, we find several insightful conclusions for future work, including the observation that LLMs can benefit significantly from increased trial and exploration. We believe our empirical study provides a new perspective for future tool learning research.

Chengrui Huang, Zhengliang Shi, Yuntao Wen, Xiuying Chen, Peng Han, Shen Gao, and Shuo Shang. 2026. What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworks. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26. IJCAI, 136–144. https://doi.org/10.24963/ijcai.2026/16

@inproceedings{ijcai2026p16,
  title     = {What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworks},
  author    = {Huang, Chengrui and Shi, Zhengliang and Wen, Yuntao and Chen, Xiuying and Han, Peng and Gao, Shen and Shang, Shuo},
  booktitle = {Proceedings of the Thirty-Fifth International Joint Conference on
               Artificial Intelligence, {IJCAI-26}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Diego Calvanese},
  pages     = {136--144},
  year      = {2026},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2026/16},
  url       = {https://doi.org/10.24963/ijcai.2026/16},
}

TY  - CPAPER
TI  - What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworks
AU  - Huang, Chengrui
AU  - Shi, Zhengliang
AU  - Wen, Yuntao
AU  - Chen, Xiuying
AU  - Han, Peng
AU  - Gao, Shen
AU  - Shang, Shuo
T2  - Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2026
SP  - 136
EP  - 144
DO  - 10.24963/ijcai.2026/16
UR  - https://doi.org/10.24963/ijcai.2026/16
ER  -