Proceedings

Federated Stochastic Bilevel Optimization with Fully First-Order Gradients

Yihan Zhang, Rohit Dhaipule, Chiu C. Tan, Haibin Ling, Hongchang Gao

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 7047-7055.

https://doi.org/10.24963/ijcai.2025/784

Keywords

  • Machine Learning: ML: Federated learning

Abstract

Federated stochastic bilevel optimization has been actively studied in recent years due to its widespread applications in machine learning. However, most existing federated stochastic bilevel optimization algorithms require the computation of second-order Hessian and Jacobian matrices, which leads to longer running times in practice. To address these challenges, we propose a novel federated stochastic variance-reduced bilevel gradient descent algorithm that relies solely on first-order oracles. Specifically, our approach does not require the computation of second-order Hessian and Jacobian matrices, significantly reducing running time. Furthermore, we introduce a novel learning rate mechanism, i.e., a constant single-time-scale learning rate, to coordinate the update of different variables. We also present a new strategy to establish the convergence rate of our algorithm. Finally, the extensive experimental results confirm the efficacy of our proposed algorithm.

Yihan Zhang, Rohit Dhaipule, Chiu C. Tan, Haibin Ling, and Hongchang Gao. 2025. Federated Stochastic Bilevel Optimization with Fully First-Order Gradients. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 7047–7055. https://doi.org/10.24963/ijcai.2025/784
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards