{"type":"ScholarlyArticle","title":"Federated Stochastic Bilevel Optimization with Fully First-Order Gradients","authors":[{"name":"Yihan Zhang","slug":"yihan-zhang","given":"Yihan","family":"Zhang","affiliation":""},{"name":"Rohit Dhaipule","slug":"rohit-dhaipule","given":"Rohit","family":"Dhaipule","affiliation":""},{"name":"Chiu C. Tan","slug":"chiu-c-tan","given":"Chiu C.","family":"Tan","affiliation":""},{"name":"Haibin Ling","slug":"haibin-ling","given":"Haibin","family":"Ling","affiliation":""},{"name":"Hongchang Gao","slug":"hongchang-gao","given":"Hongchang","family":"Gao","affiliation":""}],"abstract":"Federated stochastic bilevel optimization has been actively studied in recent years due to its widespread applications in machine learning. However, most existing federated stochastic bilevel optimization algorithms require the computation of second-order Hessian and Jacobian matrices, which leads to longer running times in practice. To address these challenges, we propose a novel federated stochastic variance-reduced bilevel gradient descent algorithm that relies solely on first-order oracles. Specifically, our approach does not require the computation of second-order Hessian and Jacobian matrices, significantly reducing running time.   Furthermore, we introduce a novel learning rate mechanism, i.e., a constant single-time-scale learning rate, to coordinate the update of different variables. We also present a new strategy to establish the convergence rate of our algorithm.  Finally, the extensive experimental results confirm the efficacy of our proposed algorithm.","doi":"10.24963/ijcai.2025/784","url":"https://ijcai.org/proceedings/2025/784","pdf":"https://ijcai.org/proceedings/2025/0784.pdf","markdown":"https://ijcai.org/proceedings/2025/784.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/784","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":784,"pages":{"first":7047,"last":7055},"keywords":["Machine Learning: ML: Federated learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}