{"type":"ScholarlyArticle","title":"Escaping Saddle Point Efficiently in Minimax and Bilevel Optimizations","authors":[{"name":"Wenhan Xian","slug":"wenhan-xian","given":"Wenhan","family":"Xian","affiliation":""},{"name":"Feihu Huang","slug":"feihu-huang","given":"Feihu","family":"Huang","affiliation":""},{"name":"Heng Huang","slug":"heng-huang","given":"Heng","family":"Huang","affiliation":""}],"abstract":"Hierarchical optimization is attracting significant attentions as it can be applied to a broad range of machine learning tasks. Recently, many algorithms are proposed to improve the theoretical results of minimax and bilevel optimizations. Among these works, a core issue that has not been well studies is to escape saddle point and find local minimum. In this paper, thus, we investigate the methods to achieve second-order optimality for nonconvex minimax and bilevel optimization. Specifically, we propose a new algorithm named PRGDA without the computation of second order derivative of the primal function. In nonconvex-strongly-concave minimax optimization, we prove that our algorithm can find a second-order stationary point with the gradient complexity that matches state-of-the-art result to find first-order stationary point. To our best knowledge, PRGDA is the first stochastic algorithm that is guaranteed to obtain the second-order stationary point for nonconvex minimax problems. In nonconvex-strongly-convex bilevel optimization, our method also achieves better gradient complexity to find local minimum. Finally, we conduct two numerical experiments to validate the performance of our new method.","doi":"10.24963/ijcai.2025/741","url":"https://ijcai.org/proceedings/2025/741","pdf":"https://ijcai.org/proceedings/2025/0741.pdf","markdown":"https://ijcai.org/proceedings/2025/741.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/741","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":741,"pages":{"first":6659,"last":6668},"keywords":["Machine Learning: ML: Optimization"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}