{"type":"ScholarlyArticle","title":"Fine-grained Analysis of Stability and Generalization for Stochastic Bilevel Optimization","authors":[{"name":"Xuelin Zhang","slug":"xuelin-zhang","given":"Xuelin","family":"Zhang","affiliation":"College of Informatics , Huazhong Agricultural University , Wuhan 430070 , China"},{"name":"Hong Chen","slug":"hong-chen","given":"Hong","family":"Chen","affiliation":"College of Informatics , Huazhong Agricultural University , Wuhan 430070 , China; Engineering Research Center of Intelligent Technology for Agriculture , Ministry of Education , Wuhan 430070 , China"},{"name":"Bin Gu","slug":"bin-gu","given":"Bin","family":"Gu","affiliation":"School of Artificial Intelligence , Jilin University , Jilin 130012 , China"},{"name":"Tieliang Gong","slug":"tieliang-gong","given":"Tieliang","family":"Gong","affiliation":"School of Computer Science and Technology , Xi'an Jiaotong University , Xi'an 710049 , China"},{"name":"Feng Zheng","slug":"feng-zheng","given":"Feng","family":"Zheng","affiliation":"Department of Computer Science and Engineering , Southern University of Science and Technology , Shenzhen 518055 , China"}],"abstract":"Stochastic bilevel optimization (SBO) has been integrated into many machine learning paradigms recently including hyperparameter optimization, meta learning,  reinforcement learning, etc. Along with the wide range of applications, there have been abundant studies on concerning  the computing  behaviors of SBO. However, the generalization guarantees of SBO methods are far less understood from the lens of statistical learning theory. In this paper, we provide a systematical generalization analysis of the first-order gradient-based bilevel optimization methods. Firstly, we establish the quantitative connections between the on-average argument stability and the generalization gap of SBO methods. Then, we derive the upper bounds of on-average argument stability for single timescale stochastic gradient descent (SGD) and two timescale SGD, where three settings (nonconvex-nonconvex (NC-NC), convex-convex (C-C) and strongly-convex-strongly-convex (SC-SC)) are considered respectively. Experimental analysis validates our theoretical findings. \r\nCompared with the previous algorithmic stability analysis, our results do not require the re-initialization of the inner-level parameters before each iteration and are suit for more general objective functions.","doi":"10.24963/ijcai.2024/609","url":"https://ijcai.org/proceedings/2024/609","pdf":"https://ijcai.org/proceedings/2024/0609.pdf","markdown":"https://ijcai.org/proceedings/2024/609.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/609","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":609,"pages":{"first":5508,"last":5516},"keywords":["Machine Learning: ML: Learning theory"],"videos":[],"references":[{"position":1,"raw":"Fan Bao, Guoqiang Wu, Chongxuan Li, Jun Zhu, and Bo Zhang. Stability and generalization of bilevel pro- gramming in hyperparameter optimization. In Advances in Neural Information Processing Systems, pages 4529-4541, 2021.","title":"Stability and generalization of bilevel programming in hyperparameter optimization","authors":"Fan Bao, Guoqiang Wu, Chongxuan Li, Jun Zhu, Bo Zhang","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Luca Bertinetto, Joao F Henriques, Philip Torr, and Andrea Vedaldi. Meta-learning with differentiable closed-form solvers. In International Conference on Learning Repre- sentations, 2018.","title":"Meta-learning with differentiable closed-form solvers","authors":"Luca Bertinetto, Joao F Henriques, Philip Torr, Andrea Vedaldi","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"Olivier Bousquet and André Elisseeff. Stability and gener- alization. Journal of Machine Learning Research, 2:499- 526, 2002.","title":"Stability and generalization","authors":"Olivier Bousquet, André Elisseeff","year":"2002","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"Jerome Bracken and James T. McGill. Mathematical pro- grams with optimization problems in the constraints. Op- erations Research, 21(1):37-44, 1973.","title":"Mathematical programs with optimization problems in the constraints","authors":"Jerome Bracken, James T Mcgill","year":"1973","venue":"Operations Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"Tianyi Chen, Yuejiao Sun, Quan Xiao, and Wotao Yin. A single-timescale method for stochastic bilevel optimiza- tion. In International Conference on Artificial Intelligence and Statistics, pages 2466-2488. PMLR, 2022.","title":"A single-timescale method for stochastic bilevel optimization","authors":"Tianyi Chen, Yuejiao Sun, Quan Xiao, Wotao Yin","year":"2022","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"Zhun Deng, Hangfeng He, and Weijie Su. Toward better gen- eralization bounds with locally elastic stability. In Inter- national Conference on Machine Learning, pages 2590- 2600. PMLR, 2021.","title":"Toward better generalization bounds with locally elastic stability","authors":"Zhun Deng, Hangfeng He, Weijie Su","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"André Elisseeff, Theodoros Evgeniou, and Massimiliano Pontil. Stability of randomized learning algorithms. Jour- nal of Machine Learning Research, 6:55-79, 2005.","title":"Stability of randomized learning algorithms","authors":"André Elisseeff, Theodoros Evgeniou, Massimiliano Pontil","year":"2005","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"Luca Franceschi, Michele Donini, Paolo Frasconi, and Mas- similiano Pontil. Forward and reverse gradient-based hy- perparameter optimization. In International Conference on Machine Learning, pages 1165-1173. PMLR, 2017.","title":"Forward and reverse gradient-based hyperparameter optimization","authors":"Luca Franceschi, Michele Donini, Paolo Frasconi, Massimiliano Pontil","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In Inter- national Conference on Machine Learning, pages 1568- 1577. PMLR, 2018.","title":"Bilevel programming for hyperparameter optimization and meta-learning","authors":"Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, Massimiliano Pontil","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Saeed Ghadimi and Mengdi Wang. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.02246, 2018.","title":"Approximation methods for bilevel programming","authors":"Saeed Ghadimi, Mengdi Wang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"Moritz Hardt, Ben Recht, and Yoram Singer. Train faster, generalize better: Stability of stochastic gradient descent. In International Conference on Machine Learning, pages 1225-1234, 2016.","title":"Train faster, generalize better: Stability of stochastic gradient descent","authors":"Moritz Hardt, Ben Recht, Yoram Singer","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"Elad Hoffer, Itay Hubara, and Daniel Soudry. Train longer, generalize better: closing the generalization gap in large batch training of neural networks. Advances in Neural In- formation Processing Systems, 30, 2017.","title":"Train longer, generalize better: closing the generalization gap in large batch training of neural networks","authors":"Elad Hoffer, Itay Hubara, Daniel Soudry","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic. SIAM Journal on Optimization, 33(1):147- 180, 2023.","title":"A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic","authors":"Mingyi Hong, Hoi-To Wai, Zhaoran Wang, Zhuoran Yang","year":"2023","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"Kaiyi Ji, Jason D Lee, Yingbin Liang, and H Vincent Poor. Convergence of meta-learning with task-specific adapta- tion over partial parameters. Advances in Neural Informa- tion Processing Systems, 33:11490-11500, 2020.","title":"Convergence of meta-learning with task-specific adaptation over partial parameters","authors":"Kaiyi Ji, Jason D Lee, Yingbin Liang, H Vincent Poor","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"Kaiyi Ji, Junjie Yang, and Yingbin Liang. Bilevel optimiza- tion: Convergence analysis and enhanced design. In Inter- national Conference on Machine Learning, pages 4882- 4892, 2021.","title":"Bilevel optimization: Convergence analysis and enhanced design","authors":"Kaiyi Ji, Junjie Yang, Yingbin Liang","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large- batch training for deep learning: Generalization gap and sharp minima. In International Conference on Learning Representations, 2017.","title":"On largebatch training for deep learning: Generalization gap and sharp minima","authors":"Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, Ping Tak, Peter Tang","year":"2017","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"Ilja Kuzborskij and Christoph Lampert. Data-dependent sta- bility of stochastic gradient descent. In International Con- ference on Machine Learning, pages 2815-2824. PMLR, 2018.","title":"Data-dependent stability of stochastic gradient descent","authors":"Ilja Kuzborskij, Christoph Lampert","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenen- baum. Human-level concept learning through probabilistic program induction. Science, 350(6266):1332-1338, 2015.","title":"Human-level concept learning through probabilistic program induction","authors":"Ruslan Brenden M Lake, Joshua B Salakhutdinov, Tenenbaum","year":"2015","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mnist/, 1998.","title":"The mnist database of handwritten digits","authors":"Yann Lecun","year":null,"venue":null,"doi":null,"url":"http://yann.lecun.com/exdb/mnist/,1998","internal_url":null},{"position":20,"raw":"Yunwen Lei and Yiming Ying. Fine-grained analysis of sta- bility and generalization for stochastic gradient descent. In International Conference on Machine Learning, pages 5809-5819. PMLR, 2020.","title":"Fine-grained analysis of stability and generalization for stochastic gradient descent","authors":"Yunwen Lei, Yiming Ying","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"Yunwen Lei, Antoine Ledent, and Marius Kloft. Sharper generalization bounds for pairwise learning. Advances in Neural Information Processing Systems, 33:21236-21246, 2020.","title":"Sharper generalization bounds for pairwise learning","authors":"Yunwen Lei, Antoine Ledent, Marius Kloft","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"Yunwen Lei, Mingrui Liu, and Yiming Ying. Generaliza- tion guarantee of SGD for pairwise learning. In Advances in Neural Information Processing Systems, pages 21216- 21228, 2021.","title":"Generalization guarantee of SGD for pairwise learning","authors":"Yunwen Lei, Mingrui Liu, Yiming Ying","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"Yunwen Lei, Zhenhuan Yang, Tianbao Yang, and Yiming Ying. Stability and generalization of stochastic gradient methods for minimax problems. In International Con- ference on Machine Learning, pages 6175-6186. PMLR, 2021.","title":"Stability and generalization of stochastic gradient methods for minimax problems","authors":"Yunwen Lei, Zhenhuan Yang, Tianbao Yang, Yiming Ying","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"Junyi Li, Bin Gu, and Heng Huang. Improved bilevel model: Fast and optimal algorithm with theoretical guar- antee. arXiv preprint arXiv:2009.00690, 2020.","title":"Improved bilevel model: Fast and optimal algorithm with theoretical guarantee","authors":"Junyi Li, Bin Gu, Heng Huang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"Tongliang Liu, Gábor Lugosi, Gergely Neu, and Dacheng Tao. Algorithmic stability and hypothesis complexity. In International Conference on Machine Learning, pages 2159-2167. PMLR, 2017.","title":"Algorithmic stability and hypothesis complexity","authors":"Tongliang Liu, Gábor Lugosi, Gergely Neu, Dacheng Tao","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"Risheng Liu, Pan Mu, Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang. A generic first-order algorithmic framework for bi-level programming beyond lower-level singleton. In In- ternational Conference on Machine Learning, pages 6305- 6315. PMLR, 2020.","title":"A generic first-order algorithmic framework for bi-level programming beyond lower-level singleton","authors":"Risheng Liu, Pan Mu, Xiaoming Yuan, Shangzhi Zeng, Jin Zhang","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"Bo Liu, Mao Ye, Stephen Wright, Peter Stone, and Qiang Liu. Bome! bilevel optimization made easy: A simple first-order approach. Advances in Neural Information Pro- cessing Systems, 35:17248-17262, 2022.","title":"Bome! bilevel optimization made easy: A simple first-order approach","authors":"Bo Liu, Mao Ye, Stephen Wright, Peter Stone, Qiang Liu","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"Jonathan Lorraine and David Duvenaud. Stochastic hy- perparameter optimization through hypernetworks. arXiv preprint arXiv:1802.09419, 2018.","title":"Stochastic hyperparameter optimization through hypernetworks","authors":"Jonathan Lorraine, David Duvenaud","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"Matthew MacKay, Paul Vicol, Jonathan Lorraine, David Duvenaud, and Roger B. Grosse. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. In International Conference on Learning Representations, 2019.","title":"Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions","authors":"Matthew Mackay, Paul Vicol, Jonathan Lorraine, David Duvenaud, Roger B Grosse","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"Yu Nesterov. Universal gradient methods for convex opti- mization problems. Mathematical Programming, 152(1- 2):381-404, 2015.","title":"Universal gradient methods for convex optimization problems","authors":"Yu Nesterov","year":"2015","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"Takayuki Okuno, Akiko Takeda, Akihiro Kawana, and Mo- tokazu Watanabe. On lp-hyperparameter learning via bilevel nonsmooth optimization. Journal of Machine Learning Research, 22:245:1-245:47, 2021.","title":"On lp-hyperparameter learning via bilevel nonsmooth optimization","authors":"Takayuki Okuno, Akiko Takeda, Akihiro Kawana, Motokazu Watanabe","year":"2021","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"David Pfau and Oriol Vinyals. Connecting generative adver- sarial networks and actor-critic methods. arXiv preprint arXiv:1610.01945, 2016.","title":"Connecting generative adversarial networks and actor-critic methods","authors":"David Pfau, Oriol Vinyals","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. Advances in Neural Information Processing Systems, 32, 2019.","title":"Meta-learning with implicit gradients","authors":"Aravind Rajeswaran, Chelsea Finn, Sergey Sham M Kakade, Levine","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In International Conference on Machine Learning, pages 4334-4343. PMLR, 2018.","title":"Learning to reweight examples for robust deep learning","authors":"Mengye Ren, Wenyuan Zeng, Bin Yang, Raquel Urtasun","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"William Rogers and T. Wagner. A finite sample distribution- free performance bound for local discrimination rules. The Annals of Statistics, 6:506-514, 1978.","title":"A finite sample distributionfree performance bound for local discrimination rules","authors":"William Rogers, T Wagner","year":"1978","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"Wei Shen, Zhenhuan Yang, Yiming Ying, and Xiaoming Yuan. Stability and optimization error of stochastic gra- dient descent for pairwise learning. Analysis and Applica- tions, 18(05):887-927, 2020.","title":"Stability and optimization error of stochastic gradient descent for pairwise learning","authors":"Wei Shen, Zhenhuan Yang, Yiming Ying, Xiaoming Yuan","year":"2020","venue":"Analysis and Applications","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"Ingo Steinwart and Andreas Christmann. Support vector ma- chines. Springer Science & Business Media, 2008.","title":"Support vector machines","authors":"Ingo Steinwart, Andreas Christmann","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"Sebastian Thrun. Lifelong learning algorithms. In Learning to learn, pages 181-209. Springer, 1998.","title":"Lifelong learning algorithms","authors":"Sebastian Thrun","year":"1998","venue":"Learning to learn","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"Sebastian Tschiatschek, Ahana Ghosh, Luis Haug, Rati Dev- idze, and Adish Singla. Learner-aware teaching: Inverse reinforcement learning with preferences and constraints. In Advances in Neural Information Processing Systems, pages 4147-4157, 2019.","title":"Learner-aware teaching: Inverse reinforcement learning with preferences and constraints","authors":"Sebastian Tschiatschek, Ahana Ghosh, Luis Haug, Rati Devidze, Adish Singla","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"Xuelin Zhang, Yingjie Wang, Liangxuan Zhu, Hong Chen, Han Li, and Lingjuan Wu. Robust variable structure dis- covery based on tilted empirical risk minimization. Applied Intelligence, 53(14):17865-17886, 2023.","title":"Robust variable structure discovery based on tilted empirical risk minimization","authors":"Xuelin Zhang, Yingjie Wang, Liangxuan Zhu, Hong Chen, Han Li, Lingjuan Wu","year":"2023","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"Xiao Zhou, Renjie Pi, Weizhong Zhang, Yong Lin, Zonghao Chen, and Tong Zhang. Probabilistic bilevel coreset selec- tion. In International Conference on Machine Learning, pages 27287-27302. PMLR, 2022.","title":"Probabilistic bilevel coreset selection","authors":"Xiao Zhou, Renjie Pi, Weizhong Zhang, Yong Lin, Zonghao Chen, Tong Zhang","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"Yi Zhou, Yingbin Liang, and Huishuai Zhang. Understand- ing generalization error of sgd in nonconvex optimization. Machine Learning, pages 1-31, 2022.","title":"Understanding generalization error of sgd in nonconvex optimization","authors":"Yi Zhou, Yingbin Liang, Huishuai Zhang","year":"2022","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"Daniel Zügner and Stephan Günnemann. Adversarial attacks on graph neural networks via meta learning. In Interna- tional Conference on Learning Representations, 2019.","title":"Adversarial attacks on graph neural networks via meta learning","authors":"Daniel Zügner, Stephan Günnemann","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}