{"type":"ScholarlyArticle","title":"HOUDINI: Escaping from Moderately Constrained Saddles","authors":[{"name":"Dmitrii Avdiukhin","slug":"dmitrii-avdiukhin","given":"Dmitrii","family":"Avdiukhin","affiliation":"Indiana University , Bloomington , IN"},{"name":"Grigory Yaroslavtsev","slug":"grigory-yaroslavtsev","given":"Grigory","family":"Yaroslavtsev","affiliation":"George Mason University , Fairfax , VA"}],"abstract":"We give polynomial time algorithms for escaping from high-dimensional saddle points under a moderate number of constraints. Given gradient access to a smooth function, we show that (noisy) gradient descent methods can escape from saddle points under a logarithmic number of inequality constraints. While analogous results exist for unconstrained and equality-constrained problems, we make progress on the major open question of convergence to second-order stationary points in the case of inequality constraints, without reliance on NP-oracles or altering the definitions to only account for certain constraints. Our results hold for both regular and stochastic gradient descent.","doi":"10.24963/ijcai.2023/383","url":"https://ijcai.org/proceedings/2023/383","pdf":"https://ijcai.org/proceedings/2023/0383.pdf","markdown":"https://ijcai.org/proceedings/2023/383.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/383","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":383,"pages":{"first":3442,"last":3450},"keywords":["Machine Learning: ML: Optimization"],"videos":[],"references":[{"position":1,"raw":"[Allen-Zhu and Li, 2018] Zeyuan Allen-Zhu and Yuanzhi Li. NEON2: finding local minima via first-order oracles.","title":"NEON2: finding local minima via first-order oracles","authors":"Allen-Zhu, Li ; Zeyuan Allen-Zhu, Yuanzhi Li","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Gar- nett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 3720-3730, 2018.","title":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018","authors":"Samy Bengio, Hanna M Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Allen-Zhu, 2018] Zeyuan Allen-Zhu. Natasha 2: Faster non-convex optimization than SGD. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Ad- vances in Neural Information Processing Systems 31: An- nual Conference on Neural Information Processing Sys- tems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 2680-2691, 2018.","title":"Zeyuan Allen-Zhu. Natasha 2: Faster non-convex optimization than SGD","authors":"Allen-Zhu","year":"2018","venue":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Anandkumar and Ge, 2016] Animashree Anandkumar and Rong Ge. Efficient approaches for escaping higher or- der saddle points in non-convex optimization. In Vitaly Feldman, Alexander Rakhlin, and Ohad Shamir, editors, Proceedings of the 29th Conference on Learning Theory, COLT 2016, New York, USA, June 23-26, 2016, volume 49 of JMLR Workshop and Conference Proceedings, pages 81-102. JMLR.org, 2016.","title":"Animashree Anandkumar and Rong Ge. Efficient approaches for escaping higher order saddle points in non-convex optimization","authors":"Ge Anandkumar","year":"2016","venue":"Proceedings of the 29th Conference on Learning Theory, COLT 2016","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Avdiukhin et al., 2019] Dmitrii Avdiukhin, Chi Jin, and Grigory Yaroslavtsev. Escaping saddle points with in- equality constraints via noisy sticky projected gradient de- scent. In Optimization for Machine Learning Workshop, 2019.","title":"Escaping saddle points with inequality constraints via noisy sticky projected gradient descent","authors":"Chi Dmitrii Avdiukhin, Grigory Jin, Yaroslavtsev","year":"2019","venue":"Optimization for Machine Learning Workshop","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bertsekas, 1997] Dimitri P Bertsekas. Nonlinear program- ming. Journal of the Operational Research Society, 48(3):334-334, 1997.","title":"Nonlinear programming","authors":"Dimitri P Bertsekas","year":"1997","venue":"Journal of the Operational Research Society","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bian et al., 2015a] Wei Bian, Xiaojun Chen, and Yinyu Ye. Complexity analysis of interior point algorithms for non- lipschitz and nonconvex minimization. Mathematical Pro- gramming, 149(1):301-327, 2015.","title":"Complexity analysis of interior point algorithms for nonlipschitz and nonconvex minimization","authors":"Bian","year":"2015","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Bian et al., 2015b] Wei Bian, Xiaojun Chen, and Yinyu Ye. Complexity analysis of interior point algorithms for non- lipschitz and nonconvex minimization. Mathematical Pro- gramming, 149(1):301-327, 2015.","title":"Complexity analysis of interior point algorithms for nonlipschitz and nonconvex minimization","authors":"Bian","year":"2015","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Boyd and Vandenberghe, 2004] Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge univer- sity press, 2004.","title":"Convex optimization","authors":"Vandenberghe Boyd","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Bubeck and others, 2015] Sébastien Bubeck et al. Convex optimization: Algorithms and complexity. Foundations and Trends® in Machine Learning, 8(3-4):231-357, 2015.","title":"Convex optimization: Algorithms and complexity","authors":"Sébastien Bubeck","year":"2015","venue":"Foundations and Trends® in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Carmon and Duchi, 2018] Yair Carmon and John C. Duchi. Analysis of krylov subspace solutions of regularized non- convex quadratic problems. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Con- ference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 10728-10738, 2018.","title":"Analysis of krylov subspace solutions of regularized nonconvex quadratic problems","authors":"Yair Carmon, John C Duchi","year":"2018","venue":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Carmon and Duchi, 2020] Yair Carmon and John C. Duchi. First-order methods for nonconvex quadratic minimiza- tion. SIAM Rev., 62(2):395-436, 2020.","title":"First-order methods for nonconvex quadratic minimization","authors":"Yair Carmon, John C Duchi","year":"2020","venue":"SIAM Rev","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Carmon et al., 2017] Yair Carmon, John C. Duchi, Oliver Hinder, and Aaron Sidford. \"convex until proven guilty\": Dimension-free acceleration of gradient descent on non- convex functions. In Doina Precup and Yee Whye Teh, ed- itors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 654-663. PMLR, 2017.","title":"convex until proven guilty\": Dimension-free acceleration of gradient descent on nonconvex functions","authors":"Yair Carmon, John C Duchi, Oliver Hinder, Aaron Sidford","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning, ICML 2017","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Cartis et al., 2018] Coralia Cartis, Nick IM Gould, and Philippe L Toint. Second-order optimality and beyond: Characterization and evaluation complexity in convexly constrained nonlinear optimization. Foundations of Com- putational Mathematics, 18(5):1073-1107, 2018.","title":"Second-order optimality and beyond: Characterization and evaluation complexity in convexly constrained nonlinear optimization","authors":"Coralia Cartis, Nick Im Gould, Philippe L Toint","year":"2018","venue":"Foundations of Computational Mathematics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Daneshmand et al., 2018] Hadi Daneshmand, Jonas Moritz Kohler, Aurélien Lucchi, and Thomas Hofmann. Escap- ing saddles with stochastic gradients. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th In- ternational Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 1163-1172. PMLR, 2018.","title":"Escaping saddles with stochastic gradients","authors":"Jonas Moritz Hadi Daneshmand, Aurélien Kohler, Thomas Lucchi, Hofmann","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Du et al., 2017] Simon S. Du, Chi Jin, Jason D. Lee, Michael I. Jordan, Aarti Singh, and Barnabás Póczos. Gra- dient descent can take exponential time to escape saddle points. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vish- wanathan, and Roman Garnett, editors, Advances in Neu- ral Information Processing Systems 30: Annual Confer- ence on Neural Information Processing Systems 2017, De- cember 4-9, 2017, Long Beach, CA, USA, pages 1067- 1077, 2017.","title":"Gradient descent can take exponential time to escape saddle points","authors":"Simon S Du, Chi Jin, Jason D Lee, Michael I Jordan, Aarti Singh, Barnabás Póczos","year":"2017","venue":"Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Fang et al., 2018] Cong Fang, Chris Junchi Li, Zhouchen Lin, and Tong Zhang. SPIDER: near-optimal non-convex optimization via stochastic path-integrated differential es- timator. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural In- formation Processing Systems 2018, NeurIPS 2018, De- cember 3-8, 2018, Montréal, Canada, pages 687-697, 2018.","title":"SPIDER: near-optimal non-convex optimization via stochastic path-integrated differential estimator","authors":"Cong Fang, Chris Junchi Li, Zhouchen Lin, Tong Zhang","year":"2018","venue":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ge et al., 2015] Rong Ge, Furong Huang, Chi Jin, and Yang Yuan. Escaping from saddle points -online stochastic gradient for tensor decomposition. In Peter Grünwald, Elad Hazan, and Satyen Kale, editors, Proceedings of The 28th Conference on Learning Theory, COLT 2015, Paris, France, July 3-6, 2015, volume 40 of JMLR Workshop and Conference Proceedings, pages 797-842. JMLR.org, 2015.","title":"Escaping from saddle points -online stochastic gradient for tensor decomposition","authors":"Rong Ge, Furong Huang, Chi Jin, Yang Yuan","year":"2015","venue":"Proceedings of The 28th Conference on Learning Theory, COLT 2015","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Haeser et al., 2019] Gabriel Haeser, Hongcheng Liu, and Yinyu Ye. Optimality condition and complexity analy- sis for linearly-constrained optimization without differen- tiability on the boundary. Mathematical Programming, 178(1):263-299, 2019.","title":"Optimality condition and complexity analysis for linearly-constrained optimization without differentiability on the boundary","authors":"Gabriel Haeser, Hongcheng Liu, Yinyu Ye","year":"2019","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Hsia and Sheu, 2018] Yong Hsia and Ruey-Lin Sheu. Trust region subproblem with a fixed number of additional linear inequality constraints has polynomial complexity. arXiv preprint arXiv:1312.1398, 2018.","title":"Trust region subproblem with a fixed number of additional linear inequality constraints has polynomial complexity","authors":"Yong Hsia, Ruey-Lin Sheu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Jin et al., 2017] Chi Jin, Rong Ge, Praneeth Netrapalli, Sham M. Kakade, and Michael I. Jordan. How to escape saddle points efficiently. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Con- ference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 1724-1732. PMLR, 2017.","title":"How to escape saddle points efficiently","authors":"Jin","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning, ICML 2017","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Jin et al., 2018] Chi Jin, Praneeth Netrapalli, and Michael I. Jordan. Accelerated gradient descent escapes saddle points faster than gradient descent. In Sébastien Bubeck, Vian- ney Perchet, and Philippe Rigollet, editors, Proceedings of the 31st Conference On Learning Theory, volume 75 of Proceedings of Machine Learning Research, pages 1042- 1085. PMLR, 06-09 Jul 2018.","title":"Accelerated gradient descent escapes saddle points faster than gradient descent","authors":"Jin","year":"2018","venue":"Proceedings of the 31st Conference On Learning Theory","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Jin et al., 2021] Chi Jin, Praneeth Netrapalli, Rong Ge, Sham M. Kakade, and Michael I. Jordan. On nonconvex optimization for machine learning: Gradients, stochastic- ity, and saddle points. J. ACM, 68(2):11:1-11:29, 2021.","title":"On nonconvex optimization for machine learning: Gradients, stochasticity, and saddle points","authors":"Jin","year":"2021","venue":"J. ACM","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lu et al., 2020] Songtao Lu, Meisam Razaviyayn, Bo Yang, Kejun Huang, and Mingyi Hong. Finding second-order stationary points efficiently in smooth nonconvex linearly constrained optimization problems. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.","title":"Finding second-order stationary points efficiently in smooth nonconvex linearly constrained optimization problems","authors":"Songtao Lu, Meisam Razaviyayn, Bo Yang, Kejun Huang, Mingyi Hong","year":"2020","venue":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mokhtari et al., 2018] Aryan Mokhtari, Asuman Ozdaglar, and Ali Jadbabaie. Escaping saddle points in constrained optimization. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, vol- ume 31.","title":"Escaping saddle points in constrained optimization","authors":"Aryan Mokhtari, Asuman Ozdaglar, Ali Jadbabaie","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"Curran Associates, Inc., 2018.","title":null,"authors":"Curran Associates, Inc","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Murty and Kabadi, 1987] Katta G. Murty and Santosh N. Kabadi. Some np-complete problems in quadratic and nonlinear programming. Mathematical Programming, 39:117-129, 1987.","title":"Some np-complete problems in quadratic and nonlinear programming","authors":"G Katta, Santosh N Murty, Kabadi","year":"1987","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Nesterov and Polyak, 2006] Yurii Nesterov and Boris T Polyak. Cubic regularization of newton method and its global performance. Mathematical Programming, 108(1):177-205, 2006.","title":"Cubic regularization of newton method and its global performance","authors":"Yurii Nesterov, Boris T Polyak","year":"2006","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Nocedal and Wright, 1999] Jorge Nocedal and Stephen J Wright. Numerical optimization. Springer, 1999.","title":"Numerical optimization","authors":"Jorge Nocedal, Stephen J Wright","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Nouiehed and Razaviyayn, 2020] Maher Nouiehed and Meisam Razaviyayn. A trust region method for finding second-order stationarity in linearly constrained noncon- vex optimization. SIAM J. Optim., 30(3):2501-2529, 2020.","title":"A trust region method for finding second-order stationarity in linearly constrained nonconvex optimization","authors":"Maher Nouiehed, Meisam Razaviyayn","year":"2020","venue":"SIAM J. Optim","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Nouiehed et al., 2020] Maher Nouiehed, Jason D Lee, and Meisam Razaviyayn. Convergence to second-order sta- tionarity for constrained non-convex optimization. arXiv preprint arXiv:1810.02024, 2020.","title":"Convergence to second-order stationarity for constrained non-convex optimization","authors":"Jason D Maher Nouiehed, Meisam Lee, Razaviyayn","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[O'Neill and Wright, 2020] Michael O'Neill and Stephen J Wright. A log-barrier Newton-CG method for bound con- strained optimization with complexity guarantees. IMA Journal of Numerical Analysis, 41(1):84-121, 04 2020.","title":"A log-barrier Newton-CG method for bound con-strained optimization with complexity guarantees","authors":"O' Neill, Wright ; Michael, O' Neill, Stephen J Wright","year":"2020","venue":"IMA Journal of Numerical Analysis","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Pan and Chen, 1999] Victor Y. Pan and Zhao Q. Chen. The complexity of the matrix eigenproblem. In Proceedings of the Thirty-First Annual ACM Symposium on Theory of Computing, STOC '99, page 507-516, New York, NY, USA, 1999. Association for Computing Machinery.","title":"The complexity of the matrix eigenproblem","authors":"Y Victor, Pan, Q Zhao, Chen","year":"1999","venue":"Proceedings of the Thirty-First Annual ACM Symposium on Theory of Computing, STOC '99","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Pan, 1987] V. Pan. Sequential and parallel complexity of approximate evaluation of polynomial zeros. Computers & Mathematics with Applications, 14(8):591-622, 1987.","title":"Sequential and parallel complexity of approximate evaluation of polynomial zeros","authors":"V Pan","year":"1987","venue":"Computers & Mathematics with Applications","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Staib et al., 2019] Matthew Staib, Sashank Reddi, Satyen Kale, Sanjiv Kumar, and Suvrit Sra. Escaping sad- dle points with adaptive gradient methods. In Kama- lika Chaudhuri and Ruslan Salakhutdinov, editors, Pro- ceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 5956-5965. PMLR, 09-15 Jun 2019.","title":"Escaping saddle points with adaptive gradient methods","authors":"Matthew Staib, Sashank Reddi, Satyen Kale, Sanjiv Kumar, Suvrit Sra","year":"2019","venue":"Proceedings of the 36th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Tripuraneni et al., 2018] Nilesh Tripuraneni, Mitchell Stern, Chi Jin, Jeffrey Regier, and Michael I. Jordan. Stochastic cubic regularization for fast nonconvex opti- mization. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, De- cember 3-8, 2018, Montréal, Canada, pages 2904-2913, 2018.","title":"Stochastic cubic regularization for fast nonconvex optimization","authors":"Nilesh Tripuraneni, Mitchell Stern, Chi Jin, Jeffrey Regier, Michael I Jordan","year":"2018","venue":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Xu et al., 2018] Yi Xu, Rong Jin, and Tianbao Yang. First- order stochastic algorithms for escaping from saddle points in almost linear time. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Con- ference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 5535-5545, 2018.","title":"Firstorder stochastic algorithms for escaping from saddle points in almost linear time","authors":"Yi Xu, Rong Jin, Tianbao Yang","year":"2018","venue":"Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhang and Li, 2021] Chenyi Zhang and Tongyang Li. Es- cape saddle points by a simple gradient-descent based al- gorithm. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 8545-8556.","title":"Chenyi Zhang and Tongyang Li. Escape saddle points by a simple gradient-descent based algorithm","authors":"Li Zhang","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"Curran Associates, Inc., 2021.","title":null,"authors":"Curran Associates, Inc","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhou and Gu, 2020] Dongruo Zhou and Quanquan Gu. Stochastic recursive variance-reduced cubic regularization methods. In Silvia Chiappa and Roberto Calandra, edi- tors, The 23rd International Conference on Artificial Intel- ligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy], volume 108 of Proceed- ings of Machine Learning Research, pages 3980-3990.","title":"Stochastic recursive variance-reduced cubic regularization methods","authors":"Gu Zhou, Dongruo Zhou, Quanquan Gu","year":"2020","venue":"The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"PMLR, 2020.","title":null,"authors":"Pmlr","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhou et al., 2020] Dongruo Zhou, Pan Xu, and Quanquan Gu. Stochastic nested variance reduction for nonconvex optimization. J. Mach. Learn. Res., 21:103:1-103:63, 2020.","title":"Stochastic nested variance reduction for nonconvex optimization","authors":"Zhou","year":"2020","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}