Riemannian Stochastic Recursive Momentum Method for non-Convex Optimization

Han, Andi; Gao, Junbin

doi:10.24963/ijcai.2021/345

Riemannian Stochastic Recursive Momentum Method for non-Convex Optimization

Andi Han, Junbin Gao

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Main Track. Pages 2505-2511.

https://doi.org/10.24963/ijcai.2021/345

Keywords

Machine Learning: Online Learning

Abstract

We propose a stochastic recursive momentum method for Riemannian non-convex optimization that achieves a nearly-optimal complexity to find epsilon-approximate solution with one sample. The new algorithm requires one-sample gradient evaluations per iteration and does not require restarting with a large batch gradient, which is commonly used to obtain a faster rate. Extensive experiment results demonstrate the superiority of the proposed algorithm. Extensions to nonsmooth and constrained optimization settings are also discussed.

Andi Han, and Junbin Gao. 2021. Riemannian Stochastic Recursive Momentum Method for non-Convex Optimization. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 2505–2511. https://doi.org/10.24963/ijcai.2021/345

@inproceedings{ijcai2021p345,
  title     = {Riemannian Stochastic Recursive Momentum Method for non-Convex Optimization},
  author    = {Han, Andi and Gao, Junbin},
  booktitle = {Proceedings of the Thirtieth International Joint Conference on
               Artificial Intelligence, {IJCAI-21}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Prof. Zhi-Hua Zhou},
  pages     = {2505--2511},
  year      = {2021},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2021/345},
  url       = {https://doi.org/10.24963/ijcai.2021/345},
}

TY  - CPAPER
TI  - Riemannian Stochastic Recursive Momentum Method for non-Convex Optimization
AU  - Han, Andi
AU  - Gao, Junbin
T2  - Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2021
SP  - 2505
EP  - 2511
DO  - 10.24963/ijcai.2021/345
UR  - https://doi.org/10.24963/ijcai.2021/345
ER  -