Proceedings

SHPOS: A Theoretical Guaranteed Accelerated Particle Optimization Sampling Method

Zhijian Li, Chao Zhang, Hui Qian, Xin Du, Lingwei Peng

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Main Track. Pages 2701-2707.

https://doi.org/10.24963/ijcai.2021/372

Keywords

  • Uncertainty in AI: Exact Probabilistic Inference
  • Uncertainty in AI: Approximate Probabilistic Inference
  • Machine Learning: Bayesian Learning

Abstract

Recently, the Stochastic Particle Optimization Sampling (SPOS) method is proposed to solve the particle-collapsing pitfall of deterministic Particle Variational Inference methods by ultilizing the stochastic Overdamped Langevin dynamics to enhance exploration. In this paper, we propose an accelerated particle optimization sampling method called Stochastic Hamiltonian Particle Optimization Sampling (SHPOS). Compared to the first-order dynamics used in SPOS, SHPOS adopts an augmented second-order dynamics, which involves an extra momentum term to achieve acceleration. We establish a non-asymptotic convergence analysis for SHPOS, and show that it enjoys a faster convergence rate than SPOS. Besides, we also propose a variance-reduced stochastic gradient variant of SHPOS for tasks with large-scale datasets and complex models. Experiments on both synthetic and real data validate our theory and demonstrate the superiority of SHPOS over the state-of-the-art.

Zhijian Li, Chao Zhang, Hui Qian, Xin Du, and Lingwei Peng. 2021. SHPOS: A Theoretical Guaranteed Accelerated Particle Optimization Sampling Method. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 2701–2707. https://doi.org/10.24963/ijcai.2021/372
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards