{"type":"ScholarlyArticle","title":"SHPOS: A Theoretical Guaranteed Accelerated Particle Optimization Sampling Method","authors":[{"name":"Zhijian Li","slug":"zhijian-li","given":"Zhijian","family":"Li","affiliation":""},{"name":"Chao Zhang","slug":"chao-zhang","given":"Chao","family":"Zhang","affiliation":""},{"name":"Hui Qian","slug":"hui-qian","given":"Hui","family":"Qian","affiliation":""},{"name":"Xin Du","slug":"xin-du","given":"Xin","family":"Du","affiliation":""},{"name":"Lingwei Peng","slug":"lingwei-peng","given":"Lingwei","family":"Peng","affiliation":""}],"abstract":"Recently, the Stochastic Particle Optimization Sampling (SPOS) method is proposed to solve the particle-collapsing pitfall of deterministic Particle Variational Inference methods by ultilizing the stochastic Overdamped Langevin dynamics to enhance exploration. In this paper, we propose an accelerated particle optimization sampling method called Stochastic Hamiltonian Particle Optimization Sampling (SHPOS). Compared to the first-order dynamics used in SPOS, SHPOS adopts an augmented second-order dynamics, which involves an extra momentum term to achieve acceleration. We establish a non-asymptotic convergence analysis for SHPOS, and show that it enjoys a faster convergence rate than SPOS. Besides, we also propose a variance-reduced stochastic gradient variant of SHPOS for tasks with large-scale datasets and complex models. Experiments on both synthetic and real data validate our theory and demonstrate the superiority of SHPOS over the state-of-the-art.","doi":"10.24963/ijcai.2021/372","url":"https://ijcai.org/proceedings/2021/372","pdf":"https://ijcai.org/proceedings/2021/0372.pdf","markdown":"https://ijcai.org/proceedings/2021/372.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/372","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Machine Learning","track":"Main Track","number":372,"pages":{"first":2701,"last":2707},"keywords":["Uncertainty in AI: Exact Probabilistic Inference","Uncertainty in AI: Approximate Probabilistic Inference","Machine Learning: Bayesian Learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}