{"type":"ScholarlyArticle","title":"Self-Repellent Random Walks on General Graphs - Achieving Minimal Sampling Variance via Nonlinear Markov Chains (Extended Abstract)","authors":[{"name":"Vishwaraj Doshi","slug":"vishwaraj-doshi","given":"Vishwaraj","family":"Doshi","affiliation":"IQVIA Inc."},{"name":"Jie Hu","slug":"jie-hu","given":"Jie","family":"Hu","affiliation":"North Carolina State University"},{"name":"Do Young Eun","slug":"do-young-eun","given":"Do Young","family":"Eun","affiliation":""}],"abstract":"We consider random walks on discrete state spaces, such as general undirected graphs, where the random walkers are designed to approximate a target quantity over the network topology via sampling and neighborhood exploration in the form of Markov chain Monte Carlo (MCMC) procedures. Given any Markov chain corresponding to a target probability distribution, we design a self-repellent random walk (SRRW) which is less likely to transition to nodes that were highly visited in the past, and more likely to transition to seldom visited nodes. For a class of SRRWs parameterized by a positive real α, we prove that the empirical distribution of the process converges almost surely to the target (stationary) distribution of the underlying Markov chain kernel. We then provide a central limit theorem and derive the exact form of the arising asymptotic co-variance matrix, which allows us to show that the SRRW with stronger repellence (larger α) always achieves a smaller asymptotic covariance, in the sense of Loewner ordering of co-variance matrices. Especially for SRRW-driven MCMC algorithms, we show that the decrease in the asymptotic sampling variance is of the order O(1/α), eventually going down to zero. After generalizing these results for a class of weighted empirical measures, we use them as a stepping stone to show that a similar performance ordering can also be obtained for distributed stochastic optimization tasks using token algorithms. More explicitly, by replacing a Markovian token by a SRRW version with the same target distribution, we show that the asymptotic co-variance of the optimization iterates decreases at rate O(1/α^2) - the performance benefit of using SRRW thereby amplified in the stochastic optimization context. Empirical results support our theoretical findings.","doi":"10.24963/ijcai.2024/929","url":"https://ijcai.org/proceedings/2024/929","pdf":"https://ijcai.org/proceedings/2024/0929.pdf","markdown":"https://ijcai.org/proceedings/2024/929.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/929","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Sister Conferences Best Papers","track":"Sister Conferences Best Papers","number":929,"pages":{"first":8394,"last":8398},"keywords":["Machine Learning: ML: Learning theory","Data Mining: DM: Mining graphs","Machine Learning: ML: Optimization"],"videos":[],"references":[{"position":1,"raw":"[Alon et al., 2007] Noga Alon, Itai Benjamini, Eyal Lubet- zky, and Sasha Sodin. Non-backtracking random walks mix faster. Communications in Contemporary Mathemat- ics, 9(04):585-603, 2007.","title":"Non-backtracking random walks mix faster","authors":"Itai Noga Alon, Eyal Benjamini, Sasha Lubetzky, Sodin","year":"2007","venue":"Communications in Contemporary Mathematics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andrieu and Livingstone, 2021] Christophe Andrieu and Samuel Livingstone. Peskun-tierney ordering for marko- vian monte carlo: Beyond the reversible scenario. The Annals of Statistics, 49(4):1958-1981, 2021.","title":"Peskun-tierney ordering for markovian monte carlo: Beyond the reversible scenario","authors":"Christophe Andrieu, Samuel Livingstone","year":"2021","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Andrieu et al., 2007] Christophe Andrieu, Ajay Jasra, Ar- naud Doucet, and Pierre Del Moral. Non-linear markov chain monte carlo. Esaim: Proceedings, 19:79-84, 01 2007.","title":"Non-linear markov chain monte carlo","authors":"Christophe Andrieu, Ajay Jasra, Arnaud Doucet, Pierre Del, Moral","year":"2007","venue":"Esaim: Proceedings","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Andrieu et al., 2011] Christophe Andrieu, Ajay Jasra, Ar- naud Doucet, and Pierre Del Moral. On nonlinear Markov chain Monte Carlo. Bernoulli, 17(3):987 -1014, 2011.","title":"On nonlinear Markov chain Monte Carlo","authors":"Christophe Andrieu, Ajay Jasra, Arnaud Doucet, Pierre Del, Moral","year":"2011","venue":"Bernoulli","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen and Hwang, 2013] Ting-Li Chen and Chii-Ruey Hwang. Accelerating reversible markov chains. Statistics & Probability Letters, 83(9):1956-1962, 2013.","title":"Accelerating reversible markov chains","authors":"Ting-Li Chen, Chii-Ruey Hwang","year":"2013","venue":"Statistics & Probability Letters","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Del Moral and Miclo, 2004] Pierre Del Moral and Laurent Miclo. On convergence of chains with occupational self- interactions. Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sci- ences, 460(2041):325-346, 2004.","title":"Pierre Del Moral and Laurent Miclo. On convergence of chains with occupational selfinteractions","authors":"Del Moral, Miclo","year":"2004","venue":"Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Del Moral and Miclo, 2006] Pierre Del Moral and Laurent Miclo. Self-interacting markov chains. Stochastic Analysis and Applications, 24:615-660, 07 2006.","title":"Pierre Del Moral and Laurent Miclo. Self-interacting markov chains","authors":"Del Moral, Miclo","year":"2006","venue":"Stochastic Analysis and Applications","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Diaconis et al., 2000] Persi Diaconis, Susan Holmes, and Radford M Neal. Analysis of a nonreversible markov chain sampler. Annals of Applied Probability, pages 726- 752, 2000.","title":"Analysis of a nonreversible markov chain sampler","authors":"Persi Diaconis, Susan Holmes, M Radford, Neal","year":"2000","venue":"Annals of Applied Probability","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Doshi et al., 2023] Vishwaraj Doshi, Jie Hu, and Do Young Eun. Self-repellent random walks on general graphs- achieving minimal sampling variance via nonlinear markov chains. In International Conference on Machine Learning. PMLR, 2023.","title":"Self-repellent random walks on general graphsachieving minimal sampling variance via nonlinear markov chains","authors":"Doshi","year":"2023","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Even, 2023] Mathieu Even. Stochastic gradient descent un- der markovian sampling schemes. In International Con- ference on Machine Learning, 2023.","title":"Stochastic gradient descent under markovian sampling schemes","authors":"Mathieu Even","year":"2023","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fort, 2015] Gersende Fort. Central limit theorems for stochastic approximation with controlled markov chain dynamics. ESAIM: Probability and Statistics, 19:60-80, 2015.","title":"Gersende Fort. Central limit theorems for stochastic approximation with controlled markov chain dynamics","authors":"Fort","year":"2015","venue":"ESAIM: Probability and Statistics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gadat et al., 2018] Sébastien Gadat, Fabien Panloup, and Sofiane Saadane. Stochastic heavy ball. Electronic Jour- nal of Statistics, 12:461-529, 2018.","title":"Stochastic heavy ball","authors":"Sébastien Gadat, Fabien Panloup, Sofiane Saadane","year":"2018","venue":"Electronic Journal of Statistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Harold J. Kushner, 1997] G. George Yin Harold J. Kush- ner. Stochastic Approximation Algorithms and Applica- tions. Springer, 1997.","title":"Stochastic Approximation Algorithms and Applications","authors":"G George Yin, Harold J Kushner","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hu et al., 2022] Jie Hu, Vishwaraj Doshi, and Do Young Eun. Efficiency ordering of stochastic gradient descent. In Advances in Neural Information Processing Systems, 2022.","title":"Efficiency ordering of stochastic gradient descent","authors":"Jie Hu, Vishwaraj Doshi, Do Young, Eun","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hu et al., 2024] Jie Hu, Vishwaraj Doshi, and Do Young Eun. Accelerating distributed stochastic optimization via self-repellent random walks. In International Conference on Learning Representations, 2024.","title":"Accelerating distributed stochastic optimization via self-repellent random walks","authors":"Jie Hu, Vishwaraj Doshi, Do Young, Eun","year":"2024","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lee et al., 2012] Chul-Ho Lee, Xin Xu, and Do Young Eun. Beyond Random Walk and Metropolis-Hastings Samplers: Why You Should Not Backtrack for Unbiased Graph Sampling. In Proceedings of the ACM SIGMET- RICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems, SIG- METRICS'12, pages 319-330, 2012.","title":"Beyond Random Walk and Metropolis-Hastings Samplers: Why You Should Not Backtrack for Unbiased Graph Sampling","authors":"Chul-Ho Lee, Xin Xu, Do Young, Eun","year":"2012","venue":"Proceedings of the ACM SIGMET-RICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems, SIG-METRICS'12","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2015] Rong-Hua Li, Jeffrey Xu Yu, Lu Qin, Rui Mao, and Tan Jin. On random walk based graph sampling. In 2015 IEEE 31st international conference on data engi- neering, pages 927-938. IEEE, 2015.","title":"On random walk based graph sampling","authors":"Rong-Hua Li, Jeffrey Xu Yu, Lu Qin, Rui Mao, Tan Jin","year":"2015","venue":"IEEE 31st international conference on data engineering","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2022] Tiejun Li, Tiannan Xiao, and Guoguo Yang. Revisiting the central limit theorems for the sgd-type methods. arXiv preprint arXiv:2207.11755, 2022.","title":"Revisiting the central limit theorems for the sgd-type methods","authors":"Tiejun Li, Tiannan Xiao, Guoguo Yang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ma et al., 2016] Yi-An Ma, Tianqi Chen, Lei Wu, and Emily B Fox. A unifying framework for devising ef- ficient and irreversible mcmc samplers. arXiv preprint arXiv:1608.05973, 2016.","title":"A unifying framework for devising efficient and irreversible mcmc samplers","authors":"Yi-An Ma, Tianqi Chen, Lei Wu, Emily B Fox","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Moral and Doucet, 2010] Pierre Del Moral and Arnaud Doucet. Interacting Markov chain Monte Carlo methods for solving nonlinear measure-valued equations. The An- nals of Applied Probability, 20(2):593 -639, 2010.","title":"Pierre Del Moral and Arnaud Doucet. Interacting Markov chain Monte Carlo methods for solving nonlinear measure-valued equations","authors":"Doucet Moral","year":"2010","venue":"The Annals of Applied Probability","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Neal, 2004] Radford M Neal. Improving asymptotic vari- ance of mcmc estimators: Non-reversible chains are better. arXiv preprint math/0407281, 2004.","title":"Improving asymptotic variance of mcmc estimators: Non-reversible chains are better","authors":"M Radford, Neal","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sun et al., 2018] Tao Sun, Yuejiao Sun, and Wotao Yin. On markov chain gradient descent. Advances in neural infor- mation processing systems, 31, 2018.","title":"On markov chain gradient descent","authors":"Tao Sun, Yuejiao Sun, Wotao Yin","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Thin et al., 2020] Achille Thin, Nikita Kotelevskii, Christophe Andrieu, Alain Durmus, Eric Moulines, and Maxim Panov. Nonreversible mcmc from conditional invertible transforms: a complete recipe with convergence guarantees. arXiv preprint arXiv:2012.15550, 2020.","title":"Nonreversible mcmc from conditional invertible transforms: a complete recipe with convergence guarantees","authors":"Nikita Achille Thin, Christophe Kotelevskii, Alain Andrieu, Eric Durmus, Maxim Moulines, Panov","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Turitsyn et al., 2011] Konstantin S Turitsyn, Michael Chertkov, and Marija Vucelja. Irreversible monte carlo algorithms for efficient sampling. Physica D: Nonlinear Phenomena, 240(4-5):410-414, 2011.","title":"Irreversible monte carlo algorithms for efficient sampling","authors":"Michael Konstantin S Turitsyn, Marija Chertkov, Vucelja","year":"2011","venue":"Physica D: Nonlinear Phenomena","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}