{"type":"ScholarlyArticle","title":"Bayesian Optimization with Switching Cost: Regret Analysis and Lookahead Variants","authors":[{"name":"Peng Liu","slug":"peng-liu","given":"Peng","family":"Liu","affiliation":"Singapore Management University"},{"name":"Haowei Wang","slug":"haowei-wang","given":"Haowei","family":"Wang","affiliation":"Rice-Rick Digitalization"},{"name":"Wei Qiyu","slug":"wei-qiyu","given":"Wei","family":"Qiyu","affiliation":"Shanghai University"}],"abstract":"Bayesian Optimization (BO) has recently received increasing attention due to its efficiency in optimizing expensive-to-evaluate functions.  For some practical problems, it is essential to consider the path-dependent switching cost between consecutive sampling locations given a total traveling budget. For example, when using a drone to locate cracks in a building wall or search for lost survivors in the wild, the search path needs to be efficiently planned given the limited battery power of the drone. Tackling such problems requires a careful cost-benefit analysis of candidate locations and balancing exploration and exploitation.  In this work, we formulate such a problem as a constrained Markov Decision Process (MDP) and solve it by proposing a new distance-adjusted multi-step look-ahead acquisition function, the distUCB, and using rollout approximation. We also provide a theoretical regret analysis of the distUCB-based Bayesian optimization algorithm. In addition, the empirical performance of the proposed algorithm is tested based on both synthetic and real data experiments, and it shows that our cost-aware non-myopic algorithm performs better than other popular alternatives.","doi":"10.24963/ijcai.2023/446","url":"https://ijcai.org/proceedings/2023/446","pdf":"https://ijcai.org/proceedings/2023/0446.pdf","markdown":"https://ijcai.org/proceedings/2023/446.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/446","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":446,"pages":{"first":4011,"last":4018},"keywords":["Machine Learning: ML: Hyperparameter optimization","Machine Learning: ML: Bayesian learning"],"videos":[],"references":[{"position":1,"raw":"[Balandat et al., 2019] Maximilian Balandat, Brian Karrer, Daniel R Jiang, Samuel Daulton, Benjamin Letham, An- drew Gordon Wilson, and Eytan Bakshy. Botorch: Pro- grammable bayesian optimization in pytorch. arXiv preprint arXiv:1910.06403, 117, 2019.","title":"Botorch: Programmable bayesian optimization in pytorch","authors":"Maximilian Balandat, Brian Karrer, R Daniel, Samuel Jiang, Benjamin Daulton, Andrew Letham, Eytan Gordon Wilson, Bakshy","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bertsekas et al., 1997] Dimitri Bertsekas, John Tsitsiklis, and Cynara Wu. Rollout algorithms for combinatorial op- timization. Journal of Heuristics, 3:245-262, 11 1997.","title":"Rollout algorithms for combinatorial optimization","authors":"Dimitri Bertsekas, John Tsitsiklis, Cynara Wu","year":"1997","venue":"Journal of Heuristics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Gardner et al., 2014] Jacob R Gardner, Matt J Kusner, Zhix- iang Eddie Xu, Kilian Q Weinberger, and John P Cunning- ham. Bayesian optimization with inequality constraints. In ICML, volume 2014, pages 937-945, 2014.","title":"Bayesian optimization with inequality constraints","authors":"Matt J Jacob R Gardner, Zhixiang Kusner, Eddie Xu, Kilian Q Weinberger, John P Cunningham","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Goodson et al., 2016] Justin Goodson, Barrett Thomas, and Jeffrey Ohlmann. A rollout algorithm framework for heuristic solutions to finite-horizon stochastic dynamic programs. European Journal of Operational Research, 258, 10 2016.","title":"A rollout algorithm framework for heuristic solutions to finite-horizon stochastic dynamic programs","authors":"Goodson","year":"2016","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hernández-Lobato et al., 2014] José Hernández-Lobato, Matthew Hoffman, and Zoubin Ghahramani. Predic- tive entropy search for efficient global optimization of black-box functions. Advances in Neural Information Processing Systems, 1, 06 2014.","title":"Predictive entropy search for efficient global optimization of black-box functions","authors":"Hernández-Lobato","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hu et al., 2022] Shouri Hu, Haowei Wang, Zhongxiang Dai, Bryan Kian Hsiang Low, and Szu Hui Ng. Ad- justed expected improvement for cumulative regret min- imization in noisy bayesian optimization. arXiv preprint arXiv:2205.04901, 2022.","title":"Adjusted expected improvement for cumulative regret minimization in noisy bayesian optimization","authors":"Shouri Hu, Haowei Wang, Zhongxiang Dai, Bryan Kian Hsiang Low, Szu Hui Ng","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jiang et al., 2020] Shali Jiang, Daniel Jiang, Maximilian Balandat, Brian Karrer, Jacob Gardner, and Roman Gar- nett. Efficient nonmyopic bayesian optimization via one- shot multi-step trees. Advances in Neural Information Pro- cessing Systems, 2020.","title":"Efficient nonmyopic bayesian optimization via oneshot multi-step trees","authors":"Shali Jiang, Daniel Jiang, Maximilian Balandat, Brian Karrer, Jacob Gardner, Roman Garnett","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lam and Willcox, 2017] Remi Lam and Karen Willcox. Lookahead bayesian optimization with inequality con- straints. Advances in neural information processing sys- tems, 12 2017.","title":"Lookahead bayesian optimization with inequality constraints","authors":"Willcox Lam, Remi Lam, Karen Willcox","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lee et al., 2021] Eric Hans Lee, David Eriksson, Valerio Perrone, and Matthias W. Seeger. A nonmyopic approach to cost-constrained bayesian optimization. 37th Confer- ence on Uncertainty in Artificial Intelligence, 2021.","title":"A nonmyopic approach to cost-constrained bayesian optimization","authors":"Eric Hans, Lee, David Eriksson, Matthias W Valerio Perrone, Seeger","year":"2021","venue":"37th Conference on Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Marchant and Ramos, 2012] Roman Marchant and Fabio Ramos. Bayesian optimisation for intelligent environmen- tal monitoring. In 2012 IEEE/RSJ international confer- ence on intelligent robots and systems, pages 2242-2249, 10 2012.","title":"Roman Marchant and Fabio Ramos. Bayesian optimisation for intelligent environmental monitoring","authors":"Ramos Marchant","year":"2012","venue":"IEEE/RSJ international conference on intelligent robots and systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Mockus and others, 1978] J. Mockus et al. The application of bayesian methods for seeking the extremum. Towards Global Optimization, 2, 1978.","title":"The application of bayesian methods for seeking the extremum","authors":"J Mockus","year":"1978","venue":"Towards Global Optimization","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Rasmussen and Williams, 2006] Carl E. Rasmussen and Christopher Williams. Gaussian processes for machine learning. MIT, 2006.","title":"Gaussian processes for machine learning","authors":"Carl E Rasmussen, Christopher Williams","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Scott et al., 2011] Warren Scott, Peter Frazier, and Warren Powell. The correlated knowledge gradient for simulation optimization of continuous parameters using gaussian pro- cess regression. SIAM Journal on Optimization, 21:996- 1026, 07 2011.","title":"The correlated knowledge gradient for simulation optimization of continuous parameters using gaussian process regression","authors":"Warren Scott, Peter Frazier, Warren Powell","year":"2011","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Srinivas et al., 2009] Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experi- mental design. arXiv preprint arXiv:0912.3995, 2009.","title":"Gaussian process optimization in the bandit setting: No regret and experimental design","authors":"Niranjan Srinivas, Andreas Krause, Matthias Sham M Kakade, Seeger","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Yue and Kontar, 2020] Xubo Yue and Raed AL Kontar. Why non-myopic bayesian optimization is promising and how far should we look-ahead? a study via rollout. In Sil- via Chiappa and Roberto Calandra, editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 2808-2818. PMLR, 26-28 Aug 2020.","title":"Why non-myopic bayesian optimization is promising and how far should we look-ahead? a study via rollout","authors":"Kontar Yue, Xubo Yue, Raed Al Kontar","year":"2020","venue":"Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}