{"type":"ScholarlyArticle","title":"Fully Distributed Bayesian Optimization with Stochastic Policies","authors":[{"name":"Javier Garcia-Barcos","slug":"javier-garcia-barcos","given":"Javier","family":"Garcia-Barcos","affiliation":"Instituto de Investigacion en Ingenieria de Aragon , University of Zaragoza"},{"name":"Ruben Martinez-Cantin","slug":"ruben-martinez-cantin","given":"Ruben","family":"Martinez-Cantin","affiliation":"Instituto de Investigacion en Ingenieria de Aragon , University of Zaragoza; Centro Universitario de la Defensa , Zaragoza jgbarcos ,"}],"abstract":"Bayesian optimization has become a popular method for applications, like the design of computer experiments or hyperparameter tuning of expensive models, where sample efficiency is mandatory. These situations or high-throughput computing, where distributed and scalable architectures are a necessity. However, Bayesian optimization is mostly sequential. Even parallel variants require certain computations between samples, limiting the parallelization bandwidth. Thompson sampling has been previously applied for distributed Bayesian optimization. But, when compared with other acquisition functions in the sequential setting, Thompson sampling is known to perform suboptimally. In this paper, we present a new method for fully distributed Bayesian optimization, which can be combined with any acquisition function. Our approach considers Bayesian optimization as a partially observable Markov decision process. In this context, stochastic policies, such as the Boltzmann policy, have some interesting properties which can also be studied for Bayesian optimization. Furthermore, the Boltzmann policy trivially allows a distributed Bayesian optimization implementation with high level of parallelism and scalability. We present results in several benchmarks and applications that shows the performance of our method.","doi":"10.24963/ijcai.2019/327","url":"https://ijcai.org/proceedings/2019/327","pdf":"https://ijcai.org/proceedings/2019/0327.pdf","markdown":"https://ijcai.org/proceedings/2019/327.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/327","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":327,"pages":{"first":2357,"last":2363},"keywords":["Planning and Scheduling: POMDPs","Machine Learning: Active Learning"],"videos":[],"references":[{"position":1,"raw":"Bull, 2011","title":null,"authors":"Bull","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"A.D. Bull. Convergence rates of efficient global optimization algorithms. JMLR, 12:2879-2904, 2011.","title":"Convergence rates of efficient global optimization algorithms","authors":"A D Bull","year":"2011","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Contal et al., 2013] E. Contal, D. Buffoni, A. Robicquet, and N. Vayatis. Parallel Gaussian process optimization with upper confidence bound and pure exploration. In ECMLKDD, pages 225-240. Springer, 2013.","title":"Parallel Gaussian process optimization with upper confidence bound and pure exploration","authors":"E Contal, D Buffoni, A Robicquet, N Vayatis","year":"2013","venue":"ECMLKDD","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Desautels et al., 2014] T. Desautels, A. Krause, and J.W. Burdick. Parallelizing exploration-exploitation trade- offs in gaussian process bandit optimization. JMLR, 15(1):3873-3923, 2014.","title":"Parallelizing exploration-exploitation tradeoffs in gaussian process bandit optimization","authors":"T Desautels, A Krause, J W Burdick","year":"2014","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Forrester et al., 2006] A. Forrester, N. Bressloff, and A. Keane. Optimization using surrogate models and partially converged computational fluid dynamics simu- lations. Proceedings of the Royal Society of London A, 462(2071):2177-2204, 2006.","title":"Optimization using surrogate models and partially converged computational fluid dynamics simulations","authors":"A Forrester, N Bressloff, A Keane","year":"2006","venue":"Proceedings of the Royal Society of London A","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ginsbourger et al., 2010] D. Ginsbourger, R. Le Riche, and L. Carraro. Kriging is well-suited to parallelize optimiza- tion. In Computational intelligence in expensive optimiza- tion problems, pages 131-162. Springer, 2010.","title":"Kriging is well-suited to parallelize optimization","authors":"D Ginsbourger, R Le Riche, L Carraro","year":"2010","venue":"Computational intelligence in expensive optimization problems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[González et al., 2016a] J. González, Z. Dai, P. Hennig, and N. Lawrence. Batch bayesian optimization via local pe- nalization. In AISTATS, pages 648-657, 2016.","title":"Batch bayesian optimization via local penalization","authors":"González","year":"2016","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[González et al., 2016b] J. González, M. Osborne, and N. Lawrence. Glasses: Relieving the myopia of bayesian optimisation. In AISTATS, pages 790-799, 2016.","title":"Glasses: Relieving the myopia of bayesian optimisation","authors":"González","year":"2016","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hennig and Schuler, 2012] P. Hennig and C.J. Schuler. En- tropy search for information efficient global optimization. JMLR, 13:1809-1837, 2012.","title":"Entropy search for information efficient global optimization","authors":"P Hennig, C J Schuler","year":"2012","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hernandez-Lobato et al., 2014] J.M. Hernandez-Lobato, M.W. Hoffman, and Z. Ghahramani. Predictive entropy search for efficient global optimization of black-box functions. In NIPS, pages 918-926, 2014.","title":"Predictive entropy search for efficient global optimization of black-box functions","authors":"Hernandez-Lobato","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hernandez-Lobato et al., 2017] J.M. Hernandez-Lobato, J. Requeima, E.O. Pyzer-Knapp, and A. Aspuru-Guzik. Parallel and distributed thompson sampling for large-scale accelerated exploration of chemical space. In ICML, pages 1470-1479, 2017.","title":"Parallel and distributed thompson sampling for large-scale accelerated exploration of chemical space","authors":"Hernandez-Lobato","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jones et al., 1998] D.R. Jones, M. Schonlau, and W.J. Welch. Efficient global optimization of expensive black- box functions. J Glob Optim, 13(4):455-492, 1998.","title":"Efficient global optimization of expensive blackbox functions","authors":"D R Jones, M Schonlau, W J Welch","year":"1998","venue":"J Glob Optim","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kaelbling et al., 1996] L.P. Kaelbling, M.L. Littman, and A.W. Moore. Reinforcement learning: A survey. JMLR, 4:237-285, 1996.","title":"Reinforcement learning: A survey","authors":"L P Kaelbling, M L Littman, A W Moore","year":"1996","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kandasamy et al., 2018] K. Kandasamy, A. Krishnamurthy, J. Schneider, and B. Poczos. Parallelised Bayesian optimi- sation via thompson sampling. In AISTATS, 2018.","title":"Parallelised Bayesian optimisation via thompson sampling","authors":"K Kandasamy, A Krishnamurthy, J Schneider, B Poczos","year":"2018","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Klein et al., 2017] A. Klein, S. Falkner, S. Bartels, P. Hen- nig, and F. Hutter. Fast Bayesian optimization of machine learning hyperparameters on large datasets. In AISTATS, 2017.","title":"Fast Bayesian optimization of machine learning hyperparameters on large datasets","authors":"A Klein, S Falkner, S Bartels, P Hennig, F Hutter","year":"2017","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lam et al., 2016] R. Lam, K. Willcox, and D.H. Wolpert. Bayesian optimization with a finite budget: An approx- imate dynamic programming approach. In NIPS, pages 883-891, 2016.","title":"Bayesian optimization with a finite budget: An approximate dynamic programming approach","authors":"R Lam, K Willcox, D H Wolpert","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lopes and Montesano, 2014] M. Lopes and L. Montesano. Active learning for autonomous intelligent agents: Explo- ration, curiosity, and interaction. arXiv:1403.1497, 2014.","title":"Active learning for autonomous intelligent agents: Exploration, curiosity, and interaction","authors":"M Lopes, L Montesano","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Martinez-Cantin et al., 2018] R. Martinez-Cantin, K. Tee, and M. McCourt. Practical bayesian optimization in the presence of outliers. In AISTATS, pages 1722-1731, 2018.","title":"Practical bayesian optimization in the presence of outliers","authors":"R Martinez-Cantin, K Tee, M Mccourt","year":"2018","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Martinez-Cantin, 2019] R. Martinez-Cantin. Funneled Bayesian optimization for design, tuning and control of autonomous systems. IEEE Trans Cybern, 49(4):1489- 1500, 2019.","title":"Funneled Bayesian optimization for design, tuning and control of autonomous systems","authors":"R Martinez-Cantin","year":"2019","venue":"IEEE Trans Cybern","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Merino-Casallo and others, 2018] F. Merino-Casallo et al. Integration of in vitro and in silico models using Bayesian optimization with an application to stochastic modeling of mesenchymal 3d cell migration. Front. Psychol., 9, 2018.","title":"Integration of in vitro and in silico models using Bayesian optimization with an application to stochastic modeling of mesenchymal 3d cell migration","authors":"F Merino-Casallo","year":"2018","venue":"Front. Psychol","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mockus et al., 1978] J. Mockus, V. Tiesis, and A. Zilinskas. The application of Bayesian methods for seeking the ex- tremum. In Towards Global Optimisation 2, pages 117- 129. Elsevier, 1978.","title":"The application of Bayesian methods for seeking the extremum","authors":"J Mockus, V Tiesis, A Zilinskas","year":"1978","venue":"Towards Global Optimisation","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Neal, 1996] R.M. Neal. Sampling from multimodal distri- butions using tempered transitions. Statistics and comput- ing, 6(4):353-366, 1996.","title":"Sampling from multimodal distributions using tempered transitions","authors":"R M Neal","year":"1996","venue":"Statistics and computing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sacks et al., 1989] J. Sacks, W.J. Welch, T.J. Mitchell, and H.P. Wynn. Design and analysis of computer experiments. Statistical Science, 4(4):409-423, 1989.","title":"Design and analysis of computer experiments","authors":"J Sacks, W J Welch, T J Mitchell, H P Wynn","year":"1989","venue":"Statistical Science","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shah and Ghahramani, 2015] A. Shah and Z. Ghahramani. Parallel predictive entropy search for batch global opti- mization of expensive objective functions. In NIPS, 2015.","title":"Parallel predictive entropy search for batch global optimization of expensive objective functions","authors":"A Shah, Z Ghahramani","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shahriari et al., 2016] B. Shahriari, K. Swersky, Z. Wang, R.P. Adams, and N. de Freitas. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1):148-175, 2016.","title":"Taking the human out of the loop: A review of Bayesian optimization","authors":"B Shahriari, K Swersky, Z Wang, R P Adams, N De Freitas","year":"2016","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Singh et al., 2000] S. Singh, T. Jaakkola, M.L. Littman, and C. Szepesvári. Convergence results for single-step on- policy reinforcement-learning algorithms. Machine learn- ing, 38(3):287-308, 2000.","title":"Convergence results for single-step onpolicy reinforcement-learning algorithms","authors":"S Singh, T Jaakkola, M L Littman, C Szepesvári","year":"2000","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Snoek et al., 2012] J. Snoek, H. Larochelle, and R. Adams. Practical Bayesian optimization of machine learning algo- rithms. In NIPS, pages 2960-2968, 2012.","title":"Practical Bayesian optimization of machine learning algorithms","authors":"J Snoek, H Larochelle, R Adams","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Srinivas et al., 2010] N. Srinivas, A. Krause, S. Kakade, and M. Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In ICML, 2010.","title":"Gaussian process optimization in the bandit setting: No regret and experimental design","authors":"N Srinivas, A Krause, S Kakade, M Seeger","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Toussaint, 2014] M. Toussaint. The Bayesian search game. In Theory and Principled Methods for Designing Meta- heuristics. Springer, 2014.","title":"The Bayesian search game","authors":"M Toussaint","year":"2014","venue":"Theory and Principled Methods for Designing Metaheuristics","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang and de Freitas, 2014] Z. Wang and N. de Freitas. Theoretical analysis of Bayesian optimisation with un- known Gaussian process hyperparameters. arXiv, 2014.","title":"Theoretical analysis of Bayesian optimisation with unknown Gaussian process hyperparameters","authors":"Wang Freitas, ; Z Wang, N De Freitas","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang and Jegelka, 2017] Z. Wang and S. Jegelka. Max- value entropy search for efficient Bayesian optimization. In ICML, volume 70, pages 3627-3635, 2017.","title":"Maxvalue entropy search for efficient Bayesian optimization","authors":"Z Wang, S Jegelka","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}