Bayesian Optimization in High Dimensions via Random Embeddings

Wang, Ziyu; Zoghi, Masrour; Hutter, Frank; Matheson, David; de Freitas, Nando

Bayesian Optimization in High Dimensions via Random Embeddings

Ziyu Wang, Masrour Zoghi, Frank Hutter, David Matheson, Nando de Freitas

Download BibTeX Cite

Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. Pages 1778-1783.

Abstract

Bayesian optimization techniques have been successfully applied to robotics, planning, sensor placement, recommendation, advertising, intelligent user interfaces and automatic algorithm configuration. Despite these successes, the approach is restricted to problems of moderate dimension, and several workshops on Bayesian optimization have identified its scaling to high dimensions as one of the holy grails of the field. In this paper, we introduce a novel random embedding idea to attack this problem. The resulting Random EMbedding Bayesian Optimization (REMBO) algorithm is very simple and applies to domains with both categorical and continuous variables. The experiments demonstrate that REMBO can effectively solve high-dimensional problems, including automatic parameter configuration of a popular mixed integer linear programming solver.

Ziyu Wang, Masrour Zoghi, Frank Hutter, David Matheson, and Nando de Freitas. 2013. Bayesian Optimization in High Dimensions via Random Embeddings. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI-13. IJCAI, 1778–1783. https://ijcai.org/proceedings/2013/263

@inproceedings{ijcai2013p263,
  title     = {Bayesian Optimization in High Dimensions via Random Embeddings},
  author    = {Wang, Ziyu and Zoghi, Masrour and Hutter, Frank and Matheson, David and de Freitas, Nando},
  booktitle = {Proceedings of the Twenty-Third International Joint Conference on
               Artificial Intelligence, {IJCAI-13}},
  publisher = {AAAI Press / International Joint Conferences on Artificial Intelligence},
  editor    = {Francesca Rossi},
  pages     = {1778--1783},
  year      = {2013},
  month     = {8},
  url       = {https://ijcai.org/proceedings/2013/263},
}

TY  - CPAPER
TI  - Bayesian Optimization in High Dimensions via Random Embeddings
AU  - Wang, Ziyu
AU  - Zoghi, Masrour
AU  - Hutter, Frank
AU  - Matheson, David
AU  - de Freitas, Nando
T2  - Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI-13
PB  - AAAI Press / International Joint Conferences on Artificial Intelligence
PY  - 2013
SP  - 1778
EP  - 1783
UR  - https://ijcai.org/proceedings/2013/263
ER  -