{"type":"ScholarlyArticle","title":"Cascaded Algorithm-Selection and Hyper-Parameter Optimization with Extreme-Region Upper Confidence Bound Bandit","authors":[{"name":"Yi-Qi Hu","slug":"yi-qi-hu","given":"Yi-Qi","family":"Hu","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing 210023 , China"},{"name":"Yang Yu","slug":"yang-yu","given":"Yang","family":"Yu","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing 210023 , China"},{"name":"Jun-Da Liao","slug":"jun-da-liao","given":"Jun-Da","family":"Liao","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing 210023 , China"}],"abstract":"An automatic machine learning (AutoML) task is to select the best algorithm and its hyper-parameters simultaneously. Previously, the hyper-parameters of all algorithms are joint as a single search space, which is not only huge but also redundant, because many dimensions of hyper-parameters are irrelevant with the selected algorithms. In this paper, we propose a cascaded approach for algorithm selection and hyper-parameter optimization. While a search procedure is employed at the level of hyper-parameter optimization, a bandit strategy runs at the level of algorithm selection to allocate the budget based on the search feedbacks. Since the bandit is required to select the algorithm with the maximum performance, instead of the average performance, we thus propose the extreme-region upper confidence bound (ER-UCB) strategy, which focuses on the extreme region of the underlying feedback distribution. We show theoretically that the ER-UCB has a regret upper bound O(K ln n) with independent feedbacks, which is as efficient as the classical UCB bandit. We also conduct experiments on a synthetic problem as well as a set of AutoML tasks. The results verify the effectiveness of the proposed method.","doi":"10.24963/ijcai.2019/351","url":"https://ijcai.org/proceedings/2019/351","pdf":"https://ijcai.org/proceedings/2019/0351.pdf","markdown":"https://ijcai.org/proceedings/2019/351.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/351","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":351,"pages":{"first":2528,"last":2534},"keywords":["Machine Learning: Transfer, Adaptation, Multi-task Learning","Heuristic Search and Game Playing: Heuristic Search and Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Adankon and Cheriet, 2009] Mathias M Adankon and Mo- hamed Cheriet. Model selection for the LS-SVM. appli- cation to handwriting recognition. Pattern Recognition, 42(12):3264-3270, 2009.","title":"Model selection for the LS-SVM. application to handwriting recognition","authors":"M Mathias, Mohamed Adankon, Cheriet","year":"2009","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Auer et al., 2002] Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256, 2002.","title":"Finite-time analysis of the multiarmed bandit problem","authors":"Auer","year":"2002","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Biem, 2003] Alain Biem. A model selection criterion for classification: Application to hmm topology optimiza- tion. In Proceedings of the 7th International Conference on Document Analysis and Recognition, pages 104-108, 2003.","title":"A model selection criterion for classification: Application to hmm topology optimization","authors":"Alain Biem","year":"2003","venue":"Proceedings of the 7th International Conference on Document Analysis and Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brazdil et al., 2003] Pavel B Brazdil, Carlos Soares, and Joaquim Pinto Da Costa. Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results. Machine Learning, 50(3):251-277, 2003.","title":"Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results","authors":"Carlos Pavel B Brazdil, Joaquim Soares, Da Pinto, Costa","year":"2003","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bubeck et al., 2012] Sébastien Bubeck, Nicolo Cesa- Bianchi, et al. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends in Machine Learning, 5(1):1-122, 2012.","title":"Regret analysis of stochastic and nonstochastic multi-armed bandit problems","authors":"Sébastien Bubeck, Nicolo Cesa-Bianchi","year":"2012","venue":"Foundations and Trends in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Carpentier and Valko, 2014] Alexandra Carpentier and Michal Valko. Extreme bandits. In Advances in Neural Information Processing Systems, pages 1089-1097, 2014.","title":"Extreme bandits","authors":"Valko Carpentier, Alexandra Carpentier, Michal Valko","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cicirello and Smith, 2005] Vincent A Cicirello and Stephen F Smith. The max k-armed bandit: A new model of exploration applied to search heuristic selection. In Proceedings of the 20th AAAI Conference on Artificial Intelligence, pages 1355-1361, 2005.","title":"The max k-armed bandit: A new model of exploration applied to search heuristic selection","authors":"A Vincent, Stephen F Cicirello, Smith","year":"2005","venue":"Proceedings of the 20th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Felício et al., 2017] Crícia Z Felício, Klérisson VR Paixão, Celia AZ Barcelos, and Philippe Preux. A multi-armed bandit model selection for cold-start user recommenda- tion. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization, pages 32-40. ACM, 2017.","title":"A multi-armed bandit model selection for cold-start user recommendation","authors":"Felício","year":"2017","venue":"Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Feurer et al., 2015] Matthias Feurer, Aaron Klein, Katha- rina Eggensperger, Jost , Manuel Blum, and Frank Hutter. Efficient and robust automated machine learning. In Ad- vances in Neural Information Processing Systems, pages 2962-2970, 2015.","title":"Efficient and robust automated machine learning","authors":"Matthias Feurer, Aaron Klein, Katharina Eggensperger, Manuel Jost, Frank Blum, Hutter","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hu et al., 2017] Yi-Qi Hu, Hong Qian, and Yang Yu. Se- quential classification-based optimization for direct policy search. In Proceedings of the 31st AAAI Conference on Artificial Intelligence, pages 2029-2035, 2017.","title":"Sequential classification-based optimization for direct policy search","authors":"Yi-Qi Hu, Hong Qian, Yang Yu","year":"2017","venue":"Proceedings of the 31st AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2018] Yi-Qi Hu, Yang Yu, and Zhi-Hua Zhou. Ex- perienced optimization with reusable directional model for hyper-parameter search. In Proceeding of the 27th Inter- national Joint Conference on Artificial Intelligence, pages 2276-2282, 2018.","title":"Experienced optimization with reusable directional model for hyper-parameter search","authors":"Yi-Qi Hu, Yang Yu, Zhi-Hua Zhou","year":"2018","venue":"Proceeding of the 27th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/315"},{"position":12,"raw":"[Hu et al., 2019] Yi-Qi Hu, Yang Yu, Wei-Wei Tu, Qiang Yang, Yuqiang Chen, and Wenyuan Dai. Multi-fidelity automatic hyper-parameter tuning via transfer series ex- pansion. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 2019.","title":"Multi-fidelity automatic hyper-parameter tuning via transfer series expansion","authors":"Yi-Qi Hu, Yang Yu, Wei-Wei Tu, Qiang Yang, Yuqiang Chen, Wenyuan Dai","year":"2019","venue":"Proceedings of the 33rd AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hutter et al., 2011] Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. Sequential model-based optimiza- tion for general algorithm configuration. LION, 5:507- 523, 2011.","title":"Sequential model-based optimization for general algorithm configuration","authors":"Hutter","year":"2011","venue":"LION","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jamieson and Talwalkar, 2016] Kevin G Jamieson and Ameet Talwalkar. Non-stochastic best arm identification and hyperparameter optimization. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, pages 240-248, 2016.","title":"Non-stochastic best arm identification and hyperparameter optimization","authors":"G Kevin, Ameet Jamieson, Talwalkar","year":"2016","venue":"Proceedings of the 19th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pedregosa et al., 2011] Fabian Pedregosa, Gael Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Pas- sos, David Cournapeau, Matthieu Brucher, Matthieu Per- rot, and Edouard Duchesnay. Scikit-learn: Machine learn- ing in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.","title":"Scikit-learn: Machine learning in Python","authors":"Fabian Pedregosa, Gael Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, Edouard Duchesnay","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sutton and Barto, 2018] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.","title":"Reinforcement learning: An introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Thornton et al., 2013] Chris Thornton, Frank Hutter, Hol- ger H Hoos, and Kevin Leyton-Brown. Auto-weka: Com- bined selection and hyperparameter optimization of clas- sification algorithms. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discov- ery and Data Mining, pages 847-855, 2013.","title":"Auto-weka: Combined selection and hyperparameter optimization of classification algorithms","authors":"Chris Thornton, Frank Hutter, Kevin Holger H Hoos, Leyton-Brown","year":"2013","venue":"Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tokic and Palm, 2011] Michel Tokic and Günther Palm. Value-difference based exploration: adaptive control be- tween epsilon-greedy and softmax. In Annual Conference on Artificial Intelligence, pages 335-346. Springer, 2011.","title":"Value-difference based exploration: adaptive control between epsilon-greedy and softmax","authors":"Palm Tokic, Michel Tokic, Günther Palm","year":"2011","venue":"Annual Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yu et al., 2016] Yang Yu, Hong Qian, and Yi-Qi Hu. Derivative-free optimization via classification. In Proceed- ings of the 30th AAAI Conference on Artificial Intelligence, pages 2286-2292, 2016.","title":"Derivative-free optimization via classification","authors":"Yang Yu, Hong Qian, Yi-Qi Hu","year":"2016","venue":"Proceedings of the 30th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}