{"type":"ScholarlyArticle","title":"AutoFolio: An Automatically Configured Algorithm Selector (Extended Abstract)","authors":[{"name":"Marius Lindauer","slug":"marius-lindauer","given":"Marius","family":"Lindauer","affiliation":"University of Freiburg"},{"name":"Frank Hutter","slug":"frank-hutter","given":"Frank","family":"Hutter","affiliation":"University of Freiburg"},{"name":"Holger H. Hoos","slug":"holger-h-hoos","given":"Holger H.","family":"Hoos","affiliation":"Leiden University University of British Columbia"},{"name":"Torsten Schaub","slug":"torsten-schaub","given":"Torsten","family":"Schaub","affiliation":"University of Potsdam; INRIA Rennes"}],"abstract":"Algorithm selection (AS) techniques -- which involve choosing from a set of algorithms the one expected to solve a given problem instance most efficiently -- have substantially improved the state of the art in solving many prominent AI problems, such as SAT, CSP, ASP, MAXSAT and QBF. Although several AS procedures have been introduced, not too surprisingly, none of them dominates all others across all AS scenarios. Furthermore, these procedures have parameters whose optimal values  vary across AS scenarios. In this extended abstract of our 2015 JAIR article of the same title, we summarize AutoFolio, which uses an algorithm configuration procedure to automatically select an AS approach and optimize its parameters for a given AS scenario. AutoFolio allows researchers and practitioners across a broad range of applications to exploit the combined power of many different AS methods and to automatically construct high-performance algorithm selectors. We demonstrate that AutoFolio was able to produce new state-of-the-art algorithm selectors for 7 well-studied AS scenarios and matches state-of-the-art performance statistically on all other scenarios. Compared to the best single algorithm for each AS scenario, AutoFolio achieved average speedup factors between 1.3 and 15.4.","doi":"10.24963/ijcai.2017/715","url":"https://ijcai.org/proceedings/2017/715","pdf":"https://ijcai.org/proceedings/2017/0715.pdf","markdown":"https://ijcai.org/proceedings/2017/715.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/715","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Journal track","track":"Journal track","number":715,"pages":{"first":5025,"last":5029},"keywords":["Combinatorial & Heuristic Search: Meta-Reasoning and Meta-heuristics","Constraints and Satisfiability: Evaluation and Analysis","Combinatorial & Heuristic Search: Evaluation and Analysis"],"videos":[],"references":[{"position":1,"raw":"[Bergstra et al., 2011] J. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl. Algorithms for hyper-parameter optimiza- tion. In Proc. of NIPS'11, pages 2546-2554, 2011.","title":"Algorithms for hyper-parameter optimization","authors":"J Bergstra, R Bardenet, Y Bengio, B Kégl","year":"2011","venue":"Proc. of NIPS'11","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bischl et al., 2016] B. Bischl, P. Kerschke, L. Kotthoff, M. Lindauer, Y. Malitsky, A. Frechétte, H. Hoos, F. Hutter, K. Leyton-Brown, K. Tierney, and J. Vanschoren. ASlib: A benchmark library for algorithm selection. AIJ, pages 41-58, 2016.","title":"ASlib: A benchmark library for algorithm selection","authors":"B Bischl, P Kerschke, L Kotthoff, M Lindauer, Y Malitsky, A Frechétte, H Hoos, F Hutter, K Leyton-Brown, K Tierney, J Vanschoren","year":"2016","venue":"AIJ","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Collautti et al., 2013] M. Collautti, Y. Malitsky, D. Mehta, and B. O'Sullivan. SNNAP: Solver-based nearest neigh- bor for algorithm portfolios. In Proc. of ECML/PKDD'13, pages 435-450, 2013.","title":"SNNAP: Solver-based nearest neighbor for algorithm portfolios","authors":"M Collautti, Y Malitsky, D Mehta, B O'sullivan","year":"2013","venue":"Proc. of ECML/PKDD'13","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fawcett and Hoos, 2016] C. Fawcett and H. Hoos. Analysing differences between algorithm configurations through ablation. Journal of Heuristics, 22(4):431-458, 2016.","title":"Analysing differences between algorithm configurations through ablation","authors":"C Fawcett, H Hoos","year":"2016","venue":"Journal of Heuristics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Feurer et al., 2015a] M. Feurer, A. Klein, K. Eggensperger, J. T. Springenberg, M. Blum, and F. Hutter. Efficient and robust automated machine learning. In Proc. of NIPS'15, 2015.","title":"Efficient and robust automated machine learning","authors":"Feurer","year":"2015","venue":"Proc. of NIPS'15","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feurer et al., 2015b] M. Feurer, T. Springenberg, and F. Hutter. Initializing Bayesian hyperparameter optimiza- tion via meta-learning. In Proc. of AAAI'15, pages 1128- 1135, 2015.","title":"Initializing Bayesian hyperparameter optimization via meta-learning","authors":"Feurer","year":"2015","venue":"Proc. of AAAI'15","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gomes and Selman, 2001] C. Gomes and B. Selman. Algo- rithm portfolios. AIJ, 126(1-2):43-62, 2001.","title":"Algorithm portfolios","authors":"C Gomes, B Selman","year":"2001","venue":"AIJ","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hoos et al., 2013] H. Hoos, B. Kaufmann, T. Schaub, and M. Schneider. Robust benchmark set selection for boolean constraint solvers. In Proc. of LION'13, pages 138-152, 2013.","title":"Robust benchmark set selection for boolean constraint solvers","authors":"H Hoos, B Kaufmann, T Schaub, M Schneider","year":"2013","venue":"Proc. of LION'13","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hoos et al., 2014] H. Hoos, M. Lindauer, and T. Schaub. claspfolio 2: Advances in algorithm selection for answer set programming. TPLP, 14:569-585, 2014.","title":"claspfolio 2: Advances in algorithm selection for answer set programming","authors":"H Hoos, M Lindauer, T Schaub","year":"2014","venue":"TPLP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hoos et al., 2015] H. Hoos, R. Kaminski, M. Lindauer, and T. Schaub. aspeed: Solver scheduling via answer set pro- gramming. TPLP, 15:117-142, 2015.","title":"aspeed: Solver scheduling via answer set programming","authors":"H Hoos, R Kaminski, M Lindauer, T Schaub","year":"2015","venue":"TPLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huberman et al., 1997] B. Huberman, R. Lukose, and T. Hogg. An economic approach to hard computational problems. Science, 275:51-54, 1997.","title":"An economic approach to hard computational problems","authors":"B Huberman, R Lukose, T Hogg","year":"1997","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hutter et al., 2009] F. Hutter, H. Hoos, K. Leyton-Brown, and T. Stützle. ParamILS: An automatic algorithm config- uration framework. JAIR, 36:267-306, 2009.","title":"ParamILS: An automatic algorithm configuration framework","authors":"F Hutter, H Hoos, K Leyton-Brown, T Stützle","year":"2009","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hutter et al., 2011] F. Hutter, H. Hoos, and K. Leyton- Brown. Sequential model-based optimization for general algorithm configuration. In Proc. of LION'11, pages 507- 523, 2011.","title":"Sequential model-based optimization for general algorithm configuration","authors":"F Hutter, H Hoos, K Leyton-Brown","year":"2011","venue":"Proc. of LION'11","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hutter et al., 2014] F. Hutter, H. Hoos, and K. Leyton- Brown. An efficient approach for assessing hyperparam- eter importance. In Proc. of ICML'14, pages 754-762, 2014.","title":"An efficient approach for assessing hyperparameter importance","authors":"F Hutter, H Hoos, K Leyton-Brown","year":"2014","venue":"Proc. of ICML'14","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kadioglu et al., 2010] S. Kadioglu, Y. Malitsky, M. Sell- mann, and K. Tierney. ISAC -instance-specific algorithm configuration. In Proc. of ECAI'10, pages 751-756, 2010.","title":"ISAC -instance-specific algorithm configuration","authors":"S Kadioglu, Y Malitsky, M Sellmann, K Tierney","year":"2010","venue":"Proc. of ECAI'10","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kadioglu et al., 2011] S. Kadioglu, Y. Malitsky, A. Sabhar- wal, H. Samulowitz, and M. Sellmann. Algorithm selec- tion and scheduling. In Proc. of CP'11, pages 454-469, 2011.","title":"Algorithm selection and scheduling","authors":"S Kadioglu, Y Malitsky, A Sabharwal, H Samulowitz, M Sellmann","year":"2011","venue":"Proc. of CP'11","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kotthoff, 2013] L. Kotthoff. LLAMA: leveraging learn- ing to automatically manage algorithms. CoRR, abs/1306.1031, 2013.","title":"LLAMA: leveraging learning to automatically manage algorithms","authors":"L Kotthoff","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kotthoff, 2014] L. Kotthoff. Algorithm selection for com- binatorial search problems: A survey. AI Magazine, pages 48-60, 2014.","title":"Algorithm selection for combinatorial search problems: A survey","authors":"L Kotthoff","year":"2014","venue":"AI Magazine","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lindauer et al., 2015a] M. Lindauer, H. Hoos, and F. Hutter. From sequential algorithm selection to parallel portfolio selection. In Proc. of LION'15, pages 1-16, 2015.","title":"From sequential algorithm selection to parallel portfolio selection","authors":"Lindauer","year":"2015","venue":"Proc. of LION'15","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lindauer et al., 2015b] M. Lindauer, H. Hoos, F. Hutter, and T. Schaub. Autofolio: An automatically configured algorithm selector. JAIR, 53:745-778, August 2015.","title":"Autofolio: An automatically configured algorithm selector","authors":"Lindauer","year":"2015","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[López-Ibáñez et al., 2016] M. López-Ibáñez, J. Dubois- Lacoste, L. Perez Caceres, M. Birattari, and T. Stützle. The irace package: Iterated racing for automatic algorithm configuration. Operations Research Perspectives, 3:43- 58, 2016.","title":"The irace package: Iterated racing for automatic algorithm configuration","authors":"M López-Ibáñez, J Dubois-Lacoste, L Perez Caceres, M Birattari, T Stützle","year":"2016","venue":"Operations Research Perspectives","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Malitsky et al., 2013] Y. Malitsky, A. Sabharwal, H. Samu- lowitz, and M. Sellmann. Algorithm portfolios based on cost-sensitive hierarchical clustering. In F. Rossi, editor, Proc. of IJCAI'13, pages 608-614, 2013.","title":"Algorithm portfolios based on cost-sensitive hierarchical clustering","authors":"Y Malitsky, A Sabharwal, H Samulowitz, M Sellmann","year":"2013","venue":"Proc. of IJCAI'13","doi":null,"url":null,"internal_url":"/proceedings/2013/97"},{"position":23,"raw":"[Maratea et al., 2014] M. Maratea, L. Pulina, and F. Ricca. A multi-engine approach to answer-set programming. TPLP, 14:841-868, 2014.","title":"A multi-engine approach to answer-set programming","authors":"M Maratea, L Pulina, F Ricca","year":"2014","venue":"TPLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Nudelman et al., 2004] E. Nudelman, K. Leyton-Brown, A. Devkar, Y. Shoham, and H. Hoos. Understanding ran- dom SAT: Beyond the clauses-to-variables ratio. In Proc. of CP'04, pages 438-452, 2004.","title":"Understanding random SAT: Beyond the clauses-to-variables ratio","authors":"E Nudelman, K Leyton-Brown, A Devkar, Y Shoham, H Hoos","year":"2004","venue":"Proc. of CP'04","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rice, 1976] J. Rice. The algorithm selection problem. Ad- vances in Computers, 15:65-118, 1976.","title":"The algorithm selection problem","authors":"J Rice","year":"1976","venue":"Advances in Computers","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Smith-Miles, 2008] K. Smith-Miles. Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM, 41(1), 2008.","title":"Cross-disciplinary perspectives on meta-learning for algorithm selection","authors":"K Smith-Miles","year":"2008","venue":"ACM","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Snoek et al., 2012] J. Snoek, H. Larochelle, and R. P. Adams. Practical Bayesian optimization of machine learn- ing algorithms. In Proc. of NIPS'12, pages 2960-2968, 2012.","title":"Practical Bayesian optimization of machine learning algorithms","authors":"J Snoek, H Larochelle, R P Adams","year":"2012","venue":"Proc. of NIPS'12","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Thornton et al., 2013] C. Thornton, F. Hutter, H. Hoos, and K. Leyton-Brown. Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In Proc. of KDD'13, pages 847-855, 2013.","title":"Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms","authors":"C Thornton, F Hutter, H Hoos, K Leyton-Brown","year":"2013","venue":"Proc. of KDD'13","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xu et al., 2008] L. Xu, F. Hutter, H. Hoos, and K. Leyton- Brown. SATzilla: Portfolio-based algorithm selection for SAT. JAIR, 32:565-606, 2008.","title":"SATzilla: Portfolio-based algorithm selection for SAT","authors":"L Xu, F Hutter, H Hoos, K Leyton-Brown","year":"2008","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xu et al., 2011] L. Xu, F. Hutter, H. Hoos, and K. Leyton- Brown. Hydra-MIP: Automated algorithm configuration and selection for mixed integer programming. In RCRA workshop on Experimental Evaluation of Algorithms for Solving Problems with Combinatorial Explosion at the In- ternational Joint Conference on Artificial Intelligence (IJ- CAI), 2011.","title":"Hydra-MIP: Automated algorithm configuration and selection for mixed integer programming","authors":"L Xu, F Hutter, H Hoos, K Leyton-Brown","year":"2011","venue":"RCRA workshop on Experimental Evaluation of Algorithms for Solving Problems with Combinatorial Explosion at the International Joint Conference on Artificial Intelligence (IJ-CAI)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}