{"type":"ScholarlyArticle","title":"DACBench: A Benchmark Library for Dynamic Algorithm Configuration","authors":[{"name":"Theresa Eimer","slug":"theresa-eimer","given":"Theresa","family":"Eimer","affiliation":""},{"name":"André Biedenkapp","slug":"andre-biedenkapp","given":"André","family":"Biedenkapp","affiliation":""},{"name":"Maximilian Reimer","slug":"maximilian-reimer","given":"Maximilian","family":"Reimer","affiliation":""},{"name":"Steven Adriansen","slug":"steven-adriansen","given":"Steven","family":"Adriansen","affiliation":""},{"name":"Frank Hutter","slug":"frank-hutter","given":"Frank","family":"Hutter","affiliation":""},{"name":"Marius Lindauer","slug":"marius-lindauer","given":"Marius","family":"Lindauer","affiliation":""}],"abstract":"Dynamic Algorithm Configuration (DAC) aims to dynamically control a target algorithm's hyperparameters in order to improve its performance.\r\nSeveral theoretical and empirical results have demonstrated the benefits of dynamically controlling hyperparameters in domains like evolutionary computation, AI Planning or deep learning.\r\nReplicating these results, as well as studying new methods for DAC, however, is difficult since existing benchmarks are often specialized and incompatible with the same interfaces.\r\nTo facilitate benchmarking and thus research on DAC, we propose DACBench, a benchmark library that seeks to collect and standardize existing DAC benchmarks from different AI domains, as well as provide a template for new ones.\r\nFor the design of DACBench, we focused on important desiderata, such as (i) flexibility, (ii) reproducibility, (iii) extensibility and (iv) automatic documentation and visualization.\r\nTo show the potential, broad applicability and challenges of DAC, we explore how a set of six initial benchmarks compare in several dimensions of difficulty.","doi":"10.24963/ijcai.2021/230","url":"https://ijcai.org/proceedings/2021/230","pdf":"https://ijcai.org/proceedings/2021/0230.pdf","markdown":"https://ijcai.org/proceedings/2021/230.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/230","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Heuristic Search and Game Playing","track":"Main Track","number":230,"pages":{"first":1668,"last":1674},"keywords":["Heuristic Search and Game Playing: Meta-Reasoning and Meta-Heuristics","Heuristic Search and Game Playing: Heuristic Search and Machine Learning","Heuristic Search and Game Playing: Evaluation and Analysis"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}