Neural Networks for Predicting Algorithm Runtime Distributions

Neural Networks for Predicting Algorithm Runtime Distributions

Katharina Eggensperger, Marius Lindauer, Frank Hutter

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 1442-1448. https://doi.org/10.24963/ijcai.2018/200

Many state-of-the-art algorithms for solving hard combinatorial problems in artificial intelligence (AI) include elements of stochasticity that lead to high variations in runtime, even for a fixed problem instance. Knowledge about the resulting runtime distributions (RTDs) of algorithms on given problem instances can be exploited in various meta-algorithmic procedures, such as algorithm selection, portfolios, and randomized restarts. Previous work has shown that machine learning can be used to individually predict mean, median and variance of RTDs. To establish a new state-of-the-art in predicting RTDs, we demonstrate that the parameters of an RTD should be learned jointly and that neural networks can do this well by directly optimizing the likelihood of an RTD given runtime observations. In an empirical study involving five algorithms for SAT solving and AI planning, we show that neural networks predict the true RTDs of unseen instances better than previous methods, and can even do so when only few runtime observations are available per training instance.
Keywords:
Heuristic Search and Game Playing: Meta-Reasoning and Meta-heuristics
Constraints and SAT: Constraints: Solvers and Tools
Constraints and SAT: Constraints: Evaluation and Analysis
Constraints and SAT: SAT: Evaluation and Analysis
Constraints and SAT: Constraints and Data Mining ; Machine Learning
Constraints and SAT: SAT: Solvers and Tools
Heuristic Search and Game Playing: Heuristic Search and Machine Learning