{"type":"ScholarlyArticle","title":"Boosting Combinatorial Problem Modeling with Machine Learning","authors":[{"name":"Michele Lombardi","slug":"michele-lombardi","given":"Michele","family":"Lombardi","affiliation":"DISI-University of Bologna"},{"name":"Michela Milano","slug":"michela-milano","given":"Michela","family":"Milano","affiliation":"DISI-University of Bologna"}],"abstract":"In the past few years, the area of Machine Learning (ML) has witnessed tremendous advancements, becoming a pervasive technology in a wide range of applications. One area that can significantly benefit from the use of ML is Combinatorial Optimization. The three pillars of constraint satisfaction and optimization problem solving, i.e., modeling, search, and optimization, can exploit ML techniques to boost their accuracy, efficiency and effectiveness. In this survey we focus on the modeling component, whose effectiveness is crucial for solving the problem. The modeling activity has been traditionally shaped by optimization and domain experts, interacting to provide realistic results. Machine Learning techniques can tremendously ease the process, and exploit the available data to either create models or refine expert-designed ones. In this survey we cover approaches that have been recently proposed to enhance the modeling process by learning either single constraints, objective functions, or the whole model. We highlight common themes to multiple approaches and draw connections with related fields of research.","doi":"10.24963/ijcai.2018/772","url":"https://ijcai.org/proceedings/2018/772","pdf":"https://ijcai.org/proceedings/2018/0772.pdf","markdown":"https://ijcai.org/proceedings/2018/772.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/772","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Survey track","track":"Survey track","number":772,"pages":{"first":5472,"last":5478},"keywords":["Constraints and SAT: Modeling;Formulation","Heuristic Search and Game Playing: Combinatorial Search and Optimisation","Constraints and SAT: Constraints and Data Mining ; Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Balafrej et al., 2015] Amine Balafrej, Christian Bessière, and Anastasia Paparrizou. Multi-armed bandits for adap- tive constraint propagation. In Proc. of IJCAI, pages 290- 296, 2015.","title":"Multi-armed bandits for adaptive constraint propagation","authors":"Amine Balafrej, Christian Bessière, Anastasia Paparrizou","year":"2015","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/47"},{"position":2,"raw":"[Beldiceanu and Simonis, 2012] Nicolas Beldiceanu and Helmut Simonis. A model seeker: Extracting global constraint models from positive examples. In Proc. of CP, pages 141-157, 2012.","title":"A model seeker: Extracting global constraint models from positive examples","authors":"Simonis Beldiceanu, Nicolas Beldiceanu, Helmut Simonis","year":"2012","venue":"Proc. of CP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bello et al., 2016] Irwan Bello, Hieu Pham, Quoc V. Le, Mohammad Norouzi, and Samy Bengio. Neural combi- natorial optimization with reinforcement learning. CoRR, abs/1611.09940, 2016.","title":"Neural combinatorial optimization with reinforcement learning","authors":"Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, Samy Bengio","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bessiere et al., 2013] Christian Bessiere, Remi Coletta, Emmanuel Hebrard, George Katsirelos, Nadjib Lazaar, Nina Narodytska, Claude-Guy Quimper, and Toby Walsh. Constraint acquisition via partial queries. In Proc. of IJ- CAI, pages 475-481, 2013.","title":"Constraint acquisition via partial queries","authors":"Bessiere","year":"2013","venue":"Proc. of IJ-CAI","doi":null,"url":null,"internal_url":"/proceedings/2013/78"},{"position":5,"raw":"[Bessiere et al., 2016] Christian Bessiere, Luc De Raedt, Lars Kotthoff, Siegfried Nijssen, Barry O'Sullivan, and Dino Pedreschi, editors. Data Mining and Constraint Programming: Foundations of a Cross-Disciplinary ap- proach. Springer, 2016.","title":"Christian Bessiere","authors":"Bessiere","year":"2016","venue":"Data Mining and Constraint Programming: Foundations of a Cross-Disciplinary approach","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bessiere et al., 2017a] Christian Bessiere, Frédéric Ko- riche, Nadjib Lazaar, and Barry O'Sullivan. Constraint acquisition. Artifcial Intelligence, 244:315-342, 2017.","title":"Christian Bessiere, Frédéric Koriche, Nadjib Lazaar, and Barry O'Sullivan. Constraint acquisition","authors":"Bessiere","year":"2017","venue":"Artifcial Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bessiere et al., 2017b] Christian Bessiere, Luc De Raedt, Tias Guns, Lars Kotthoff, Mirco Nanni, Siegfried Nijssen, Barry O'Sullivan, Anastasia Paparrizou, Dino Pedreschi, and Helmut Simonis. The inductive constraint program- ming loop. IEEE Intelligent Systems, 32(5):44-52, 2017.","title":"The inductive constraint programming loop","authors":"Bessiere","year":"2017","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Bonfietti et al., 2015] Alessio Bonfietti, Michele Lombardi, and Michela Milano. Embedding decision trees and ran- dom forests in constraint programming. In Proc. of CPAIOR, pages 74-90, 2015.","title":"Embedding decision trees and random forests in constraint programming","authors":"Alessio Bonfietti, Michele Lombardi, Michela Milano","year":"2015","venue":"Proc. of CPAIOR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Campigotto et al., 2011] Paolo Campigotto, Andrea Passerini, and Roberto Battiti. Active Learning of Com- binatorial Features for Interactive Optimization, pages 336-350. Springer Berlin Heidelberg, Berlin, Heidelberg, 2011.","title":"Active Learning of Combinatorial Features for Interactive Optimization","authors":"Paolo Campigotto, Andrea Passerini, Roberto Battiti","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Christofides et al., 2013] Panagiotis D. Christofides, Ric- cardo Scattolini, David Muñoz de la Peña, and Jinfeng Liu. Distributed model predictive control: A tutorial review and future research directions. Computers & Chemical Engi- neering, 51:21-41, 2013.","title":"Distributed model predictive control: A tutorial review and future research directions","authors":"D Panagiotis, Riccardo Christofides, David Scattolini, Jinfeng Muñoz De La Peña, Liu","year":"2013","venue":"Computers & Chemical Engineering","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Donti et al., 2017] Priya Donti, J. Zico Kolter, and Brandon Amos. Task-based end-to-end model learning in stochas- tic optimization. In I.","title":"Task-based end-to-end model learning in stochastic optimization","authors":"J Zico Priya Donti, Brandon Kolter, Amos","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Sys- tems 30, pages 5490-5500.","title":"Advances in Neural Information Processing Systems","authors":"U V Guyon, S Luxburg, H Bengio, R Wallach","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Dragone et al., 2018] Paolo Dragone, Stefano Teso, and Andrea Passerini. Constructive preference elicitation. Front. Robotics and AI, 2018, 2018.","title":"Constructive preference elicitation","authors":"Paolo Dragone, Stefano Teso, Andrea Passerini","year":"2018","venue":"Front. Robotics and AI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Fischetti and Jo, 2018] Matteo Fischetti and Jason Jo. Deep neural networks as 0-1 mixed integer linear programs: A feasibility study. In Proc. of CPAIOR, 2018.","title":"Deep neural networks as 0-1 mixed integer linear programs: A feasibility study","authors":"Matteo Fischetti, Jason Jo","year":"2018","venue":"Proc. of CPAIOR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Galassi et al., 2018] Andrea Galassi, Michele Lombardi, Paola Mello, and Michela Milano. Model agnostic solu- tion of csps via deep learning: a preliminary study. In Proc. of CPAIOR, 2018.","title":"Model agnostic solution of csps via deep learning: a preliminary study","authors":"Andrea Galassi, Michele Lombardi, Paola Mello, Michela Milano","year":"2018","venue":"Proc. of CPAIOR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gilan and Dilkina, 2015] Siamak Safarzadegan Gilan and Bistra Dilkina. Sustainable building design: A challenge at the intersection of machine learning and design opti- mization. In Proc. of AAAI Workshop on Computational Sustainability, 2015.","title":"Sustainable building design: A challenge at the intersection of machine learning and design optimization","authors":"Dilkina ; Gilan, Safarzadegan Siamak, Bistra Gilan, Dilkina","year":"2015","venue":"Proc. of AAAI Workshop on Computational Sustainability","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hottung et al., 2017] Andre Hottung, Shunji Tanaka, and Kevin Tierney. Deep learning assisted heuristic tree search for the container pre-marshalling problem. In arXiv:1709.09972v1, 2017.","title":"Deep learning assisted heuristic tree search for the container pre-marshalling problem","authors":"Andre Hottung, Shunji Tanaka, Kevin Tierney","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Huang et al., 2017] Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. Safety verification of deep neural networks. In Proc. of CAV, pages 3-29, 2017.","title":"Safety verification of deep neural networks","authors":"Xiaowei Huang, Marta Kwiatkowska, Sen Wang, Min Wu","year":"2017","venue":"Proc. of CAV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Hutter et al., 2011] Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. Sequential model-based optimiza- tion for general algorithm configuration. In Proc. of LION, pages 507-523, 2011.","title":"Sequential model-based optimization for general algorithm configuration","authors":"Frank Hutter, H Holger, Kevin Hoos, Leyton-Brown","year":"2011","venue":"Proc. of LION","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Hutter et al., 2014] Frank Hutter, Lin Xu, Holger H. Hoos, and Kevin Leyton-Brown. Algorithm runtime prediction: Methods & evaluation. Artificial Intelligence, 206:79-111, 2014.","title":"Algorithm runtime prediction: Methods & evaluation","authors":"Frank Hutter, Lin Xu, H Holger, Kevin Hoos, Leyton-Brown","year":"2014","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Katz et al., 2017] Guy Katz, Clark W. Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. Reluplex: An efficient SMT solver for verifying deep neural net- works. In Proc. of CAV, pages 97-117, 2017.","title":"Reluplex: An efficient SMT solver for verifying deep neural networks","authors":"Guy Katz, Clark W Barrett, David L Dill, Kyle Julian, Mykel J Kochenderfer","year":"2017","venue":"Proc. of CAV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"Manuel Laguna. Optquest optimization of complex systems. In OptQuest White Papers, 2011.","title":"Optquest optimization of complex systems","authors":"Manuel Laguna","year":"2011","venue":"OptQuest White Papers","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lallouet and Legtchenko, 2007] Arnaud Lallouet and An- drei Legtchenko. Building consistencies for partially defined constraints with decision trees and neural net- works. International Journal on Artificial Intelligence Tools, 16(4):683-706, 2007.","title":"Arnaud Lallouet and Andrei Legtchenko. Building consistencies for partially defined constraints with decision trees and neural networks","authors":"Legtchenko Lallouet","year":"2007","venue":"International Journal on Artificial Intelligence Tools","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Lallouet et al., 2010] Arnaud Lallouet, Matthieu Lopez, Li- onel Martin, and Christel Vrain. On learning constraint problems. In Proc. of IJCAI, pages 45-52, 2010.","title":"On learning constraint problems","authors":"Arnaud Lallouet, Matthieu Lopez, Lionel Martin, Christel Vrain","year":"2010","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Liberto et al., 2016] G. Liberto, S. Kadioglu, K. Leo, and Y. Malitshy. Dash: Dynamic approach for switching heuristics. European Journal of Operational Research, 2016.","title":"Dash: Dynamic approach for switching heuristics","authors":"G Liberto, S Kadioglu, K Leo, Y Malitshy","year":"2016","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Lombardi and Gualandi, 2016] Michele Lombardi and Ste- fano Gualandi. A lagrangian propagator for artificial neu- ral networks in constraint programming. Constraints, 21(4):435-462, 2016.","title":"A lagrangian propagator for artificial neural networks in constraint programming","authors":"Michele Lombardi, Stefano Gualandi","year":"2016","venue":"Constraints","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Lombardi et al., 2017] Michele Lombardi, Michela Milano, and Andrea Bartolini. Empirical decision model learn- ing. Artificial Intelligence, 244(Supplement C):343 -367, 2017. Combining Constraint Solving with Mining and Learning.","title":"Empirical decision model learning","authors":"Michele Lombardi, Michela Milano, Andrea Bartolini","year":"2017","venue":"Combining Constraint Solving with Mining and Learning","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Loth et al., 2013] Manuel Loth, Michèle Sebag, Youssef Hamadi, and Marc Schoenauer. Bandit-based search for constraint programming. In Proc. of CP, pages 464-480, 2013.","title":"Bandit-based search for constraint programming","authors":"Manuel Loth, Michèle Sebag, Youssef Hamadi, Marc Schoenauer","year":"2013","venue":"Proc. of CP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Mairy, 2011] Jean-Baptiste Mairy. Reinforced adaptive large neighborhood search. In Proc. of CP, 2011.","title":"Reinforced adaptive large neighborhood search","authors":"Jean-Baptiste Mairy","year":"2011","venue":"Proc. of CP","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Malitsky and Sellmann, 2012] Yuri Malitsky and Meinolf Sellmann. Instance-specific algorithm configuration as a method for non-model-based portfolio generation. In Proc. of CPAIOR, pages 244-259, 2012.","title":"Instance-specific algorithm configuration as a method for non-model-based portfolio generation","authors":"Sellmann Malitsky, Yuri Malitsky, Meinolf Sellmann","year":"2012","venue":"Proc. of CPAIOR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Say et al., 2017] Buser Say, Ga Wu, Yu Qing Zhou, and Scott Sanner. Nonlinear hybrid planning with deep net learned transition models and mixed-integer linear pro- gramming. In Proc. of IJCAI, pages 750-756, 2017.","title":"Nonlinear hybrid planning with deep net learned transition models and mixed-integer linear programming","authors":"Buser Say, Ga Wu, Yu Qing Zhou, Scott Sanner","year":"2017","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/104"},{"position":33,"raw":"[Verwer et al., 2017] Sicco Verwer, Yingqian Zhang, and Qing Chuan Ye. Auction optimization using regression trees and linear models as integer programs. Artificial In- telligence, 244:368-395, 2017.","title":"Auction optimization using regression trees and linear models as integer programs","authors":"Sicco Verwer, Yingqian Zhang, Qing Chuan, Ye","year":"2017","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Vu et al., 2017] Ky Khac Vu, Claudia D'Ambrosio, Youssef Hamadi, and Leo Liberti. Surrogate-based methods for black-box optimization. ITOR, 24(3):393-424, 2017.","title":"Surrogate-based methods for black-box optimization","authors":"Khac Ky, Claudia D Vu, Youssef 'ambrosio, Leo Hamadi, Liberti","year":"2017","venue":"ITOR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wilson and Sahinidis, 2017] Zachary T. Wilson and Niko- laos V. Sahinidis. The ALAMO approach to machine learning. Computers & Chemical Engineering, 106:785- 795, 2017.","title":"The ALAMO approach to machine learning","authors":"Zachary T Wilson, Nikolaos V Sahinidis","year":"2017","venue":"Computers & Chemical Engineering","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Xu et al., 2008] Lin Xu, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. Satzilla: Portfolio-based algorithm selection for SAT. Journal of Artificial Intelligence Re- search, 32:565-606, 2008.","title":"Satzilla: Portfolio-based algorithm selection for SAT","authors":"Lin Xu, Frank Hutter, H Holger, Kevin Hoos, Leyton-Brown","year":"2008","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}