{"type":"ScholarlyArticle","title":"Layered and Staged Monte Carlo Tree Search for SMT Strategy Synthesis","authors":[{"name":"Zhengyang Lu","slug":"zhengyang-lu","given":"Zhengyang","family":"Lu","affiliation":"University of Waterloo"},{"name":"Stefan Siemer","slug":"stefan-siemer","given":"Stefan","family":"Siemer","affiliation":"University of Göttingen and CIDAS"},{"name":"Piyush Jha","slug":"piyush-jha","given":"Piyush","family":"Jha","affiliation":"Georgia Institute of Technology"},{"name":"Joel Day","slug":"joel-day","given":"Joel","family":"Day","affiliation":"Loughborough University"},{"name":"Florin Manea","slug":"florin-manea","given":"Florin","family":"Manea","affiliation":"University of Göttingen and CIDAS"},{"name":"Vijay Ganesh","slug":"vijay-ganesh","given":"Vijay","family":"Ganesh","affiliation":"Georgia Institute of Technology"}],"abstract":"Modern SMT solvers, such as Z3, offer user-controllable strategies that allow solver users the ability to tailor solving strategies for their unique set of instances, thus dramatically enhancing the solver performance for their specific use cases. However, this approach of strategy customization presents a significant challenge: handcrafting an optimized strategy for a class of SMT instances remains a complex and demanding task for both solver developers and users alike.\r\n\r\nIn this paper, we address this problem of automated SMT strategy synthesis via a novel Monte Carlo Tree Search (MCTS) based method. Our method treats strategy synthesis as a sequential decision-making process, whose search tree corresponds to the strategy space, and employs MCTS to navigate this vast search space. The key innovations that enable our method to identify effective strategies, while keeping costs low, are the ideas of layered and staged MCTS search. These novel heuristics allow for a deeper and more efficient exploration of the strategy space, enabling us to synthesize more effective strategies than the default ones in state-of-the-art (SOTA) SMT solvers. We implement our method, dubbed Z3alpha, as part of the Z3 SMT solver. Through extensive evaluations across six important SMT logics, Z3alpha demonstrates superior performance compared to the SOTA synthesis tool FastSMT, the default Z3 solver, and the CVC5 solver on most benchmarks. Remarkably, on a challenging QF_BV benchmark set, Z3alpha solves 42.7% more instances than the default strategy in Z3.","doi":"10.24963/ijcai.2024/211","url":"https://ijcai.org/proceedings/2024/211","pdf":"https://ijcai.org/proceedings/2024/0211.pdf","markdown":"https://ijcai.org/proceedings/2024/211.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/211","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Constraint Satisfaction and Optimization","track":"Main Track","number":211,"pages":{"first":1907,"last":1915},"keywords":["Search: S: Heuristic search","Search: S: Algorithm portfolios and configuration","Knowledge Representation and Reasoning: KRR: Automated reasoning and theorem proving","Constraint Satisfaction and Optimization: CSO: Solvers and tools"],"videos":[],"references":[{"position":1,"raw":"[Balunovic et al., 2018] Mislav Balunovic, Pavol Bielik, and Martin Vechev. Learning to solve SMT formulas. In Advances in Neural Information Processing Systems 31, pages 10337-10348.","title":"Learning to solve SMT formulas","authors":"Balunovic","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barrett et al., 2016] Clark Barrett, Pascal Fontaine, and Ce- sare Tinelli. The satisfiability modulo theories library (SMT-LIB). http://smtlib.cs.uiowa.edu/index.shtml, 2016. Accessed: 2024-01-16.","title":"The satisfiability modulo theories library","authors":"Clark Barrett, Pascal Fontaine, Cesare Tinelli","year":"2016","venue":null,"doi":null,"url":"http://smtlib.cs.uiowa.edu/index.shtml","internal_url":null},{"position":3,"raw":"[Berzish et al., 2017] Murphy Berzish, Vijay Ganesh, and Yunhui Zheng. Z3str3: A string solver with theory-aware heuristics. In 2017 Formal Methods in Computer Aided Design (FMCAD), pages 55-59. IEEE, 2017.","title":"Z3str3: A string solver with theory-aware heuristics","authors":"Murphy Berzish, Vijay Ganesh, Yunhui Zheng","year":"2017","venue":"Formal Methods in Computer Aided Design (FMCAD)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Berzish et al., 2021] Murphy Berzish, Mitja Kulczynski, Federico Mora, Florin Manea, Joel D Day, Dirk Nowotka, and Vijay Ganesh. An SMT solver for regular expressions and linear arithmetic over string length. In International Conference on Computer Aided Verification, pages 289- 312. Springer, 2021.","title":"An SMT solver for regular expressions and linear arithmetic over string length","authors":"Murphy Berzish, Mitja Kulczynski, Federico Mora, Florin Manea, Joel D Day, Dirk Nowotka, Vijay Ganesh","year":"2021","venue":"International Conference on Computer Aided Verification","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bobot et al., 2023] Franc ¸ois Bobot, Martin Bromberger, and Jochen Hoenicke. SMT-COMP 2023. https:// smt-comp.github.io/2023/, 2023. Accessed: 2024-01-16.","title":"Franc ¸ois Bobot","authors":"Bobot","year":"2023","venue":null,"doi":null,"url":"https://smt-comp.github.io/2023/,2023","internal_url":null},{"position":6,"raw":"[Browne et al., 2012] Cameron B Browne, Edward Pow- ley, Daniel Whitehouse, Simon M Lucas, Peter I Cowl- ing, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in games, 4(1):1-43, 2012.","title":"A survey of monte carlo tree search methods","authors":"Edward Cameron B Browne, Daniel Powley, Whitehouse, M Simon, Peter I Lucas, Philipp Cowling, Stephen Rohlfshagen, Diego Tavener, Spyridon Perez, Simon Samothrakis, Colton","year":"2012","venue":"IEEE Transactions on Computational Intelligence and AI in games","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cadar et al., 2008] Cristian Cadar, Vijay Ganesh, Peter M Pawlowski, David L Dill, and Dawson R Engler. EXE: Au- tomatically generating inputs of death. ACM Transactions on Information and System Security (TISSEC), 12(2):1- 38, 2008.","title":"EXE: Automatically generating inputs of death","authors":"Cristian Cadar, Vijay Ganesh, M Peter, David L Pawlowski, Dawson R Dill, Engler","year":"2008","venue":"ACM Transactions on Information and System Security (TISSEC)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cameron et al., 2022] Chris Cameron, Jason Hartford, Tay- lor Lundy, Tuan Truong, Alan Milligan, Rex Chen, and Kevin Leyton-Brown. Monte carlo forest search: UNSAT solver synthesis via reinforcement learning. arXiv preprint arXiv:2211.12581, 2022.","title":"Monte carlo forest search: UNSAT solver synthesis via reinforcement learning","authors":"Chris Cameron, Jason Hartford, Taylor Lundy, Tuan Truong, Alan Milligan, Rex Chen, Kevin Leyton-Brown","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chen et al., 2021] Zhenbang Chen, Zehua Chen, Ziqi Shuai, Guofeng Zhang, Weiyu Pan, Yufeng Zhang, and Ji Wang. Synthesize solving strategy for symbolic execu- tion. In Proceedings of the 30th ACM SIGSOFT Interna- tional Symposium on Software Testing and Analysis, pages 348-360, 2021.","title":"Synthesize solving strategy for symbolic execution","authors":"Chen","year":"2021","venue":"Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[De Moura and Bjørner, 2008] Leonardo De Moura and Nikolaj Bjørner. Z3: An efficient SMT solver. In In- ternational conference on Tools and Algorithms for the Construction and Analysis of Systems, pages 337-340. Springer, 2008.","title":"Leonardo De Moura and Nikolaj Bjørner. Z3: An efficient SMT solver","authors":"De Moura, Bjørner","year":"2008","venue":"International conference on Tools and Algorithms for the Construction and Analysis of Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[De Moura and Bjørner, 2011] Leonardo De Moura and Nikolaj Bjørner. Satisfiability modulo theories: intro- duction and applications. Communications of the ACM, 54(9):69-77, 2011.","title":"Satisfiability modulo theories: introduction and applications","authors":"De Moura, Leonardo De Bjørner, Nikolaj Moura, Bjørner","year":"2011","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[De Moura and Passmore, 2013] Leonardo De Moura and Grant Olney Passmore. The strategy challenge in SMT solving. Automated Reasoning and Mathematics: Essays in Memory of William W. McCune, pages 15-44, 2013.","title":"The strategy challenge in SMT solving","authors":"De Moura, Leonardo De Passmore, Grant Moura, Olney Passmore","year":"2013","venue":"Automated Reasoning and Mathematics: Essays in Memory of William W. McCune","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[De Waard et al., 2016] Maarten De Waard, Diederik M Roi- jers, and Sander CJ Bakkes. Monte carlo tree search with options for general video game playing. In 2016 IEEE Conference on Computational Intelligence and Games (CIG), pages 1-8. IEEE, 2016.","title":"Monte carlo tree search with options for general video game playing","authors":"De Waard","year":"2016","venue":"IEEE Conference on Computational Intelligence and Games (CIG)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gelly and Silver, 2011] Sylvain Gelly and David Silver. Monte-carlo tree search and rapid action value estima- tion in computer go. Artificial Intelligence, 175(11):1856- 1875, 2011.","title":"Monte-carlo tree search and rapid action value estimation in computer go","authors":"Silver Gelly, Sylvain Gelly, David Silver","year":"2011","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gurfinkel et al., 2015] Arie Gurfinkel, Temesghen Kahsai, Anvesh Komuravelli, and Jorge A Navas. The SeaHorn verification framework. In International Conference on Computer Aided Verification, pages 343-361. Springer, 2015.","title":"The SeaHorn verification framework","authors":"Arie Gurfinkel, Temesghen Kahsai, Anvesh Komuravelli, Jorge A Navas","year":"2015","venue":"International Conference on Computer Aided Verification","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jha et al., 2024] Piyush Jha, Zhengyu Li, Zhengyang Lu, Curtis Bright, and Vijay Ganesh. AlphaMapleSAT: An MCTS-based cube-and-conquer SAT solver for hard com- binatorial problems. arXiv preprint arXiv:2401.13770, 2024.","title":"AlphaMapleSAT: An MCTS-based cube-and-conquer SAT solver for hard combinatorial problems","authors":"Piyush Jha, Zhengyu Li, Zhengyang Lu, Curtis Bright, Vijay Ganesh","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Johanson, 2013] Michael Johanson. Measuring the size of large no-limit poker games. arXiv preprint arXiv:1302.7008, 2013.","title":"Measuring the size of large no-limit poker games","authors":"Michael Johanson","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Khalil et al., 2022] Elias B Khalil, Pashootan Vaezipoor, and Bistra Dilkina. Finding backdoors to integer pro- grams: a monte carlo tree search framework. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 36, pages 3786-3795, 2022.","title":"Finding backdoors to integer programs: a monte carlo tree search framework","authors":"Pashootan Elias B Khalil, Bistra Vaezipoor, Dilkina","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kocsis and Szepesvári, 2006] Levente Kocsis and Csaba Szepesvári. Bandit based monte-carlo planning. In Eu- ropean conference on machine learning, pages 282-293. Springer, 2006.","title":"Bandit based monte-carlo planning","authors":"Szepesvári Kocsis, Levente Kocsis, Csaba Szepesvári","year":"2006","venue":"European conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Koza, 1994] John R Koza. Genetic programming as a means for programming computers by natural selection. Statistics and computing, 4:87-112, 1994.","title":"Genetic programming as a means for programming computers by natural selection","authors":"John R Koza","year":"1994","venue":"Statistics and computing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kroening and Strichman, 2016] Daniel Kroening and Ofer Strichman. Decision Procedures: An Alogorithm Point of View. Springer, 2016.","title":"Decision Procedures: An Alogorithm Point of View","authors":"Strichman Kroening, Daniel Kroening, Ofer Strichman","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lu, 2023] Zhengyang Lu. AlphaSMT: A reinforcement learning guided SMT solver. Master's thesis, University of Waterloo, 2023.","title":"AlphaSMT: A reinforcement learning guided SMT solver","authors":"Lu ;, Zhengyang Lu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mankowitz et al., 2023] Daniel J Mankowitz, Andrea Michi, Anton Zhernov, Marco Gelmi, Marco Selvi, Cosmin Paduraru, Edouard Leurent, Shariq Iqbal, Jean- Baptiste Lespiau, Alex Ahern, et al. Faster sorting algorithms discovered using deep reinforcement learning. Nature, 618(7964):257-263, 2023.","title":"Faster sorting algorithms discovered using deep reinforcement learning","authors":"Andrea Daniel J Mankowitz, Anton Michi, Marco Zhernov, Marco Gelmi, Cosmin Selvi, Edouard Paduraru, Shariq Leurent, Jean-Baptiste Iqbal, Alex Lespiau, Ahern","year":"2023","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Microsoft, 2023] Microsoft. Online Z3 guide. https:// microsoft.github.io/z3guide/, 2023. Accessed: 2024-01- 16.","title":"Microsoft. Online Z3 guide","authors":null,"year":"2023","venue":null,"doi":null,"url":"https://microsoft.github.io/z3guide/,2023","internal_url":null},{"position":25,"raw":"[Mora et al., 2021] Federico Mora, Murphy Berzish, Mitja Kulczynski, Dirk Nowotka, and Vijay Ganesh. Z3str4: A multi-armed string solver. In Formal Methods: 24th Inter- national Symposium, FM 2021, Virtual Event, November 20-26, 2021, Proceedings 24, pages 389-406. Springer, 2021.","title":"Z3str4: A multi-armed string solver","authors":"Federico Mora, Murphy Berzish, Mitja Kulczynski, Dirk Nowotka, Vijay Ganesh","year":"2021","venue":"Formal Methods: 24th International Symposium, FM 2021, Virtual Event","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Poole and Mackworth, 2010] David L Poole and Alan K Mackworth. Artificial Intelligence: foundations of com- putational agents. Cambridge University Press, 2010.","title":"Artificial Intelligence: foundations of computational agents","authors":"L David, Alan K Poole, Mackworth","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Pulina and Tacchella, 2012] Luca Pulina and Armando Tac- chella. Challenging SMT solvers to verify neural net- works. Ai Communications, 25(2):117-135, 2012.","title":"Luca Pulina and Armando Tacchella. Challenging SMT solvers to verify neural networks","authors":"Tacchella Pulina","year":"2012","venue":"Ai Communications","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Ramírez et al., 2016] Nicolás Gálvez Ramírez, Youssef Hamadi, Eric Monfroy, and Frédéric Saubion. Evolving SMT strategies. In 2016 IEEE 28th International Confer- ence on Tools with Artificial Intelligence (ICTAI), pages 247-254. IEEE, 2016.","title":"Evolving SMT strategies","authors":"Nicolás Gálvez Ramírez, Youssef Hamadi, Eric Monfroy, Frédéric Saubion","year":"2016","venue":"IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Robbins, 1952] Herbert Robbins. Some aspects of the se- quential design of experiments. Bull. Amer. Math. Soc., 58(5):527-535, 1952.","title":"Herbert Robbins. Some aspects of the sequential design of experiments","authors":"Robbins","year":"1952","venue":"Bull. Amer. Math. Soc","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Ross et al., 2011] Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and struc- tured prediction to no-regret online learning. In Proceed- ings of the fourteenth international conference on artificial intelligence and statistics, pages 627-635. JMLR Work- shop and Conference Proceedings, 2011.","title":"A reduction of imitation learning and structured prediction to no-regret online learning","authors":"Stéphane Ross, Geoffrey Gordon, Drew Bagnell","year":"2011","venue":"Proceedings of the fourteenth international conference on artificial intelligence and statistics","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Sabharwal et al., 2012] Ashish Sabharwal, Horst Samu- lowitz, and Chandra Reddy. Guiding combinatorial op- timization with UCT. In Integration of AI and OR Tech- niques in Contraint Programming for Combinatorial Opti- mzation Problems: 9th International Conference, CPAIOR 2012, Nantes, France, May 28-June1, 2012. Proceedings 9, pages 356-361. Springer, 2012.","title":"Guiding combinatorial optimization with UCT","authors":"Ashish Sabharwal, Horst Samulowitz, Chandra Reddy","year":"2012","venue":"Integration of AI and OR Techniques in Contraint Programming for Combinatorial Optimzation Problems: 9th International Conference, CPAIOR 2012","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Silver et al., 2016] David Silver, Aja Huang, Chris J Maddi- son, Arthur Guez, Laurent Sifre, George Van Den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Pan- neershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484-489, 2016.","title":"Mastering the game of go with deep neural networks and tree search","authors":"David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den, Marc Driessche, Lanctot","year":"2016","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Silver et al., 2017] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815, 2017.","title":"Mastering chess and shogi by self-play with a general reinforcement learning algorithm","authors":"David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Song et al., 2008] Dawn Song, David Brumley, Heng Yin, Juan Caballero, Ivan Jager, Min Gyung Kang, Zhenkai Liang, James Newsome, Pongsin Poosankam, and Prateek Saxena. BitBlaze: A new approach to computer secu- rity via binary analysis. In Information Systems Security: 4th International Conference, ICISS 2008, Hyderabad, In- dia, December 16-20, 2008. Proceedings 4, pages 1-25. Springer, 2008.","title":"BitBlaze: A new approach to computer security via binary analysis","authors":"Dawn Song, David Brumley, Juan Heng Yin, Ivan Caballero, Min Gyung Jager, Zhenkai Kang, James Liang, Pongsin Newsome, Prateek Poosankam, Saxena","year":"2008","venue":"Information Systems Security: 4th International Conference, ICISS 2008","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Sun et al., 2023] Fangzheng Sun, Yang Liu, Jian-Xun Wang, and Hao Sun. Symbolic physics learner: Discov- ering governing equations via monte carlo tree search. In The Eleventh International Conference on Learning Rep- resentations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.","title":"Symbolic physics learner: Discovering governing equations via monte carlo tree search","authors":"Fangzheng Sun, Yang Liu, Jian-Xun Wang, Hao Sun","year":"2023","venue":"The Eleventh International Conference on Learning Representations, ICLR 2023","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Sutton et al., 1999] Richard S Sutton, Doina Precup, and Satinder Singh. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial intelligence, 112(1-2):181-211, 1999.","title":"Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning","authors":"Doina Richard S Sutton, Satinder Precup, Singh","year":"1999","venue":"Artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang et al., 2018] Zhenya Zhang, Gidon Ernst, Sean Sed- wards, Paolo Arcaini, and Ichiro Hasuo. Two-layered fal- sification of hybrid systems guided by monte carlo tree search. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 37(11):2894-2905, 2018.","title":"Two-layered falsification of hybrid systems guided by monte carlo tree search","authors":"Zhenya Zhang, Gidon Ernst, Sean Sedwards, Paolo Arcaini, Ichiro Hasuo","year":"2018","venue":"IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}