{"type":"ScholarlyArticle","title":"Scaling Up Unbiased Search-based Symbolic Regression","authors":[{"name":"Paul Kahlmeyer","slug":"paul-kahlmeyer","given":"Paul","family":"Kahlmeyer","affiliation":"University Jena"},{"name":"Joachim Giesen","slug":"joachim-giesen","given":"Joachim","family":"Giesen","affiliation":"University Jena"},{"name":"Michael Habeck","slug":"michael-habeck","given":"Michael","family":"Habeck","affiliation":"University Hospital Jena"},{"name":"Henrik Voigt","slug":"henrik-voigt","given":"Henrik","family":"Voigt","affiliation":"University Jena"}],"abstract":"In a regression task, a function is learned from labeled data to predict the labels at new data points. The goal is to achieve small prediction errors. In symbolic regression, the goal is more ambitious, namely, to learn an interpretable function that makes small prediction errors. This additional goal largely rules out the standard approach used in regression, that is, reducing the learning problem to learning parameters of an expansion of basis functions by optimization. Instead, symbolic regression methods search for a good solution in a space of symbolic expressions. To cope with the typically vast search space, most symbolic regression methods make implicit, or sometimes even explicit, assumptions about its structure. Here, we argue that the only obvious structure of the search space is that it contains small expressions, that is, expressions that can be decomposed into a few subexpressions. We show that systematically searching spaces of small expressions finds solutions that are more accurate and more robust against noise than those obtained by state-of-the-art symbolic regression methods. In particular, systematic search outperforms state-of-the-art symbolic regressors in terms of its ability to recover the true underlying symbolic expressions on established benchmark data sets.","doi":"10.24963/ijcai.2024/471","url":"https://ijcai.org/proceedings/2024/471","pdf":"https://ijcai.org/proceedings/2024/0471.pdf","markdown":"https://ijcai.org/proceedings/2024/471.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/471","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":471,"pages":{"first":4264,"last":4272},"keywords":["Machine Learning: ML: Explainable/Interpretable machine learning","Machine Learning: ML: Regression","Search: S: Search and machine learning","Machine Learning: ML: Symbolic methods"],"videos":[],"references":[{"position":1,"raw":"[Aho et al., 1986] Alfred V. Aho, Ravi Sethi, and Jeffrey D. Ullman. Compilers: Principles, Techniques, and Tools. Addison-Wesley, 1986.","title":"Compilers: Principles, Techniques, and Tools","authors":"Alfred V Aho, Ravi Sethi, Jeffrey D Ullman","year":"1986","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aryadoust, 2015] Vahid Aryadoust. Application of evolu- tionary algorithm-based symbolic regression to language assessment: Toward nonlinear modeling. Psychological Test and Assessment Modeling, 57(3):301, 2015.","title":"Application of evolutionary algorithm-based symbolic regression to language assessment: Toward nonlinear modeling","authors":"Vahid Aryadoust","year":"2015","venue":"Psychological Test and Assessment Modeling","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bartlett et al., 2023] Deaglan J Bartlett, Harry Desmond, and Pedro G Ferreira. Exhaustive symbolic regression. IEEE Transactions on Evolutionary Computation, 2023.","title":"Exhaustive symbolic regression","authors":"Bartlett","year":"2023","venue":"IEEE Transactions on Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Can and Heavey, 2011] Birkan Can and Cathal Heavey. Comparison of experimental designs for simulation-based symbolic regression of manufacturing systems. Comput- ers & Industrial Engineering, 61(3):447-462, 2011.","title":"Comparison of experimental designs for simulation-based symbolic regression of manufacturing systems","authors":"Heavey Can, Birkan Can, Cathal Heavey","year":"2011","venue":"Computers & Industrial Engineering","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cocke, 1970] John Cocke. Global common subexpression elimination. In Robert S. Northcote, editor, Proceedings of a Symposium on Compiler Optimization, pages 20-24. ACM, 1970.","title":"Global common subexpression elimination","authors":"John Cocke","year":"1970","venue":"Proceedings of a Symposium on Compiler Optimization","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feynman et al., 2011] Richard P. Feynman, Robert B. Leighton, and Matthew Sands. The Feynman Lectures on Physics, Vol. II: The New Millennium Edition: Mainly Electromagnetism and Matter. The Feynman Lectures on Physics. Basic Books, 2011.","title":"The New Millennium Edition: Mainly Electromagnetism and Matter","authors":"Richard P Feynman, Robert B Leighton, Matthew Sands","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Guimerà et al., 2020] Roger Guimerà, Ignasi Reichardt, Antoni Aguilar-Mogas, Francesco A. Massucci, Manuel Miranda, Jordi Pallarès, and Marta Sales-Pardo. A bayesian machine scientist to aid in the solution of challenging scientific problems. Science Advances, 6(5):eaav6971, 2020.","title":"A bayesian machine scientist to aid in the solution of challenging scientific problems","authors":"Roger Guimerà, Ignasi Reichardt, Antoni Aguilar-Mogas, Francesco A Massucci, Manuel Miranda, Jordi Pallarès, Marta Sales-Pardo","year":"2020","venue":"Science Advances","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hoerl and Kennard, 1970] Arthur E. Hoerl and Robert W. Kennard. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics, 12:55-67, 1970.","title":"Ridge Regression: Biased Estimation for Nonorthogonal Problems","authors":"Arthur E Hoerl, Robert W Kennard","year":"1970","venue":"Technometrics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Holland, 1975] John H. Holland. Adaptation in Natural and Artificial Systems. University of Michigan Press, 1975. second edition, 1992.","title":"Adaptation in Natural and Artificial Systems","authors":"John H Holland","year":"1975","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jaccard, 1902] Paul Jaccard. Lois de distribution florale dans la zone alpine. Bulletin de la Société vaudoise des sciences naturelles, 38:69-130, 01 1902.","title":"Lois de distribution florale dans la zone alpine","authors":"Paul Jaccard","year":"1902","venue":"Bulletin de la Société vaudoise des sciences naturelles","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jin et al., 2020] Ying Jin, Weilin Fu, Jian Kang, Jiadong Guo, and Jian Guo. Bayesian symbolic regression, 2020. [Kamienny et al., 2022] Pierre-Alexandre Kamienny, Stéphane d'Ascoli, Guillaume Lample, and Francois Charton. End-to-end symbolic regression with trans- formers. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.","title":"End-to-end symbolic regression with transformers","authors":"Ying Jin, Weilin Fu, Jian Kang, Jiadong Guo, Jian Guo, ; Kamienny","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kammerer et al., 2020] Lukas Kammerer, Gabriel Kron- berger, Bogdan Burlacu, Stephan M Winkler, Michael Kommenda, and Michael Affenzeller. Symbolic regres- sion by exhaustive search: Reducing the search space us- ing syntactical constraints and efficient semantic structure deduplication. Genetic programming theory and practice XVII, pages 79-99, 2020.","title":"Symbolic regression by exhaustive search: Reducing the search space using syntactical constraints and efficient semantic structure deduplication","authors":"Kammerer","year":"2020","venue":"Genetic programming theory and practice XVII","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Keren et al., 2023] Liron Simon Keren, Alex Liberzon, and Teddy Lazebnik. A computational framework for physics- informed symbolic regression with straightforward in- tegration of domain knowledge. Scientific Reports, 13(1):1249, 2023.","title":"A computational framework for physicsinformed symbolic regression with straightforward integration of domain knowledge","authors":"Liron Simon, Keren, Alex Liberzon, Teddy Lazebnik","year":"2023","venue":"Scientific Reports","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kommenda et al., 2020] Michael Kommenda, Bogdan Burlacu, Gabriel Kronberger, and Michael Affenzeller. Parameter identification for symbolic regression using nonlinear least squares. Genetic Programming and Evolvable Machines, 21(3):471-501, 2020.","title":"Parameter identification for symbolic regression using nonlinear least squares","authors":"Michael Kommenda, Bogdan Burlacu, Gabriel Kronberger, Michael Affenzeller","year":"2020","venue":"Genetic Programming and Evolvable Machines","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Koza, 1994] John R Koza. Genetic programming as a means for programming computers by natural selection. Statistics and computing, 4:87-112, 1994.","title":"Genetic programming as a means for programming computers by natural selection","authors":"John R Koza","year":"1994","venue":"Statistics and computing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[La Cava et al., 2016] William La Cava, Lee Spector, and Kourosh Danai. Epsilon-lexicase selection for regression. In Proceedings of the Genetic and Evolutionary Computa- tion Conference 2016, GECCO '16, page 741-748, New York, NY, USA, 2016. Association for Computing Ma- chinery.","title":"Epsilon-lexicase selection for regression","authors":"La Cava","year":"2016","venue":"Proceedings of the Genetic and Evolutionary Computation Conference 2016, GECCO '16","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[La Cava et al., 2021] William La Cava, Patryk Orze- chowski, Bogdan Burlacu, Fabricio de Franca, Marco Vir- golin, Ying Jin, Michael Kommenda, and Jason Moore. Contemporary symbolic regression methods and their rel- ative performance. In J. Vanschoren and S. Yeung, ed- itors, Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1.","title":"Contemporary symbolic regression methods and their relative performance","authors":"La William, Patryk Cava, Bogdan Orzechowski, Fabricio Burlacu, Marco De Franca, Ying Virgolin, Michael Jin, Jason Kommenda, Moore","year":null,"venue":"Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"Curran, 2021.","title":null,"authors":"Curran","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Landajuela et al., 2021] Mikel Landajuela, Brenden K Pe- tersen, Sookyung Kim, Claudio P Santiago, Ruben Glatt, Nathan Mundhenk, Jacob F Pettit, and Daniel Faissol. Dis- covering symbolic policies with deep reinforcement learn- ing. In Marina Meila and Tong Zhang, editors, Pro- ceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learn- ing Research, pages 5979-5989. PMLR, 18-24 Jul 2021.","title":"Discovering symbolic policies with deep reinforcement learning","authors":"Mikel Landajuela, Sookyung Brenden K Petersen, Claudio P Kim, Ruben Santiago, Nathan Glatt, Jacob F Mundhenk, Daniel Pettit, Faissol","year":"2021","venue":"Proceedings of the 38th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Landajuela et al., 2022] Mikel Landajuela, Chak Shing Lee, Jiachen Yang, Ruben Glatt, Claudio P Santiago, Ig- nacio Aravena, Terrell Mundhenk, Garrett Mulcahy, and Brenden K Petersen. A unified framework for deep sym- bolic regression. In S. Koyejo, S. Mohamed, A. Agar- wal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 33985-33998.","title":"A unified framework for deep symbolic regression","authors":"Mikel Landajuela, Chak Shing Lee, Jiachen Yang, Ruben Glatt, Claudio P Santiago, Ignacio Aravena, Terrell Mundhenk, Garrett Mulcahy, Brenden K Petersen","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"Curran Associates, Inc., 2022.","title":null,"authors":"Curran Associates, Inc","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu and Tegmark, 2021] Ziming Liu and Max Tegmark. Machine Learning Conservation Laws from Trajectories. Physical Review Letters, 126(18):180604, May 2021.","title":"Machine Learning Conservation Laws from Trajectories","authors":"Ziming Liu, Max Tegmark","year":"2021","venue":"Physical Review Letters","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu and Tegmark, 2022] Ziming Liu and Max Tegmark. Machine-learning hidden symmetries. Physical Review Letters, 128(18):180201, May 2022.","title":"Machine-learning hidden symmetries","authors":"Ziming Liu, Max Tegmark","year":"2022","venue":"Physical Review Letters","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Meurer et al., 2017] Aaron Meurer, Christopher P. Smith, Mateusz Paprocki, Ondřej Čertík, Sergey B. Kirpichev, Matthew Rocklin, AMiT Kumar, Sergiu Ivanov, Jason K. Moore, Sartaj Singh, Thilina Rathnayake, Sean Vig, Brian E. Granger, Richard P. Muller, Francesco Bonazzi, Harsh Gupta, Shivam Vats, Fredrik Johansson, Fabian Pedregosa, Matthew J. Curry, Andy R. Terrel, Štěpán Roučka, Ashutosh Saboo, Isuru Fernando, Sumith Kulal, Robert Cimrman, and Anthony Scopatz. Sympy: symbolic computing in python. PeerJ Computer Science, 3:e103, January 2017.","title":"Sympy: symbolic computing in python","authors":"Aaron Meurer, Christopher P Smith, Mateusz Paprocki, Ondřej Čertík, B Sergey, Matthew Kirpichev, Amit Rocklin, Sergiu Kumar, Jason K Ivanov, Sartaj Moore, Thilina Singh, Sean Rathnayake, Brian E Vig, Richard P Granger, Francesco Muller, Harsh Bonazzi, Shivam Gupta, Fredrik Vats, Fabian Johansson, Matthew J Pedregosa, Andy R Curry, Štěpán Terrel, Ashutosh Roučka, Isuru Saboo, Sumith Fernando, Robert Kulal, Anthony Cimrman, Scopatz","year":"2017","venue":"PeerJ Computer Science","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mundhenk et al., 2021] T. Mundhenk, Mikel Landajuela, Ruben Glatt, Claudio Santiago, Daniel Faissol, and Bren- den Petersen. Symbolic regression via neural-guided ge- netic programming population seeding. 11 2021.","title":"Symbolic regression via neural-guided genetic programming population seeding","authors":"T Mundhenk, Mikel Landajuela, Ruben Glatt, Claudio Santiago, Daniel Faissol, Brenden Petersen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Petersen et al., 2021] Brenden K Petersen, Mikel Landa- juela Larma, Terrell N. Mundhenk, Claudio Prata Santi- ago, Soo Kyung Kim, and Joanne Taery Kim. Deep sym- bolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. In Interna- tional Conference on Learning Representations, 2021.","title":"Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients","authors":"Mikel Landajuela Brenden K Petersen, Terrell N Larma, Claudio Prata Mundhenk, Soo Santiago, Kyung Kim, Joanne Taery Kim","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Quade et al., 2016] Markus Quade, Markus Abel, Kamran Shafi, Robert K. Niven, and Bernd R. Noack. Prediction of dynamical systems by symbolic regression. Physical Review E, 94:012214, 2016.","title":"Prediction of dynamical systems by symbolic regression","authors":"Markus Quade, Markus Abel, Kamran Shafi, Robert K Niven, Bernd R Noack","year":"2016","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Schmidt and Lipson, 2007] Michael Schmidt and Hod Lip- son. Comparison of tree and graph encodings as function of problem complexity. In Proceedings of the 9th An- nual Conference on Genetic and Evolutionary Computa- tion, GECCO '07, page 1674-1679, New York, NY, USA, 2007. Association for Computing Machinery.","title":"Michael Schmidt and Hod Lipson. Comparison of tree and graph encodings as function of problem complexity","authors":"Lipson Schmidt","year":"2007","venue":"Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO '07","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Schmidt and Lipson, 2009] Michael Schmidt and Hod Lip- son. Distilling free-form natural laws from experimental data. Science, 324(5923):81-85, 2009.","title":"Michael Schmidt and Hod Lipson. Distilling free-form natural laws from experimental data","authors":"Lipson Schmidt","year":"2009","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Stephens, 2016] Trevor Stephens. Genetic programming in python, with a scikit-learn inspired api: gplearn, 2016.","title":"Genetic programming in python, with a scikit-learn inspired api: gplearn","authors":"Trevor Stephens","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Strogatz, 2000] Steven H. Strogatz. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chem- istry and Engineering. Westview Press, 2000.","title":"Nonlinear Dynamics and Chaos: With Applications to Physics","authors":"H Steven, Strogatz","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tibshirani, 1996] Robert Tibshirani. Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statisti- cal Society (Series B), 58:267-288, 1996.","title":"Regression Shrinkage and Selection via the Lasso","authors":"Robert Tibshirani","year":"1996","venue":"Journal of the Royal Statistical Society (Series B)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Truscott and Korns, 2011] Philip D. Truscott and Michael F. Korns. Detecting shadow economy sizes with symbolic regression. Genetic Programming Theory and Practice IX, pages 195-210, 2011).","title":"Detecting shadow economy sizes with symbolic regression","authors":"Philip D Truscott, Michael F Korns","year":"2011","venue":"Genetic Programming Theory and Practice IX","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Udrescu and Tegmark, 2020] Silviu-Marian Udrescu and Max Tegmark. Ai feynman: A physics-inspired method for symbolic regression. Science Advances, 6(16):eaay2631, 2020.","title":"Ai feynman: A physics-inspired method for symbolic regression","authors":"Marian Silviu, Max Udrescu, Tegmark","year":"2020","venue":"Science Advances","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Udrescu et al., 2020] Silviu-Marian Udrescu, Andrew Tan, Jiahai Feng, Orisvaldo Neto, Tailin Wu, and Max Tegmark. Ai feynman 2.0: Pareto-optimal symbolic re- gression exploiting graph modularity. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, vol- ume 33, pages 4860-4871.","title":"Ai feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity","authors":"Silviu-Marian, Andrew Udrescu, Jiahai Tan, Orisvaldo Feng, Tailin Neto, Max Wu, Tegmark","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"Curran Associates, Inc., 2020.","title":null,"authors":"Curran Associates, Inc","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Virgolin et al., 2021] Marco Virgolin, Tanja Alderliesten, Cees Witteveen, and Peter AN Bosman. Improving model- based genetic programming for symbolic regression of small expressions. Evolutionary computation, 29(2):211- 237, 2021.","title":"Improving modelbased genetic programming for symbolic regression of small expressions","authors":"Marco Virgolin, Tanja Alderliesten, Cees Witteveen, A N Peter, Bosman","year":"2021","venue":"Evolutionary computation","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wang et al., 2019] Yiqun Wang, Nicholas Wagner, and James M. Rondinelli. Symbolic regression in materials science. MRS Communications, 9(3):793-805, 2019.","title":"Symbolic regression in materials science","authors":"Yiqun Wang, Nicholas Wagner, James M Rondinelli","year":"2019","venue":"MRS Communications","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Wright, 1921] Sewall Wright. Correlation and causation. Journal of Agricultural Research, 20(3):557-585, 1921.","title":"Correlation and causation","authors":"Wright ;, Sewall Wright","year":"1921","venue":"Journal of Agricultural Research","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}