{"type":"ScholarlyArticle","title":"InnateCoder: Learning Programmatic Options with Foundation Models","authors":[{"name":"Rubens O. Moraes","slug":"rubens-o-moraes","given":"Rubens O.","family":"Moraes","affiliation":""},{"name":"Quazi Asif Sadmine","slug":"quazi-asif-sadmine","given":"Quazi","family":"Asif Sadmine","affiliation":""},{"name":"Hendrik Baier","slug":"hendrik-baier","given":"Hendrik","family":"Baier","affiliation":""},{"name":"Levi H. S. Lelis","slug":"levi-h-s-lelis","given":"Levi H. S.","family":"Lelis","affiliation":""}],"abstract":"Outside of transfer learning settings, reinforcement learning agents start their learning process from a clean slate. As a result, such agents have to go through a slow process to learn even the most obvious skills required to solve a problem. In this paper, we present InnateCoder, a system that leverages human knowledge encoded in foundation models to provide programmatic policies that encode ``innate skills'' in the form of temporally extended actions, or options. In contrast to existing approaches to learning options, InnateCoder learns them from the general human knowledge encoded in foundation models in a zero-shot setting, and not from the knowledge the agent gains by interacting with the environment. Then, InnateCoder searches for a programmatic policy by combining the programs encoding these options into larger and more complex programs. We hypothesized that InnateCoder's way of learning and using options could improve the sampling efficiency of current methods for learning programmatic policies. Empirical results in MicroRTS and Karel the Robot support our hypothesis, since they show that InnateCoder is more sample efficient than versions of the system that do not use options or learn them from experience.","doi":"10.24963/ijcai.2025/851","url":"https://ijcai.org/proceedings/2025/851","pdf":"https://ijcai.org/proceedings/2025/0851.pdf","markdown":"https://ijcai.org/proceedings/2025/851.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/851","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":851,"pages":{"first":7652,"last":7660},"keywords":["Multidisciplinary Topics and Applications: MTA: Computer games","Planning and Scheduling: PS: Markov decisions processes","Machine Learning: ML: Reinforcement learning","Search: S: Game playing"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}