{"type":"ScholarlyArticle","title":"Synthesis and Verification of Transformer Programs","authors":[{"name":"Hongjian Jiang","slug":"hongjian-jiang","given":"Hongjian","family":"Jiang","affiliation":""},{"name":"Matthew Hague","slug":"matthew-hague","given":"Matthew","family":"Hague","affiliation":""},{"name":"Philipp Rümmer","slug":"philipp-rummer","given":"Philipp","family":"Rümmer","affiliation":""},{"name":"Anthony W. Lin","slug":"anthony-w-lin","given":"Anthony W.","family":"Lin","affiliation":""}],"abstract":"C-RASP is a simple programming language that was recently shown to capture \r\nconcepts expressible by transformers.\r\nIn this paper, we develop new algorithmic techniques for automatically \r\nverifying C-RASPs. \r\nTo this end, we establish a connection to the \r\nverification of synchronous dataflow programs in Lustre, which enables us to \r\nexploit state-of-the-art model checkers utilizing highly optimized \r\nSMT-solvers. Our second contribution addresses learning a \r\nC-RASP program in the first place. To this end, we provide a new algorithm \r\nfor learning a C-RASP from examples using local search. \r\nWe demonstrate efficacy of our implementation for benchmarks of C-RASPs in the\r\nliterature, in particular in connection to the following applications:\r\n(1) transformer program optimization, and (2) \r\nconstrained learning of transformer programs (based on a partial specification).","doi":"10.24963/ijcai.2026/434","url":"https://ijcai.org/proceedings/2026/434","pdf":"https://ijcai.org/proceedings/2026/0434.pdf","markdown":"https://ijcai.org/proceedings/2026/434.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/434","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Knowledge Representation and Reasoning","track":"Main Track","number":434,"pages":{"first":3899,"last":3907},"keywords":["Knowledge Representation and Reasoning: Knowledge representation languages","Knowledge Representation and Reasoning: Learning and reasoning","Knowledge Representation and Reasoning: Automated reasoning and theorem proving","Machine Learning: Attention models"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}