{"type":"ScholarlyArticle","title":"Constrained Sequential Inference in Machine Learning Using Constraint Programming","authors":[{"name":"Virasone Manibod","slug":"virasone-manibod","given":"Virasone","family":"Manibod","affiliation":""},{"name":"David Saikali","slug":"david-saikali","given":"David","family":"Saikali","affiliation":""},{"name":"Gilles Pesant","slug":"gilles-pesant","given":"Gilles","family":"Pesant","affiliation":""}],"abstract":"Sequence models in machine learning often struggle to exhibit long-term structure. We consider this problem at inference time in the context of enforcing constraints that are not necessarily featured in the dataset on which the generative model was trained. The difficulty lies in imposing previously-unseen structure while staying close to the training dataset. It is particularly hard for long-term structure, which requires balancing foresight over many yet-to-be generated tokens and the immediacy of next-token predictions from the sequence model. We address this problem by introducing our neurosymbolic framework GeAI-BLAnC. The learned probabilities of the sequence model are mixed in with the marginal probabilities computed from a constraint programming / belief propagation framework applied to a constraint programming model expressing the desired structure.\r\nThe next predicted token is then selected from the resulting probability distribution. Experiments in the context of molecule and music generation show that we can achieve the  structure imposed post-training without straying too much from the structure of the dataset learned during training.","doi":"10.24963/ijcai.2025/293","url":"https://ijcai.org/proceedings/2025/293","pdf":"https://ijcai.org/proceedings/2025/0293.pdf","markdown":"https://ijcai.org/proceedings/2025/293.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/293","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Constraint Satisfaction and Optimization","track":"Main Track","number":293,"pages":{"first":2628,"last":2636},"keywords":["Machine Learning: ML: Structured prediction","Constraint Satisfaction and Optimization: CSO: Constraint programming","Machine Learning: ML: Neuro-symbolic methods/Abductive Learning","Machine Learning: ML: Generative models"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}