{"type":"ScholarlyArticle","title":"Markov Constraint as Large Language Model Surrogate","authors":[{"name":"Alexandre Bonlarron","slug":"alexandre-bonlarron","given":"Alexandre","family":"Bonlarron","affiliation":"Université Côte d'Azur , Inria , France; Université Côte d'Azur , CNRS , I3S , France"},{"name":"Jean-Charles Régin","slug":"jean-charles-regin","given":"Jean-Charles","family":"Régin","affiliation":"Université Côte d'Azur , CNRS , I3S , France"}],"abstract":"This paper presents NgramMarkov, a variant of the Markov constraints. It is dedicated to text generation in constraint programming (CP).\r\nIt involves a set of n-grams (i.e., sequence of n words) associated with probabilities given by a large language model (LLM). It limits the product of the probabilities of the n-gram of a sentence. \r\n\r\nThe propagator of this constraint can be seen as an extension of the ElementaryMarkov constraint propagator, incorporating the LLM distribution instead of the maximum likelihood estimation of n-grams. It uses a gliding threshold, i.e., it rejects n-grams whose local probabilities are too low, to guarantee balanced solutions.\r\nIt can also be combined with a \"look-ahead\" approach to remove n-grams that are very unlikely to lead to acceptable sentences for a fixed-length horizon. This idea is based on the MDDMarkovProcess constraint propagator, but without explicitly using an MDD (Multi-Valued Decision Diagram).\r\n\r\nThe experimental results show that the generated text is valued in a similar way to the LLM perplexity function.\r\nUsing this new constraint dramatically reduces the number of candidate sentences produced, improves computation times, and allows larger corpora or smaller n-grams to be used. A real-world problem has been solved for the first time using 4-grams instead of 5-grams.","doi":"10.24963/ijcai.2024/204","url":"https://ijcai.org/proceedings/2024/204","pdf":"https://ijcai.org/proceedings/2024/0204.pdf","markdown":"https://ijcai.org/proceedings/2024/204.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/204","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Constraint Satisfaction and Optimization","track":"Main Track","number":204,"pages":{"first":1844,"last":1852},"keywords":["Constraint Satisfaction and Optimization: CSO: Applications","Constraint Satisfaction and Optimization: CSO: Modeling","Natural Language Processing: NLP: Language generation","Constraint Satisfaction and Optimization: CSO: Constraint programming"],"videos":[],"references":[{"position":1,"raw":"[Bonlarron et al., 2023] Alexandre Bonlarron, Aurélie Cal- abrèse, Pierre Kornprobst, and Jean-Charles Régin. Con- straints first: A new mdd-based model to generate sen- tences under constraints. In Edith Elkind, editor, Proceed- ings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pages 1893-1901. In- ternational Joint Conferences on Artificial Intelligence Or- ganization, 8 2023. Main Track.","title":"Constraints first: A new mdd-based model to generate sentences under constraints","authors":"Alexandre Bonlarron, Aurélie Calabrèse, Pierre Kornprobst, Jean-Charles Régin","year":"2023","venue":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23","doi":null,"url":null,"internal_url":"/proceedings/2023/210"},{"position":2,"raw":"[Dao et al., 2022] Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35:16344-16359, 2022.","title":"Flashattention: Fast and memory-efficient exact attention with io-awareness","authors":"Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, Christopher Ré","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Demassey et al., 2006] Sophie Demassey, Gilles Pesant, and Louis-Martin Rousseau. A cost-regular based hybrid column generation approach. Constraints, 11(4):315-333, 2006.","title":"A cost-regular based hybrid column generation approach","authors":"Sophie Demassey, Gilles Pesant, Louis-Martin Rousseau","year":"2006","venue":"Constraints","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gehrmann et al., 2019] Sebastian Gehrmann, Hendrik Stro- belt, and Alexander Rush. GLTR: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Compu- tational Linguistics: System Demonstrations, pages 111- 116, Florence, Italy, July 2019. Association for Computa- tional Linguistics.","title":"GLTR: Statistical detection and visualization of generated text","authors":"Sebastian Gehrmann, Hendrik Strobelt, Alexander Rush","year":"2019","venue":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long Short-Term Memory. Neural Computation, 9(8):1735-1780, 11 1997.","title":"Long Short-Term Memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Jacob et al., 2018] Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.","title":"Quantization and training of neural networks for efficient integer-arithmetic-only inference","authors":"Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, Dmitry Kalenichenko","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jurafsky and Martin, 2009] Dan Jurafsky and James H. Martin. Speech and language processing : an introduction to natural language processing, computational linguistics, and speech recognition. Pearson Prentice Hall, Upper Sad- dle River, N.J., 2009.","title":"Speech and language processing : an introduction to natural language processing, computational linguistics, and speech recognition","authors":"Dan Jurafsky, James H Martin","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2021] Yixian Liu, Liwen Zhang, Wenjuan Han, Yue Zhang, and Kewei Tu. Constrained text generation with global guidance -case study on commongen. CoRR, abs/2103.07170, 2021.","title":"Constrained text generation with global guidance -case study on commongen","authors":"Yixian Liu, Liwen Zhang, Wenjuan Han, Yue Zhang, Kewei Tu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lu et al., 2022] Ximing Lu, Sean Welleck, Peter West, Li- wei Jiang, Jungo Kasai, Daniel Khashabi, Ronan Le Bras, Lianhui Qin, Youngjae Yu, Rowan Zellers, Noah A. Smith, and Yejin Choi. NeuroLogic A*esque decoding: Con- strained text generation with lookahead heuristics. In Pro- ceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 780-799, Seattle, United States, July 2022. Association for Computational Linguistics.","title":"NeuroLogic A*esque decoding: Constrained text generation with lookahead heuristics","authors":"Ximing Lu, Sean Welleck, Peter West, Liwei Jiang, Jungo Kasai, Daniel Khashabi, Le Ronan, Lianhui Bras, Youngjae Qin, Rowan Yu, Noah A Zellers, Yejin Smith, Choi","year":"2022","venue":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Morin and Quimper, 2014] Michael Morin and Claude-Guy Quimper. The markov transition constraint. In Integra- tion of AI and OR Techniques in Constraint Programming: 11th International Conference, CPAIOR 2014, Cork, Ire- land, May 19-23, 2014. Proceedings 11, pages 405-421. Springer, 2014. [OpenAI, 2023] OpenAI. Gpt-4 technical report, 2023.","title":"The markov transition constraint","authors":"Michael Morin, Claude-Guy Quimper","year":"2014","venue":"Integration of AI and OR Techniques in Constraint Programming: 11th International Conference, CPAIOR 2014","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Pachet and Roy, 2011] Franc ¸ois Pachet and Pierre Roy. Markov constraints: Steerable generation of markov se- quences. Constraints, 16(2):148-172, apr 2011.","title":"Franc ¸ois Pachet and Pierre Roy. Markov constraints: Steerable generation of markov sequences","authors":"Roy Pachet","year":"2011","venue":"Constraints","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Pachet et al., 2011] Francois Pachet, Pierre Roy, and Gabriele Barbieri. Finite-length markov processes with constraints. In Proceedings of the Twenty-Second Inter- national Joint Conference on Artificial Intelligence, IJ- CAI'11, pages 635-642. AAAI Press, 2011.","title":"Finite-length markov processes with constraints","authors":"Francois Pachet, Pierre Roy, Gabriele Barbieri","year":"2011","venue":"Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, IJ-CAI'11","doi":null,"url":null,"internal_url":"/proceedings/2011/113"},{"position":13,"raw":"[Papadopoulos et al., 2014] Alexandre Papadopoulos, Pierre Roy, and Franc ¸ois Pachet. Avoiding plagiarism in markov sequence generation. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, AAAI'14, page 2731-2737. AAAI Press, 2014.","title":"Avoiding plagiarism in markov sequence generation","authors":"Alexandre Papadopoulos, Pierre Roy, Franc ¸ois, Pachet","year":"2014","venue":"Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, AAAI'14","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Papadopoulos et al., 2015] Alexandre Papadopoulos, Pierre Roy, Jean-Charles Régin, and Franc ¸ois Pachet. Generating all possible palindromes from ngram corpora. In Proceed- ings of the 24th International Conference on Artificial In- telligence, IJCAI'15, page 2489-2495. AAAI Press, 2015.","title":"Generating all possible palindromes from ngram corpora","authors":"Alexandre Papadopoulos, Pierre Roy, Jean-Charles Régin, Franc ¸ois, Pachet","year":"2015","venue":"Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI'15","doi":null,"url":null,"internal_url":"/proceedings/2015/353"},{"position":15,"raw":"[Perez and Régin, 2017] Guillaume Perez and Jean-Charles Régin. MDDs: Sampling and probability constraints. In Proceedings of the International Conference on Principles and Practice of Constraint Programming, page 226-242, 2017.","title":"MDDs: Sampling and probability constraints","authors":"Régin Perez, Guillaume Perez, Jean-Charles Régin","year":"2017","venue":"Proceedings of the International Conference on Principles and Practice of Constraint Programming","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Perez and Régin, 2017] Guillaume Perez and Jean-Charles Régin. Soft and cost MDD propagators. In Proceedings of the AAAI Conference on Artificial Intelligence, 2017.","title":"Soft and cost MDD propagators","authors":"Régin Perez, Guillaume Perez, Jean-Charles Régin","year":"2017","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pesant and Régin, 2005] Gilles Pesant and Jean-Charles Régin. Spread: A balancing constraint based on statistics. In Peter van Beek, editor, Principles and Practice of Con- straint Programming -CP 2005, pages 460-474, Berlin, Heidelberg, 2005. Springer Berlin Heidelberg.","title":"Spread: A balancing constraint based on statistics","authors":"Régin Pesant, Gilles Pesant, Jean-Charles Régin","year":"2005","venue":"Principles and Practice of Constraint Programming -CP 2005","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Post and Vilar, 2018] Matt Post and David Vilar. Fast lexi- cally constrained decoding with dynamic beam allocation for neural machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1314-1324, New Orleans, Louisiana, June 2018. Association for Com- putational Linguistics.","title":"Fast lexically constrained decoding with dynamic beam allocation for neural machine translation","authors":"Vilar Post, Matt Post, David Vilar","year":"2018","venue":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Roy and Pachet, 2013] Pierre Roy and Francois Pachet. En- forcing meter in finite-length markov sequences. Pro- ceedings of the AAAI Conference on Artificial Intelligence, 27(1):854-861, Jun. 2013.","title":"Enforcing meter in finite-length markov sequences","authors":"Roy, Pachet, Pierre Roy, Francois Pachet","year":"2013","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Schaus et al., 2007] Pierre Schaus, Yves Deville, Pierre Dupont, and Jean-Charles Régin. The deviation con- straint. In Pascal Van Hentenryck and Laurence Wolsey, editors, Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems, pages 260-274, Berlin, Heidelberg, 2007. Springer Berlin Heidelberg.","title":"The deviation constraint","authors":"Pierre Schaus, Yves Deville, Pierre Dupont, Jean-Charles Régin","year":"2007","venue":"Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shannon, 1951] C. E. Shannon. Prediction and entropy of printed english. The Bell System Technical Journal, 30(1):50-64, 1951.","title":"Prediction and entropy of printed english","authors":"C E Shannon","year":"1951","venue":"The Bell System Technical Journal","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Simoulin and Crabbé, 2021] Antoine Simoulin and Benoit Crabbé. Un modèle Transformer Génératif Pré-entrainé pour le franc ¸ais. In Pascal Denis, Natalia Grabar, Amel Fraisse, Rémi Cardon, Bernard Jacquemin, Eric Kergosien, and Antonio Balvet, editors, Traitement Au- tomatique des Langues Naturelles, pages 246-255, Lille, France, 2021. ATALA.","title":"Antoine Simoulin and Benoit Crabbé. Un modèle Transformer Génératif Pré-entrainé pour le franc ¸ais","authors":"Crabbé Simoulin","year":"2021","venue":"Traitement Automatique des Langues Naturelles","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Suresh et al., 2021] Ananda Theertha Suresh, Brian Roark, Michael Riley, and Vlad Schogol. Approximating prob- abilistic models as weighted finite automata. Computa- tional Linguistics, 47(2):221-254, 2021.","title":"Approximating probabilistic models as weighted finite automata","authors":"Ananda Theertha Suresh, Brian Roark, Michael Riley, Vlad Schogol","year":"2021","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Touvron et al., 2023] Hugo Touvron, Thibaut Lavril, Gau- tier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- othée Lacroix, Baptiste Rozière, Naman Goyal, Eric Ham- bro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023.","title":"Llama: Open and efficient foundation language models","authors":"Touvron","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wolf et al., 2020] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Fun- towicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of- the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing: System Demonstrations, Online, Octo- ber 2020. Association for Computational Linguistics.","title":"Transformers: State-ofthe-art natural language processing","authors":"Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Clara Patrick Von Platen, Yacine Ma, Julien Jernite, Canwen Plu, Teven Xu, Sylvain Le Scao, Mariama Gugger, Quentin Drame, Alexander M Lhoest, Rush","year":"2020","venue":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}