{"type":"ScholarlyArticle","title":"Expressing Musical Ideas with Constraint Programming Using a Model of Tonal Harmony","authors":[{"name":"Damien Sprockeels","slug":"damien-sprockeels","given":"Damien","family":"Sprockeels","affiliation":"UCLouvain , Louvain-la-Neuve , Belgium"},{"name":"Peter Van Roy","slug":"peter-van-roy","given":"Peter","family":"Van Roy","affiliation":"UCLouvain , Louvain-la-Neuve , Belgium"}],"abstract":"The realm of music composition with artificial intelligence stands as a pertinent and evolving field, attracting increasing interest and exploration in contemporary research and practice. This paper presents a constraint-programming based approach to generating four-voice diatonic chord progressions according to established rules of tonal harmony. It uses the strength of constraint programming as a formal logic to rigorously model musical rules and to offer complete control over the set of rules that are enforced. This allows composers to iteratively interact with the model, adding and removing constraints, allowing them to shape the solutions according to their preferences.  We define a constraint model of basic tonal harmony, called Diatony. We show that our implementation using the Gecode solver finds optimal solutions in reasonable time and we show how it can be used by a composer to aid in their composition process.","doi":"10.24963/ijcai.2024/858","url":"https://ijcai.org/proceedings/2024/858","pdf":"https://ijcai.org/proceedings/2024/0858.pdf","markdown":"https://ijcai.org/proceedings/2024/858.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/858","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"AI, Arts & Creativity","track":"AI, Arts & Creativity","number":858,"pages":{"first":7753,"last":7761},"keywords":["Methods and resources: Techniques for modeling and simulation of creativity","Theory and philosophy of arts and creativity in AI systems: Computational paradigms, architectures and models for creativity","Application domains: Problem Solving","Application domains: Science, math and programming"],"videos":[],"references":[{"position":1,"raw":"[Ableton AG, 1999] Ableton AG. Ableton Live, 1999.","title":"Ableton AG. Ableton Live","authors":"A G Ableton","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Anders and Miranda, 2011] Torsten Anders and Eduardo R Miranda. Constraint programming systems for modeling music theories and composition. ACM Computing Surveys (CSUR), 43(4):1-38, 2011.","title":"Constraint programming systems for modeling music theories and composition","authors":"Anders, Miranda ;, Torsten Anders, Eduardo R Miranda","year":"2011","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"Sibelius, 1993.","title":null,"authors":"Sibelius","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Briot and Pachet, 2017] Jean-Pierre Briot and Franc ¸ois Pa- chet. Music generation by deep learning-challenges and directions. arXiv preprint arXiv:1712.04371, 2017.","title":"Music generation by deep learning-challenges and directions","authors":"Pachet Briot","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Briot et al., 2020] Jean-Pierre Briot, Gaëtan Hadjeres, and Franc ¸ois-David Pachet. Deep learning techniques for mu- sic generation, volume 1. Springer, 2020.","title":"Deep learning techniques for music generation","authors":"Jean-Pierre Briot, Gaëtan Hadjeres, Franc ¸ois-David Pachet","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cappart et al., 2021] Quentin Cappart, Thierry Moisan, Louis-Martin Rousseau, Isabeau Prémont-Schwarz, and Andre A Cire. Combining reinforcement learning and con- straint programming for combinatorial optimization. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 35, pages 3677-3687, 2021.","title":"Combining reinforcement learning and constraint programming for combinatorial optimization","authors":"Quentin Cappart, Louis-Martin Thierry Moisan, Isabeau Rousseau, Andre A Prémont-Schwarz, Cire","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Casini et al., 2018] Luca Casini, Gustavo Marfia, and Marco Roccetti. Some reflections on the potential and limitations of deep learning for automated music genera- tion. In 2018 IEEE 29th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), pages 27-31, 2018.","title":"Some reflections on the potential and limitations of deep learning for automated music generation","authors":"Luca Casini, Gustavo Marfia, Marco Roccetti","year":"2018","venue":"IEEE 29th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Delecluse et al., 2022] Augustin Delecluse, Pierre Schaus, and Pascal Van Hentenryck. Sequence variables for rout- ing problems. In 28th International Conference on Princi- ples and Practice of Constraint Programming (CP 2022). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2022.","title":"Sequence variables for routing problems","authors":"Augustin Delecluse, Pierre Schaus, Pascal Van Hentenryck","year":"2022","venue":"28th International Conference on Principles and Practice of Constraint Programming","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dong et al., 2018] Hao-Wen Dong, Wen-Yi Hsiao, Li-Chia Yang, and Yi-Hsuan Yang. Musegan: Multi-track sequen- tial generative adversarial networks for symbolic music generation and accompaniment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.","title":"Musegan: Multi-track sequential generative adversarial networks for symbolic music generation and accompaniment","authors":"Hao-Wen, Wen-Yi Dong, Li-Chia Hsiao, Yi-Hsuan Yang, Yang","year":"2018","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Duha, 2016] Isabelle Duha. L'harmonie en liberté: de la mémoire à l'improvisation. Gérard Billaudot, Armiane Imp., 2016.","title":"L'harmonie en liberté: de la mémoire à l'improvisation","authors":"Duha","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gauldin, 2004] Robert Gauldin. Harmonic Practice in Tonal Music. Second Edition. W. W. Norton and Com- pany, Inc, 2004. [Gecode Team, 2019] Gecode Team. Gecode: Generic Con- straint Development Environment, 2019. Last accessed on 19/02/2023.","title":"Harmonic Practice in Tonal Music","authors":"Robert Gauldin","year":"2004","venue":"Generic Constraint Development Environment","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hadjeres and Nielsen, 2020] Gaëtan Hadjeres and Frank Nielsen. Anticipation-rnn: Enforcing unary constraints in sequence generation, with application to interactive music generation. Neural Computing and Applications, 32(4):995-1005, 2020.","title":"Anticipation-rnn: Enforcing unary constraints in sequence generation, with application to interactive music generation","authors":"Nielsen Hadjeres, Gaëtan Hadjeres, Frank Nielsen","year":"2020","venue":"Neural Computing and Applications","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ignatiev et al., 2022] Alexey Ignatiev, Yacine Izza, Peter J Stuckey, and Joao Marques-Silva. Using maxsat for ef- ficient explanations of tree ensembles. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 3776-3785, 2022.","title":"Using maxsat for efficient explanations of tree ensembles","authors":"Alexey Ignatiev, Yacine Izza, Peter J Stuckey, Joao Marques-Silva","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kizilay et al., 2020] Damla Kizilay, Pascal Van Henten- ryck, and Deniz T Eliiyi. Constraint programming mod- els for integrated container terminal operations. European Journal of Operational Research, 286(3):945-962, 2020.","title":"Constraint programming models for integrated container terminal operations","authors":"Damla Kizilay, Pascal Van Hentenryck, Deniz T Eliiyi","year":"2020","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kotary et al., 2021] James Kotary, Ferdinando Fioretto, Pascal Van Hentenryck, and Bryan Wilder. End-to- end constrained optimization learning: A survey. arXiv preprint arXiv:2103.16378, 2021.","title":"End-toend constrained optimization learning: A survey","authors":"James Kotary, Ferdinando Fioretto, Pascal Van Hentenryck, Bryan Wilder","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lam et al., 2022] Edward Lam, Guy Desaulniers, and Pe- ter J Stuckey. Branch-and-cut-and-price for the electric vehicle routing problem with time windows, piecewise- linear recharging and capacitated recharging stations. Computers & Operations Research, 145:105870, 2022.","title":"Branch-and-cut-and-price for the electric vehicle routing problem with time windows, piecewiselinear recharging and capacitated recharging stations","authors":"Edward Lam, Guy Desaulniers, Peter J Stuckey","year":"2022","venue":"Computers & Operations Research","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mao et al., 2018] Huanru Henry Mao, Taylor Shin, and Garrison Cottrell. Deepj: Style-specific music generation. In 2018 IEEE 12th International Conference on Semantic Computing (ICSC), pages 377-382. IEEE, 2018.","title":"Deepj: Style-specific music generation","authors":"Mao","year":"2018","venue":"IEEE 12th International Conference on Semantic Computing (ICSC)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Meehan, 1979] James R Meehan. An artificial intelligence approach to tonal music theory. In Proceedings of the 1979 annual conference, pages 116-120, 1979.","title":"An artificial intelligence approach to tonal music theory","authors":"James R Meehan","year":"1979","venue":"Proceedings of the 1979 annual conference","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[MuseScore BVBA, 2002] MuseScore BVBA. MuseScore, 2002.","title":null,"authors":"Bvba Musescore, Musescore","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nam et al., 2018] Juhan Nam, Keunwoo Choi, Jongpil Lee, Szu-Yu Chou, and Yi-Hsuan Yang. Deep learning for audio-based music classification and tagging: Teaching computers to distinguish Rock from Bach. IEEE signal processing magazine, 36(1):41-51, 2018.","title":"Deep learning for audio-based music classification and tagging: Teaching computers to distinguish Rock from Bach","authors":"Juhan Nam, Keunwoo Choi, Jongpil Lee, Szu-Yu Chou, Yi-Hsuan Yang","year":"2018","venue":"IEEE signal processing magazine","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nethercote et al., 2007] Nicholas Nethercote, Peter J Stuckey, Ralph Becket, Sebastian Brand, Gregory J Duck, and Guido Tack. Minizinc: Towards a standard CP modelling language. In International Conference on Principles and Practice of Constraint Programming, pages 529-543. Springer, 2007.","title":"Minizinc: Towards a standard CP modelling language","authors":"Nicholas Nethercote, J Peter, Ralph Stuckey, Sebastian Becket, Gregory J Brand, Guido Duck, Tack","year":"2007","venue":"International Conference on Principles and Practice of Constraint Programming","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pachet and Roy, 2001] Franc ¸ois Pachet and Pierre Roy. Mu- sical harmonization with constraints: A survey. Con- straints, 6:7-19, 2001.","title":"Franc ¸ois Pachet and Pierre Roy. Musical harmonization with constraints: A survey","authors":"Roy Pachet","year":"2001","venue":"Constraints","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pachet and Roy, 2014] Franc ¸ois Pachet and Pierre Roy. Im- itative leadsheet generation with user constraints. In ECAI 2014, pages 1077-1078. IOS Press, 2014.","title":"Franc ¸ois Pachet and Pierre Roy. Imitative leadsheet generation with user constraints","authors":"Roy Pachet","year":"2014","venue":"ECAI 2014","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Purwins et al., 2019] Hendrik Purwins, Bo Li, Tuomas Vir- tanen, Jan Schlüter, Shuo-Yiin Chang, and Tara Sainath. Deep learning for audio signal processing. IEEE Journal of Selected Topics in Signal Processing, 13(2):206-219, 2019.","title":"Deep learning for audio signal processing","authors":"Hendrik Purwins, Bo Li, Tuomas Virtanen, Jan Schlüter, Shuo-Yiin, Tara Chang, Sainath","year":"2019","venue":"IEEE Journal of Selected Topics in Signal Processing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sandred, 2021] Örjan Sandred. Constraint-solving systems in music creation. Handbook of Artificial Intelligence for Music: Foundations, Advanced Approaches, and Develop- ments for Creativity, pages 327-344, 2021.","title":"Örjan Sandred. Constraint-solving systems in music creation","authors":"Sandred","year":"2021","venue":"Handbook of Artificial Intelligence for Music: Foundations, Advanced Approaches, and Developments for Creativity","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Shukla and Banka, 2018] Shipra Shukla and Haider Banka. An automatic chord progression generator based on rein- forcement learning. In 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pages 55-59. IEEE, 2018.","title":"An automatic chord progression generator based on reinforcement learning","authors":"Shipra Shukla, Haider Banka","year":"2018","venue":"International Conference on Advances in Computing, Communications and Informatics (ICACCI)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Smoliar, 1972] Stephen W Smoliar. Music theory-a pro- gramming linguistic approach. In Proceedings of the ACM annual conference-Volume 2, pages 1001-1014, 1972.","title":"Music theory-a programming linguistic approach","authors":"W Stephen, Smoliar","year":"1972","venue":"Proceedings of the ACM annual conference","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sprockeels et al., 2023] Damien Sprockeels, Thibault Waf- flard, Peter Van Roy, and Karim Haddad. A constraint formalization of Fux's counterpoint. Actes des Journées d'Informatique Musicale 2023, pages 68-77, 2023.","title":"A constraint formalization of Fux's counterpoint","authors":"Damien Sprockeels, Thibault Wafflard, Peter Van Roy, Karim Haddad","year":"2023","venue":"Actes des Journées d'Informatique Musicale","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Tagg, 1982] Philip Tagg. Analysing popular music: theory, method and practice. Popular music, 2:37-67, 1982.","title":"Analysing popular music: theory, method and practice","authors":"Philip Tagg, Tagg","year":"1982","venue":"Popular music","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Truchet and Codognet, 2004] Charlotte Truchet and Philippe Codognet. Musical constraint satisfaction problems solved with adaptive search. Soft Computing, 8(9):633-640, 2004.","title":"Charlotte Truchet and Philippe Codognet. Musical constraint satisfaction problems solved with adaptive search","authors":"Codognet Truchet","year":"2004","venue":"Soft Computing","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Tsouros et al., 2023] Dimos Tsouros, Senne Berden, and Tias Guns. Learning to learn in interactive constraint ac- quisition. arXiv preprint arXiv:2312.10795, 2023.","title":"Learning to learn in interactive constraint acquisition","authors":"Dimos Tsouros, Senne Berden, Tias Guns","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Van Hentenryck, 1989] Pascal Van Hentenryck. Constraint satisfaction in logic programming. MIT press, 1989.","title":"Constraint satisfaction in logic programming","authors":"Pascal Van Hentenryck","year":"1989","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Verhaeghe et al., 2020] Hélene Verhaeghe, Siegfried Ni- jssen, Gilles Pesant, Claude-Guy Quimper, and Pierre Schaus. Learning optimal decision trees using constraint programming. Constraints, 25:226-250, 2020.","title":"Learning optimal decision trees using constraint programming","authors":"Verhaeghe","year":"2020","venue":"Constraints","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wu et al., 2021] Yusong Wu, Ethan Manilow, Yi Deng, Rigel Swavely, Kyle Kastner, Tim Cooijmans, Aaron Courville, Cheng-Zhi Anna Huang, and Jesse Engel. Midi- ddsp: Detailed control of musical performance via hierar- chical modeling. arXiv preprint arXiv:2112.09312, 2021.","title":"Mididdsp: Detailed control of musical performance via hierarchical modeling","authors":"Wu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang, 2021] Jingwen Zhang. Music feature extraction and classification algorithm based on deep learning. Scientific Programming, 2021:1-9, 2021.","title":"Music feature extraction and classification algorithm based on deep learning","authors":"Jingwen Zhang","year":"2021","venue":"Scientific Programming","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}