{"type":"ScholarlyArticle","title":"Pattern-Based Music Generation with Wasserstein Autoencoders and PRC Descriptions","authors":[{"name":"Valentijn Borghuis","slug":"valentijn-borghuis","given":"Valentijn","family":"Borghuis","affiliation":"Eindhoven University of Technology; MUSI-CO 3 DINFO , University of Florence"},{"name":"Luca Angioloni","slug":"luca-angioloni","given":"Luca","family":"Angioloni","affiliation":""},{"name":"Lorenzo Brusci","slug":"lorenzo-brusci","given":"Lorenzo","family":"Brusci","affiliation":"MUSI-CO 3 DINFO , University of Florence"},{"name":"Paolo Frasconi","slug":"paolo-frasconi","given":"Paolo","family":"Frasconi","affiliation":""}],"abstract":"We demonstrate a pattern-based MIDI music generation\r\nsystem with a generation strategy based on\r\nWasserstein autoencoders and a novel variant of pianoroll\r\ndescriptions of patterns which employs separate\r\nchannels for note velocities and note durations\r\nand can be fed into classic DCGAN-style convolutional\r\narchitectures. We trained the system on two\r\nnew datasets (in the acid-jazz and high-pop genres)\r\ncomposed by musicians in our team with music\r\ngeneration in mind. Our demonstration shows\r\nthat moving smoothly in the latent space allows us\r\nto generate meaningful sequences of four-bars patterns.","doi":"10.24963/ijcai.2020/751","url":"https://ijcai.org/proceedings/2020/751","pdf":"https://ijcai.org/proceedings/2020/0751.pdf","markdown":"https://ijcai.org/proceedings/2020/751.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/751","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Demos","track":"Demos","number":751,"pages":{"first":5225,"last":5227},"keywords":["Human-Computer Interactive Systems: general","Machine Learning: general"],"videos":[],"references":[{"position":1,"raw":"[Borghuis et al., 2018] Tijn Borghuis, Alessandro Tibo, Si- mone Conforti, Luca Canciello, Lorenzo Brusci, and Paolo Frasconi. Off the Beaten Track: Using Deep Learning to Interpolate Between Music Genres. arXiv:1804.09808, April 2018.","title":"Off the Beaten Track: Using Deep Learning to Interpolate Between Music Genres","authors":"Tijn Borghuis, Alessandro Tibo, Simone Conforti, Luca Canciello, Lorenzo Brusci, Paolo Frasconi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boulanger-Lewandowski et al., 2012] Nicolas Boulanger- Lewandowski, Yoshua Bengio, and Pascal Vincent. Mod- eling temporal dependencies in high-dimensional se- quences: Application to polyphonic music generation and transcription. In Proceedings of the 29th International Conference on Machine Learning, ICML 2012, Edin- burgh, Scotland, UK, June 26 -July 1, 2012, 2012.","title":"Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription","authors":"Boulanger-Lewandowski","year":"2012","venue":"Proceedings of the 29th International Conference on Machine Learning, ICML 2012","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Briot et al., 2019] Jean-Pierre Briot, Gaëtan Hadjeres, and Franc ¸ois Pachet. Deep Learning Techniques for Music Generation, Computational Synthesis and Creative Sys- tems. Springer Nature, 2019.","title":"Deep Learning Techniques for Music Generation","authors":"Jean-Pierre Briot, Gaëtan Hadjeres, Franc ¸ois, Pachet","year":"2019","venue":"Computational Synthesis and Creative Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dong and Yang, 2018] Hao-Wen Dong and Yi-Hsuan Yang. Convolutional generative adversarial networks with binary neurons for polyphonic music generation. In Proceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, Septem- ber 23-27, 2018, pages 190-196, 2018.","title":"Convolutional generative adversarial networks with binary neurons for polyphonic music generation","authors":"Wen Hao, Yi-Hsuan Dong, Yang","year":"2018","venue":"Proceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong et al., 2018] Hao-Wen Dong, Wen-Yi Hsiao, Li-Chia Yang, and Yi-Hsuan Yang. Musegan: Multi-track sequen- tial generative adversarial networks for symbolic music generation and accompaniment. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pages 34-41, 2018.","title":"Musegan: Multi-track sequential generative adversarial networks for symbolic music generation and accompaniment","authors":"Hao-Wen, Wen-Yi Dong, Li-Chia Hsiao, Yi-Hsuan Yang, Yang","year":"2018","venue":"Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dosovitskiy and Brox, 2016] Alexey Dosovitskiy and Thomas Brox. Generating images with perceptual sim- ilarity metrics based on deep networks. In Advances in Neural Information Processing Systems 29, pages 658-666, 2016.","title":"Generating images with perceptual similarity metrics based on deep networks","authors":"Brox Dosovitskiy, Alexey Dosovitskiy, Thomas Brox","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances in Neural Informa- tion Processing Systems 27, pages 2672-2680. 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2019] Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Ian Simon, Curtis Hawthorne, Noam Shazeer, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu, and Douglas Eck. Music trans- former: Generating music with long-term structure. In 7th International Conference on Learning Representations, 2019.","title":"Music transformer: Generating music with long-term structure","authors":"Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Ian Simon, Curtis Hawthorne, Noam Shazeer, Andrew M Dai, Matthew D Hoffman, Monica Dinculescu, Douglas Eck","year":"2019","venue":"7th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kaibel and Peinhardt, 2006] V. Kaibel and M. Peinhardt. On the bottleneck shortest path problem. Technical re- port, Otto-von-Guericke-Univ. Magdeburg, Magdeburg, Germany, 2006.","title":"On the bottleneck shortest path problem","authors":"V Kaibel, M Peinhardt","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kingma and Welling, 2014] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In 2nd Inter- national Conference on Learning Representations, 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"2nd International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Radford et al., 2016] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial net- works. In 4th International Conference on Learning Representations, 2016.","title":"Unsupervised representation learning with deep convolutional generative adversarial networks","authors":"Radford","year":"2016","venue":"4th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Roberts et al., 2018] Adam Roberts, Jesse H. Engel, Colin Raffel, Curtis Hawthorne, and Douglas Eck. A hierar- chical latent vector model for learning long-term struc- ture in music. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stock- holmsmässan, Stockholm, Sweden, July 10-15, 2018, vol- ume 80, pages 4361-4370. PMLR, 2018.","title":"A hierarchical latent vector model for learning long-term structure in music","authors":"Adam Roberts, Jesse H Engel, Colin Raffel, Curtis Hawthorne, Douglas Eck","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Tolstikhin et al., 2018] Ilya O. Tolstikhin, Olivier Bousquet, Sylvain Gelly, and Bernhard Schölkopf. Wasserstein auto- encoders. In 6th International Conference on Learning Representations, 2018.","title":"Wasserstein autoencoders","authors":"O Ilya, Olivier Tolstikhin, Sylvain Bousquet, Bernhard Gelly, Schölkopf","year":"2018","venue":"6th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[White, 2016] Tom White. Sampling generative net- works: Notes on a few effective techniques. CoRR, abs/1609.04468, 2016.","title":"Tom White. Sampling generative networks: Notes on a few effective techniques","authors":"White","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Yang et al., 2017] Li-Chia Yang, Szu-Yu Chou, and Yi- Hsuan Yang. Midinet: A convolutional generative adver- sarial network for symbolic-domain music generation. In Proceedings of the 18th International Society for Music Information Retrieval Conference, ISMIR 2017, Suzhou, China, October 23-27, 2017, pages 324-331, 2017.","title":"Midinet: A convolutional generative adversarial network for symbolic-domain music generation","authors":"Li-Chia Yang, Szu-Yu Chou, Yi-Hsuan Yang","year":"2017","venue":"Proceedings of the 18th International Society for Music Information Retrieval Conference, ISMIR 2017","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}