{"type":"ScholarlyArticle","title":"Tradformer: A Transformer Model of Traditional Music Transcriptions","authors":[{"name":"Luca Casini","slug":"luca-casini","given":"Luca","family":"Casini","affiliation":"University of Bologna , Department of Computer Science and Engineering . Bologna , Italy"},{"name":"Bob L. T. Sturm","slug":"bob-l-t-sturm","given":"Bob L. T.","family":"Sturm","affiliation":"Kungliga Tekniska Högskola , Stockholm , Sweden"}],"abstract":"We explore the transformer neural network architecture for modeling music, \r\nspecifically Irish and Swedish traditional dance music.\r\nGiven the repetitive structures of these kinds of music, the transformer should be as successful with fewer parameters and complexity as the hitherto most successful model, a vanilla long short-term memory network.\r\nWe find that achieving good performance with the transformer is not straightforward,\r\nand careful consideration is needed for the sampling strategy, \r\nevaluating intermediate outputs in relation to engineering choices, \r\nand finally analyzing what the model learns.\r\nWe discuss these points with several illustrations, \r\nproviding reusable insights for engineering other music generation systems.\r\nWe also report the high performance of our final transformer model \r\nin a competition of music generation systems\r\nfocused on a type of Swedish dance.","doi":"10.24963/ijcai.2022/681","url":"https://ijcai.org/proceedings/2022/681","pdf":"https://ijcai.org/proceedings/2022/0681.pdf","markdown":"https://ijcai.org/proceedings/2022/681.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/681","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Research Papers","track":"AI and Arts","number":681,"pages":{"first":4915,"last":4920},"keywords":["Theory and philosophy of arts and creativity in AI systems: Evaluation of artistic or creative outputs, or of systems","Application domains: Music","Methods and resources: Machine learning, deep learning, neural models, reinforcement learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987164","length_sec":91,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985307","length_sec":559,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Branwen, 2019] Gwern Branwen. Gpt-2 folk music. https://www.gwern.net/GPT-2-music, 2019.","title":"Gwern Branwen. Gpt-2 folk music","authors":"Branwen","year":"2019","venue":null,"doi":null,"url":"https://www.gwern.net/GPT-2-music","internal_url":null},{"position":2,"raw":"[Casini et al., 2018] Luca Casini, Gustavo Marfia, and Marco Roccetti. Some reflections on the potential and limitations of deep learning for automated music genera- tion. In 2018 IEEE 29th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), pages 27-31, 2018.","title":"Some reflections on the potential and limitations of deep learning for automated music generation","authors":"Luca Casini, Gustavo Marfia, Marco Roccetti","year":"2018","venue":"IEEE 29th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Devlin et al., 2018] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04805, 2018.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hallström et al., 2019] Eric Hallström, Simon Mossmyr, Bob Sturm, Victor Vegeborn, and Jonas Wedin. From Jigs and Reels to Schottisar och Polskor: Generating Scandinavian-like Folk Music with Deep Recurrent Net- works. In The 16th Sound & Music Computing Confer- ence, Malaga, Spain, 28-31 May 2019, 2019.","title":"From Jigs and Reels to Schottisar och Polskor: Generating Scandinavian-like Folk Music with Deep Recurrent Networks","authors":"Eric Hallström, Simon Mossmyr, Bob Sturm, Victor Vegeborn, Jonas Wedin","year":"2019","venue":"The 16th Sound & Music Computing Conference","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Holtzman et al., 2019] Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In International Conference on Learning Representations, 2019.","title":"The curious case of neural text degeneration","authors":"Holtzman","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2018] Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Ian Simon, Curtis Hawthorne, Noam Shazeer, Andrew M Dai, Matthew D Hoffman, Monica Dinculescu, and Douglas Eck. Music trans- former: Generating music with long-term structure. In International Conference on Learning Representations, 2018.","title":"Music transformer: Generating music with long-term structure","authors":"Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Ian Simon, Curtis Hawthorne, Noam Shazeer, Andrew M Dai, Matthew D Hoffman, Monica Dinculescu, Douglas Eck","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Payne, 2019] Christine Payne. Musenet. http://openai.com/blog/musenet, 2019.","title":"Musenet","authors":"Christine Payne","year":"2019","venue":null,"doi":null,"url":"http://openai.com/blog/musenet","internal_url":null},{"position":9,"raw":"[Pearce et al., 2002] M. Pearce, D. Meredith, and G. Wig- gins. Motivations and methodologies for automation of the compositional process. Musicae Scientiae, 6(2):119-147, 2002.","title":"Motivations and methodologies for automation of the compositional process","authors":"M Pearce, D Meredith, G Wiggins","year":"2002","venue":"Musicae Scientiae","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Radford et al., 2018] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving lan- guage understanding by generative pre-training. https://openai.com/blog/language-unsupervised/, 2018.","title":"Improving language understanding by generative pre-training","authors":"Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever","year":null,"venue":null,"doi":null,"url":"https://openai.com/blog/language-unsupervised/,2018","internal_url":null},{"position":11,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":"OpenAI blog","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Roccetti et al., 2019] Marco Roccetti, Giovanni Delnevo, Luca Casini, Nicolò Zagni, and Giuseppe Cappiello. A paradox in ml design: less data for a smarter water me- tering cognification experience. In proceedings of the 5th EAI international conference on smart objects and Tech- nologies for Social Good, pages 201-206, 2019.","title":"A paradox in ml design: less data for a smarter water metering cognification experience","authors":"Marco Roccetti, Giovanni Delnevo, Luca Casini, Nicolò Zagni, Giuseppe Cappiello","year":"2019","venue":"proceedings of the 5th EAI international conference on smart objects and Technologies for Social Good","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Shaham and Levy, 2021] Uri Shaham and Omer Levy. What do you get when you cross beam search with nucleus sampling? arXiv preprint arXiv:2107.09729, 2021.","title":"What do you get when you cross beam search with nucleus sampling","authors":"Levy ; Shaham, Uri Shaham, Omer Levy","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sturm and Ben-Tal, 2017] Bob L Sturm and Oded Ben-Tal. Taking the models back to music practice: Evaluating generative transcription models built using deep learning. Journal of Creative Music Systems, 2:32-60, 2017.","title":"Taking the models back to music practice: Evaluating generative transcription models built using deep learning","authors":"Ben-Tal ; ] Sturm, L Bob, Oded Sturm, Ben-Tal","year":"2017","venue":"Journal of Creative Music Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sturm and Maruri-Aguilar, 2021] B. L. T. Sturm and H. Maruri-Aguilar. The Ai Music Generation Challenge 2020: Double jigs in the style of O'Neill's \"1001\". Journal of Creative Music Systems, 2021.","title":"The Ai Music Generation Challenge 2020: Double jigs in the style of O'Neill's \"1001","authors":"B L T Sturm, H Maruri-Aguilar","year":"2021","venue":"Journal of Creative Music Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sturm et al., 2016] B. L. Sturm, J. F. Santos, O. Ben-Tal, and I. Korshunova. Music transcription modelling and composition using deep learning. In Proc. Conf. Computer Simulation of Musical Creativity, Huddersfield, UK, 2016.","title":"Music transcription modelling and composition using deep learning","authors":"B L Sturm, J F Santos, O Ben-Tal, I Korshunova","year":"2016","venue":"Proc. Conf. Computer Simulation of Musical Creativity","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sturm, 2018] B. L. Sturm. How stuff works: LSTM model of folk music transcriptions. In Proc. Joint Workshop on Machine Learning for Music, ICML, 2018.","title":"How stuff works: LSTM model of folk music transcriptions","authors":"B L Sturm","year":"2018","venue":"Proc. Joint Workshop on Machine Learning for Music","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wiegreffe and Pinter, 2019] Sarah Wiegreffe and Yuval Pin- ter. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Lan- guage Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP-IJCNLP), pages 11-20, 2019.","title":"Attention is not not explanation","authors":"Sarah Wiegreffe, Yuval Pinter","year":"2019","venue":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}