{"type":"ScholarlyArticle","title":"Temporal Pyramid Pooling Convolutional Neural Network for Cover Song Identification","authors":[{"name":"Zhesong Yu","slug":"zhesong-yu","given":"Zhesong","family":"Yu","affiliation":"Institute of Computer Science and Technology , Peking University"},{"name":"Xiaoshuo Xu","slug":"xiaoshuo-xu","given":"Xiaoshuo","family":"Xu","affiliation":"Institute of Computer Science and Technology , Peking University"},{"name":"Xiaoou Chen","slug":"xiaoou-chen","given":"Xiaoou","family":"Chen","affiliation":"Institute of Computer Science and Technology , Peking University"},{"name":"Deshun Yang","slug":"deshun-yang","given":"Deshun","family":"Yang","affiliation":"Institute of Computer Science and Technology , Peking University"}],"abstract":"Cover song identification is an important problem in the field of Music Information Retrieval. Most existing methods rely on hand-crafted features and sequence alignment methods, and further breakthrough is hard to achieve. In this paper, Convolutional Neural Networks (CNNs) are used for representation learning toward this task. We show that they could be naturally adapted to deal with key transposition in cover songs. Additionally, Temporal Pyramid Pooling is utilized to extract information on different scales and transform songs with different lengths into fixed-dimensional representations. Furthermore, a training scheme is designed to enhance the robustness of our model. Extensive experiments demonstrate that combined with these techniques, our approach is robust against musical variations existing in cover songs and outperforms state-of-the-art methods on several datasets with low time complexity.","doi":"10.24963/ijcai.2019/673","url":"https://ijcai.org/proceedings/2019/673","pdf":"https://ijcai.org/proceedings/2019/0673.pdf","markdown":"https://ijcai.org/proceedings/2019/673.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/673","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":673,"pages":{"first":4846,"last":4852},"keywords":["Machine Learning: Deep Learning","Multidisciplinary Topics and Applications: Information Retrieval","Multidisciplinary Topics and Applications: Art and Music"],"videos":[],"references":[{"position":1,"raw":"[Bello, 2007] Juan Pablo Bello. Audio-based cover song re- trieval using approximate chord sequences: Testing shifts, gaps, swaps and beats. In International Society for Music Information Retrieval Conference, pages 239-244, 2007.","title":"Audio-based cover song retrieval using approximate chord sequences: Testing shifts, gaps, swaps and beats","authors":"Pablo Juan, Bello","year":"2007","venue":"International Society for Music Information Retrieval Conference","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bertin-Mahieux and Ellis, 2012] Thierry Bertin-Mahieux and Daniel PW Ellis. Large-scale cover song recognition using the 2d fourier transform magnitude. In International 4 see some examples on SHS100K-TEST https://www.dropbox. com/s/308z225hy31d3xa/RankList.xlsx?dl=0 Society for Music Information Retrieval Conference, pages 241-246, 2012.","title":"Large-scale cover song recognition using the 2d fourier transform magnitude","authors":"Thierry Bertin, -Mahieux, Daniel Pw Ellis","year":"2012","venue":"International 4 see some examples on SHS100K-TEST","doi":null,"url":"https://www.dropbox.com/s/308z225hy31d3xa/RankList.xlsx","internal_url":null},{"position":3,"raw":"[Brown, 1991] Judith C Brown. Calculation of a constant q spectral transform. The Journal of the Acoustical Society of America, 89(1):425-434, 1991.","title":"Calculation of a constant q spectral transform","authors":"Judith C Brown","year":"1991","venue":"The Journal of the Acoustical Society of America","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chang et al., 2017] Sungkyun Chang, Juheon Lee, Sang Keun Choe, and Kyogu Lee. Audio cover song identification using convolutional neural network. In Workshop Machine Learning for Audio Signal Processing at NIPS, 2017.","title":"Audio cover song identification using convolutional neural network","authors":"Chang","year":"2017","venue":"Workshop Machine Learning for Audio Signal Processing at NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2018] Ning Chen, Wei Li, and Haidong Xiao. Fusing similarity functions for cover song identification. Multimedia Tools and Applications, pages 2629-2652, 2018.","title":"Fusing similarity functions for cover song identification","authors":"Ning Chen, Wei Li, Haidong Xiao","year":"2018","venue":"Multimedia Tools and Applications","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cheng et al., 2017] Yao Cheng, Xiaoou Chen, Deshun Yang, and Xiaoshuo Xu. Effective music feature ncp: En- hancing cover song recognition with music transcription. In Proceedings of the 40th International ACM SIGIR Con- ference on Research and Development in Information Re- trieval, pages 925-928. ACM, 2017.","title":"Effective music feature ncp: Enhancing cover song recognition with music transcription","authors":"Yao Cheng, Xiaoou Chen, Deshun Yang, Xiaoshuo Xu","year":"2017","venue":"Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ellis and Poliner, 2007] Daniel PW Ellis and Graham E Po- liner. Identifying cover songs with chroma features and dy- namic programming beat tracking. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1429-1432, 2007.","title":"Identifying cover songs with chroma features and dynamic programming beat tracking","authors":"Poliner ; Ellis, P W Daniel, Graham E Ellis, Poliner","year":"2007","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fujishima, 1999] Takuya Fujishima. Realtime chord recog- nition for musical sound: a system using common lisp mu- sic. In ICMC, pages 464-467, 1999.","title":"Realtime chord recognition for musical sound: a system using common lisp music","authors":"Takuya Fujishima","year":"1999","venue":"ICMC","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Grosche and Müller, 2012] Peter Grosche and Meinard Müller. Toward characteristic audio shingles for efficient cross-version music retrieval. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 473-476. IEEE, 2012.","title":"Toward characteristic audio shingles for efficient cross-version music retrieval","authors":"Müller Grosche, Peter Grosche, Meinard Müller","year":"2012","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Spatial pyramid pooling in deep con- volutional networks for visual recognition. IEEE trans- actions on pattern analysis and machine intelligence, 37(9):1904-1916, 2015.","title":"Spatial pyramid pooling in deep convolutional networks for visual recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2015","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Khadkevich and Omologo, 2013] Maksim Khadkevich and Maurizio Omologo. Large-scale cover song identification using chord profiles. In International Society for Music Information Retrieval Conference, pages 233-238, 2013.","title":"Maksim Khadkevich and Maurizio Omologo. Large-scale cover song identification using chord profiles","authors":"Omologo Khadkevich","year":"2013","venue":"International Society for Music Information Retrieval Conference","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Martin et al., 2012] Benjamin Martin, Daniel G Brown, Pierre Hanna, and Pascal Ferraro. Blast for audio se- quences alignment: a fast scalable cover identification. In International Society for Music Information Retrieval Conference, pages 529-534, 2012.","title":"Blast for audio sequences alignment: a fast scalable cover identification","authors":"Benjamin Martin, Pierre Daniel G Brown, Pascal Hanna, Ferraro","year":"2012","venue":"International Society for Music Information Retrieval Conference","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[McFee et al., 2015] Brian McFee, Colin Raffel, Dawen Liang, Daniel PW Ellis, Matt McVicar, Eric Battenberg, and Oriol Nieto. librosa: Audio and music signal analysis in python. In Proceedings of the 14th python in science conference, pages 18-25, 2015.","title":"librosa: Audio and music signal analysis in python","authors":"Brian Mcfee, Colin Raffel, Dawen Liang, P W Daniel, Matt Ellis, Eric Mcvicar, Oriol Battenberg, Nieto","year":"2015","venue":"Proceedings of the 14th python in science conference","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Osmalsky et al., 2015] Julien Osmalsky, Jean-Jacques Em- brechts, Peter Foster, and Simon Dixon. Combining fea- tures for cover song identification. In International Society for Music Information Retrieval Conference, pages 462- -468, 2015.","title":"Combining features for cover song identification","authors":"Julien Osmalsky, Jean-Jacques Embrechts, Peter Foster, Simon Dixon","year":"2015","venue":"International Society for Music Information Retrieval Conference","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Seetharaman and Rafii, 2017] Prem Seetharaman and Zafar Rafii. Cover song identification with 2d fourier transform sequences. In IEEE International Conference on Acous- tics, Speech and Signal Processing, pages 616-620, 2017.","title":"Cover song identification with 2d fourier transform sequences","authors":"Rafii Seetharaman, Prem Seetharaman, Zafar Rafii","year":"2017","venue":"IEEE International Conference on Acoustics, Speech and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Serrà et al., 2008] Joan Serrà, Emilia Gómez, Perfecto Her- rera, and Xavier Serrà. Chroma binary similarity and lo- cal alignment applied to cover song identification. IEEE Transactions on Audio, Speech, and Language Processing, 16(6):1138-1151, 2008.","title":"Chroma binary similarity and local alignment applied to cover song identification","authors":"Joan Serrà, Emilia Gómez, Perfecto Herrera, Xavier Serrà","year":"2008","venue":"IEEE Transactions on Audio, Speech, and Language Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Serrà et al., 2009] Joan Serrà, Xavier Serrà, and Ralph G Andrzejak. Cross recurrence quantification for cover song identification. New Journal of Physics, 11(9):093017, 2009.","title":"Cross recurrence quantification for cover song identification","authors":"Joan Serrà, Xavier Serrà, Ralph G Andrzejak","year":"2009","venue":"New Journal of Physics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Serrà et al., 2012] Joan Serrà, Holger Kantz, Xavier Serrà, and Ralph G Andrzejak. Predictability for music descrip- tor time series and its application to cover song detection. IEEE Transactions on Audio, Speech, and Language Pro- cessing, 20(2):514-525, 2012.","title":"Predictability for music descriptor time series and its application to cover song detection","authors":"Joan Serrà, Holger Kantz, Xavier Serrà, Ralph G Andrzejak","year":"2012","venue":"IEEE Transactions on Audio, Speech, and Language Processing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Silva et al., 2016] Diego F Silva, Chin-Chin M Yeh, Gus- tavo Enrique de Almeida Prado Alves Batista, Eamonn Keogh, et al. Simple: assessing music similarity using subsequences joins. In International Society for Music In- formation Retrieval Conference, pages 23-29, 2016.","title":"Simple: assessing music similarity using subsequences joins","authors":"Silva","year":"2016","venue":"International Society for Music Information Retrieval Conference","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Silva et al., 2018] Diego Furtado Silva, Felipe Vieira Fal- cao, and Nazareno Andrade. Summarizing and comparing music data and its application on cover song identification. In International Society for Music Information Retrieval Conference, pages 732-739, 2018.","title":"Summarizing and comparing music data and its application on cover song identification","authors":"Silva","year":"2018","venue":"International Society for Music Information Retrieval Conference","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Confer- ence on Learning Representations, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tzanetakis et al., 2003] George Tzanetakis, Andrey Er- molinskyi, and Perry Cook. Pitch histograms in audio and symbolic music information retrieval. Journal of New Mu- sic Research, 32(2):143-152, 2003.","title":"Pitch histograms in audio and symbolic music information retrieval","authors":"George Tzanetakis, Andrey Ermolinskyi, Perry Cook","year":"2003","venue":"Journal of New Music Research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017] Peng Wang, Yuanzhouhan Cao, Chun- hua Shen, Lingqiao Liu, and Heng Tao Shen. Temporal pyramid pooling-based convolutional neural network for action recognition. IEEE Transactions on Circuits and Systems for Video Technology, 27(12):2613-2622, 2017.","title":"Temporal pyramid pooling-based convolutional neural network for action recognition","authors":"Peng Wang, Yuanzhouhan Cao, Chunhua Shen, Lingqiao Liu, Heng Tao Shen","year":"2017","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xu et al., 2018] Xiaoshuo Xu, Xiaoou Chen, and Deshun Yang. Key-invariant convolutional neural network toward efficient cover song identification. In 2018 IEEE Inter- national Conference on Multimedia and Expo, pages 1-6. IEEE, 2018.","title":"Key-invariant convolutional neural network toward efficient cover song identification","authors":"Xu","year":"2018","venue":"2018 IEEE International Conference on Multimedia and Expo","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zölzer, 2011] Udo Zölzer. DAFX: digital audio effects. John Wiley & Sons, 2011.","title":"DAFX: digital audio effects","authors":"Udo Zölzer, Zölzer","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}