{"type":"ScholarlyArticle","title":"Weakly Supervised Audio Source Separation via Spectrum Energy Preserved Wasserstein Learning","authors":[{"name":"Ning Zhang","slug":"ning-zhang","given":"Ning","family":"Zhang","affiliation":""},{"name":"Junchi Yan","slug":"junchi-yan","given":"Junchi","family":"Yan","affiliation":"Shanghai Jiao Tong University , Shanghai , P.R. China"},{"name":"Yuchen Zhou","slug":"yuchen-zhou","given":"Yuchen","family":"Zhou","affiliation":"IBM Research -China , Beijing , P.R. China"}],"abstract":"Separating audio mixtures into individual instrument tracks has been a standing challenge. We introduce a novel weakly supervised audio source separation approach based on deep adversarial learning. Specifically, our loss function adopts the Wasserstein distance which directly measures the distribution distance between the separated sources and the real sources for each individual source. Moreover, a global regularization term is added to fulfill the spectrum energy preservation property regardless separation. Unlike state-of-the-art weakly supervised models which often involve deliberately devised constraints or careful model selection, our approach need little prior model specification on the data, and can be straightforwardly learned in an end-to-end fashion. We show that the proposed method performs competitively on public benchmark against state-of-the-art weakly supervised methods.","doi":"10.24963/ijcai.2018/636","url":"https://ijcai.org/proceedings/2018/636","pdf":"https://ijcai.org/proceedings/2018/0636.pdf","markdown":"https://ijcai.org/proceedings/2018/636.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/636","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":636,"pages":{"first":4574,"last":4580},"keywords":["Machine Learning Applications: Applications of Unsupervised Learning","Natural Language Processing: Speech"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.","title":"Wasserstein gan","authors":"Martin Arjovsky, Soumith Chintala, Léon Bottou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cardoso, 1998] J Cardoso. Multidimensional independent compo- nent analysis. In ICASSP, pages 1941-1944, 1998.","title":"Multidimensional independent component analysis","authors":"Cardoso","year":"1998","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chandna et al., 2017] Pritish Chandna, Marius Miron, Jordi Janer, and Emilia Gómez. Monoaural audio source separation using deep convolutional neural networks. In International Conference on Latent Variable Analysis and Signal Separation, pages 258-266. Springer, 2017.","title":"Monoaural audio source separation using deep convolutional neural networks","authors":"Pritish Chandna, Marius Miron, Jordi Janer, Emilia Gómez","year":"2017","venue":"International Conference on Latent Variable Analysis and Signal Separation","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Comon and Jutten, 2010] Pierre Comon and C. Jutten. Handbook of Blind Source Separation: Independent Component Analysis and Separation. 2010.","title":"Handbook of Blind Source Separation: Independent Component Analysis and Separation","authors":"Pierre Comon, C Jutten","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fan et al., 2017] Zhe-Cheng Fan, Yen-Lin Lai, and Jyh-Shing R. Jang. Svsgan: Singing voice separation via generative adversarial network. arXiv preprint arXiv:1710.11428, 2017.","title":"Svsgan: Singing voice separation via generative adversarial network","authors":"Zhe-Cheng Fan, Yen-Lin Lai, R Jyh-Shing, Jang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fitzgerald et al., 2011] Derry Fitzgerald, Rajesh Jaiswal, Eugene Coyle, and Scott Rickard. Shifted nmf using an efficient constant- q transform for monaural sound source separation. Dublin Institute of Technology, 2011.","title":"Shifted nmf using an efficient constantq transform for monaural sound source separation","authors":"Derry Fitzgerald, Rajesh Jaiswal, Eugene Coyle, Scott Rickard","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goto et al., 2003] Masataka Goto, Hiroki Hashiguchi, Takuichi Nishimura, and Ryuichi Oka. Rwc music database: Music genre database and musical instrument sound database. Ipsj Sig Notes, 3:19-26, 2003.","title":"Rwc music database: Music genre database and musical instrument sound database","authors":"Masataka Goto, Hiroki Hashiguchi, Takuichi Nishimura, Ryuichi Oka","year":"2003","venue":"Ipsj Sig Notes","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gulrajani et al., 2017] Ishaan Gulrajani, Faruk Ahmed, Martin Ar- jovsky, Vincent Dumoulin, and Aaron Courville. Improved train- ing of wasserstein gans. arXiv preprint arXiv:1704.00028, 2017.","title":"Improved training of wasserstein gans","authors":"Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Aaron Vincent Dumoulin, Courville","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2015] Po-Sen Huang, Minje Kim, Mark Hasegawa- Johnson, and Paris Smaragdis. Joint optimization of masks and deep recurrent neural networks for monaural source separation. IEEE/ACM Transactions on Audio, Speech and Language Pro- cessing, 23(12):2136-2147, 2015.","title":"Joint optimization of masks and deep recurrent neural networks for monaural source separation","authors":"Huang","year":"2015","venue":"IEEE/ACM Transactions on Audio, Speech and Language Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Computer Science, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liutkus et al., 2011] Antoine Liutkus, Roland Badeau, and Gel Richard. Gaussian processes for underdetermined source separa- tion. IEEE Transactions on Signal Processing, 59(7):3155-3167, 2011.","title":"Gaussian processes for underdetermined source separation","authors":"Liutkus","year":"2011","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liutkus et al., 2014] Antoine Liutkus, Derry Fitzgerald, Zafar Rafii, Bryan Pardo, and Laurent Daudet. Kernel additive models for source separation. IEEE Transactions on Signal Processing, 62(16):4298-4310, 2014.","title":"Kernel additive models for source separation","authors":"Antoine Liutkus, Derry Fitzgerald, Zafar Rafii, Bryan Pardo, Laurent Daudet","year":"2014","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liutkus et al., 2015] Antoine Liutkus, Derry Fitzgerald, and Zafar Rafii. Scalable audio separation with light kernel additive mod- elling. In ICASSP, pages 76-80, 2015.","title":"Scalable audio separation with light kernel additive modelling","authors":"Liutkus","year":"2015","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liutkus et al., 2017] Antoine Liutkus, Fabian-Robert Stöter, Zafar Rafii, Daichi Kitamura, Bertrand Rivet, Nobutaka Ito, Nobutaka Ono, and Julie Fontecave. The 2016 signal separation evalua- tion campaign. In International Conference on Latent Variable Analysis and Signal Separation, pages 323-332. Springer, 2017.","title":"The 2016 signal separation evaluation campaign","authors":"Antoine Liutkus, Fabian-Robert Stöter, Zafar Rafii, Daichi Kitamura, Bertrand Rivet, Nobutaka Ito, Nobutaka Ono, Julie Fontecave","year":"2017","venue":"International Conference on Latent Variable Analysis and Signal Separation","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nugraha et al., 2016] Aditya Arie Nugraha, Antoine Liutkus, and Emmanuel Vincent. Multichannel audio source separation with deep neural networks. IEEE Transactions on Audio, Speech, and Language Processing, 24(9):1652-1664, 2016.","title":"Multichannel audio source separation with deep neural networks","authors":"Arie Aditya, Antoine Nugraha, Emmanuel Liutkus, Vincent","year":"2016","venue":"IEEE Transactions on Audio, Speech, and Language Processing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ozerov and Fevotte, 2010] Alexey Ozerov and Cric Fevotte. Multi- channel nonnegative matrix factorization in convolutive mixtures for audio source separation. IEEE Transactions on Audio Speech and Language Processing, 18(3):550-563, 2010.","title":"Alexey Ozerov and Cric Fevotte. Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation","authors":"Fevotte Ozerov","year":"2010","venue":"IEEE Transactions on Audio Speech and Language Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ozerov et al., 2012] Alexey Ozerov, Emmanuel Vincent, and Frédéric Bimbot. A general flexible framework for the handling of prior information in audio source separation. IEEE Transactions on Audio, Speech, and Language Processing, 20(4):1118-1133, 2012.","title":"A general flexible framework for the handling of prior information in audio source separation","authors":"Alexey Ozerov, Emmanuel Vincent, Frédéric Bimbot","year":"2012","venue":"IEEE Transactions on Audio, Speech, and Language Processing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Podder et al., 2014] Prajoy Podder, Tanvir Zaman Khan, Mam- dudul Haque Khan, and M Muktadir Rahman. Comparative performance analysis of hamming, hanning and blackman win- dow. International Journal of Computer Applications, 96(18):1-7, 2014.","title":"Comparative performance analysis of hamming, hanning and blackman window","authors":"Podder","year":"2014","venue":"International Journal of Computer Applications","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Radford et al., 2015] Alec Radford, Luke Metz, and Soumith Chin- tala. Unsupervised representation learning with deep convolu- tional generative adversarial networks. Computer Science, 2015.","title":"Unsupervised representation learning with deep convolutional generative adversarial networks","authors":"Radford","year":"2015","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rafii and Pardo, 2013] Zafar Rafii and Bryan Pardo. Repeating pattern extraction technique (repet): A simple method for mu- sic/voice separation. IEEE Transactions on Audio, Speech, and Language Processing, 21(1), 2013.","title":"Repeating pattern extraction technique (repet): A simple method for music/voice separation","authors":"Zafar Rafii, Bryan Pardo","year":"2013","venue":"IEEE Transactions on Audio, Speech, and Language Processing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 234-241. Springer, 2015.","title":"U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sandsten, 2016] Maria Sandsten. Time-frequency anal- ysis of time-varying signals and non-stationary pro- cesses. Lund University, Centre for Mathematical Sciences, available online at http://www. maths. lth. se/matstat/kurser/masm26/2016/TIMEFREQkompendie, 2016.","title":"Time-frequency analysis of time-varying signals and non-stationary processes","authors":"Maria Sandsten","year":"2016","venue":null,"doi":null,"url":"http://www.maths.lth.se/matstat/kurser/masm26/2016/TIMEFREQkompendie","internal_url":null},{"position":24,"raw":"[Stoller et al., 2018] Daniel Stoller, Sebastian Ewert, and Simon Dixon. Adversarial semi-supervised audio source separation ap- plied to singing voice extraction. In ICASSP. IEEE, 2018.","title":"Adversarial semi-supervised audio source separation applied to singing voice extraction","authors":"Daniel Stoller, Sebastian Ewert, Simon Dixon","year":"2018","venue":"ICASSP. IEEE","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Subakan and Smaragdis, 2017] Y.Cem Subakan and Paris Smaragdis. Generative adversarial source separation. arXiv preprint arXiv:1710.10779, 2017.","title":"Generative adversarial source separation","authors":"Y, Cem Subakan, Paris Smaragdis","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Uhlich et al., 2015] Stefan Uhlich, Franck Giron, and Yuki Mitsu- fuji. Deep neural network based instrument extraction from music. In Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on, pages 2135-2139. IEEE, 2015.","title":"Deep neural network based instrument extraction from music","authors":"Stefan Uhlich, Franck Giron, Yuki Mitsufuji","year":"2015","venue":"Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Uhlich et al., 2017] Stefan Uhlich, Marcello Porcu, Franck Giron, Michael Enenkl, Thomas Kemp, Naoya Takahashi, and Yuki Mit- sufuji. Improving music source separation based on deep neu- ral networks through data augmentation and network blending. ICASSP, 2017.","title":"Improving music source separation based on deep neural networks through data augmentation and network blending","authors":"Stefan Uhlich, Marcello Porcu, Franck Giron, Michael Enenkl, Thomas Kemp, Naoya Takahashi, Yuki Mitsufuji","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Vincent et al., 2006] E. Vincent, R. Gribonval, and C. Fevotte. Performance measurement in blind audio source separation. IEEE Transactions on Audio Speech and Language Processing, 14(4):1462-1469, 2006.","title":"Performance measurement in blind audio source separation","authors":"E Vincent, R Gribonval, C Fevotte","year":"2006","venue":"IEEE Transactions on Audio Speech and Language Processing","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Vincent et al., 2014] Emmanuel Vincent, Nancy Bertin, Remi Gri- bonval, and Frederic Bimbot. From blind to guided audio source separation: How models and side information can improve the sep- aration of sound. IEEE Signal Processing Magazine, 31(3):107- 115, 2014.","title":"From blind to guided audio source separation: How models and side information can improve the separation of sound","authors":"Emmanuel Vincent, Nancy Bertin, Remi Gribonval, Frederic Bimbot","year":"2014","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Vincent, 2007] Emmanuel Vincent. Complex nonconvex l p norm minimization for underdetermined source separation. In Interna- tional Conference on Independent Component Analysis and Signal Separation, pages 430-437, 2007.","title":"Complex nonconvex l p norm minimization for underdetermined source separation","authors":"Emmanuel Vincent","year":"2007","venue":"International Conference on Independent Component Analysis and Signal Separation","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}